Boosting Adversarial Training with Masked Adaptive Ensemble
Authors: Fudong Lin
Xu Yuan
Nian-Feng Tzeng
Venue: Submitted to ICLR 2023 (2023)
Keywords: Deep Learning Security
Adversarial Example Detection
Adversarial Training

Abstract: Adversarial training (AT) can help improve the robustness of a deep neural network (DNN) against potential adversarial attacks by intentionally injecting adversarial examples into the training data, but this way inevitably incurs standard accuracy degradation to some extent, thereby calling for a trade-off between standard accuracy and robustness. Besides, the prominent AT solutions are vulnerable to sparse attacks, due to “robustness overfitting” upon dense attacks, often adopted by AT to produce a threat model. To tackle such shortcomings, this paper proposes a novel framework, including a detector and a classifier bridged by our newly developed adaptive ensemble. Specifically, a Guided Backpropagation-based detector is designed to sniff adversarial examples, driven by our empirical observation. Meanwhile, a classifier with two encoders is employed for extracting visual representations respectively from clean images and adversarial examples. The adaptive ensemble approach also enables us to mask off a random subset of image patches within input data, eliminating potential adversarial effects when encountering malicious inputs with negligible standard accuracy degradation. As such, our approach enjoys improved robustness, able to withstand both dense and sparse attacks, while maintaining high standard accuracy. Experimental results exhibit that our detector and classifier outperform their state-of-the-art counterparts, in terms of detection accuracy, standard accuracy, and adversarial robustness. For example, on CIFAR-10, our detector achieves the best detection accuracy of 99.6% under dense attacks and of 98.5% under sparse attacks. Our classifier achieves the best standard accuracy of 91.2% and the best robustness against dense attack (or sparse attack) of 57.5% (or 54.8%).

Under review as a conference paper at ICLR 2023
BOOSTING ADVERSARIAL TRAINING WITH MASKED ADAPTIVE ENSEMBLE
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial training (AT) can help improve the robustness of a deep neural net- work (DNN) against potential adversarial attacks by intentionally injecting ad- versarial examples into the training data, but this way inevitably incurs standard accuracy degradation to some extent, thereby calling for trade-off between stan- dard accuracy and robustness. Besides, the prominent AT solutions are vulner- able to sparse attacks, due to "robustness overfitting" upon dense attacks, often adopted by AT to produce a threat model. To tackle such shortcomings, this pa- per proposes novel framework, including detector and classifier bridged by our newly developed adaptive ensemble. Specifically, a Guided Backpropagation- based detector is designed to sniff adversarial examples, driven by empirical observation. Meanwhile, a classifier with two encoders is employed for extract- ing visual representations respectively from clean images and adversarial exam- ples. The adaptive ensemble approach also enables us to mask off a random sub- set of image patches within input data, eliminating potential adversarial effects when encountering malicious inputs with negligible standard accuracy degrada- tion. As such, our approach enjoys improved robustness, able to withstand both dense and sparse attacks, while maintaining high standard accuracy. Experimen- tal results exhibit that our detector and classifier outperform their state-of-the-art counterparts, in terms of detection accuracy, standard accuracy, and adversarial robustness. For example, on CIFAR-10, 10, our detector achieves the best detection accuracy of 99.6% under dense attacks and of 98.5% under sparse attacks. Our classifier achieves the best standard accuracy of 91.22 91.2% and the best robustness against dense attack (or sparse attack) of 57.5% (or 54.8%).
INTRODUCTION
Deep neural networks (DNNs) have been reported to be vulnerable to adversarial attacks. That is, maliciously crafting clean images under a small distance can mislead DNNs into incorrect predic- tions. Such vulnerability prevents DNNs' wide adoption in critical domains, such as healthcare, autonomous driving. finances, among many others. a nutshell, adversarial attacks can be roughly grouped into two categories, i.e. the dense attack and the sparse attack. The former (e.g.. Goodfel- low et al. (2055): Moosavi-Deezoool et al. (2016), 6); Madry et al.) (2018); Croce & Hein| (2020): Yao al. (2021) )) tends perturb almost all pixels on the clean image, whereas the latter (e.g.. Papernot et al. (2016;; Carlini & Wagner (2017); Modas et al. (2019); Dong et al. (2020); PPintor et al. (2021)
Zhu al. (2021) modifies only a limited number of pixels fool the DNN models.
So far, adversarial training (AT) is widely accepted as the most effective method to improve DNNs' robustness against adversarial attacks, by intentionally injecting adversarial examples into the train- ing data. In particular, multi-step ATs Madry et al. (2018): Zhhng et al. (2019); ; Jia et al. (2022) perform multi-step dense attacks (e.g., PGD attack) to find the worst-case adversarial examples for training, achieving state-of-the tf-he-t- robustness but incurring a significant computational overhead. On the other hand, by using the single-step dense attack (e.g., FGSM attack), one-step AT's Wong et al. (2020); Andriushchenko & Flammarion (2020); : Kim et al. (2021); Li et al.) (2022); ; Wang et al. 2022) can significantly reduce the computational overhead while achieving decent robustness under dense attacks. Despite effectiveness, existing ATs suffer from two shortcomings:
trade-
off between standard accuracy (i.e., the accuracy on clean images) and adversarial robustness (i.e., the accuracy adversarial examples), with improved robustness yielding non-negligible standard

---

Under review as a conference paper at ICLR 2023
accuracy degradation and ii) robustness overfitting on dense attacks, making improved robustness vulnerable to sparse attacks.
One promising direction to address the trade-off between standard accuracy and adversarial robust- ness is via a detection/rejection mechanism, that is, training an additional detector to reject malicious Roth input data, with various detection techniques proposed Roth et al.) 22019;; Ma & Liu 2019); Yin et al..
(2220) Raghuram et al.] (202T): Tramer (2022). Unfortunately, the detection/nejection mechanism still ineffective to defend against sparse attacks, as sparse attacks only perturb a limited number of pixels. Even worse, the delection/vejection mechanism can be applied merely to a limited num- ber of scenarios. For example, it cannot be generalized to the application domains where natural adversarial examples exist, as reported in a recent study Hendrycks et al. (2021).
In this work, we consider the robustness under a more general and challenging scenario (than that addressed earlier by the detection/rejeccion mechanism), where the malicious input is not allowed to be rejected. Note that robust model under such a scenario is crucial for applying DNNs to critical domains. For example, an autonomous driving car is expected to recognize road sign even has been maliciously crafted. Our goal is to develop a novel framework, including a detector and a classifier, to boost adversarial training for improving DNN's robustness against both dense and sparse attacks at a small expense of standard accuracy degradation. Specifically, our frame- work is adversarially trained by using one-step least- likst-lieely adversarial training, adopted from Fast Adversarial Training WWogg et al. (2020) with slight modification (see Section A.2 in Appendix for details). We incorporate two new designs in our detector to make adversarial examples more notice- able. First, we resort to Guided Backpropagation Springenberg et al. (2015) to expose adversarial perturbations, driven by our empirical observations. Second, the Soft- Nearest Neighbors Loss (SNN Loss) Salakhutdiny & Hinton ): Frosst et al. (2019) is tailored to push adversarial examples away from their corresponding clean images. As such, our detector effective in sniffing both dense attack-genente geteea-generated and sparse attack- generated adversarial examples.
Our classifier includes two encoders for extracting visual representations respectively from clean images and adversarial examples, aiming to alleviate the negative effect of adversarial training on standard accuracy. We separate the training process into "pre-training" and "fine-tuning" for rep- resentation learning and classífication, respectively. In the pre-training, high-quality representations and encourage pairwise similarity between clean image and its adver- sarial example. Specifically, extend Masked Autoencoders (MAE) He al. (002)). , i.e., learning visual representations by reconstructing the masked images, for adversarial training via a new de- sign. That is, we reconstruct images from pair of masked clean image and masked adversarial example, for representation learning, with a contrastive loss on visual representations to encourage pair similarity. In the fine-tuning of classification, we freeze the weights on the two encoders and fine-tune an MLP (Multi-layer Perceptron) for accurate classification by using our proposed adap- tive ensemble to bridge the detector and the classifier. Meanwhile, our adaptive ensemble allows us to mask off an arbitrary subset of image patches within the input, enabling our approach to mitigate potential adversarial effects when encountering malicious inputs with negligible standard accuracy degradation. Extensive experiments have been carried out on three popular benchmarks, with the re- sults demonstrating that our solutions outperform tftt--ffthhe-art detection and adversarial training techniques in terms of detection accuracy, standard accuracy, and robustness.
goal is to jointly learn
2 RELATED WORK
Our work closely relates to two research scopes, i.e., detection/rejection mechanisms and adversarial training approaches. This section discusses how our work relates to, and differs from, prior studies.
Detecting adversarial examples (AEs) and then rejecting them (i.e., detec- tion/rejection mechanism) can improve the model robustness. That is, the input will be rejected the detector classifies it as an adversarial example. Popular detection techniques include Odds Roth ct al. (2019), which considers the difference between clean images and AEs in terms of log-odds; NIC & Liu (2019). which checks channel invariants within DNNs; GA et al. (2020), which resorts to multiple binary classifiers; JTLA Raghuram et al. 2021). which proposes a detection framework by employing internal layer representations, among many others Lee et al. (2018); Yang ct al. (2020); Sheikholeslamm et al. (202I). Unfortunately, existing detection methods are typically ineffective in sniffing sparse attack aeteck-ee AES, which just modify limited numbers of pixels.
Detection Mechanisms.
2

---

Under review as a conference paper at ICLR 2023
Besides, the detection/rejection mechanism only works in limited scenarios. For example, it cannot be generalized to domains where natural adversarial examples exist. Differently, our work resorts to the Guided-Backe Backpropogation technique, which can largely expose adversarial perturbations, based on our empirical observation. Then, we adopt the Soft-Nearest Neighbors (SNN) loss, which can further maximize differences between clean images and adversarial examples.
Adversarial Training Approaches. Adversarial training (AT) aims to improve the model robust- ness by intentionally injecting adversarial examples into the training data. For example, PGD- AT Madry et al.] (2018) proposes multi-step attack to find the worst case of training data, TRADES AahannnZaaggga et al. (2019) addresses the limitation of PGD-AT by utilizing theoretically sound cuasification-calibre
loss, EAT Tramer et al. (2018) uses an ensemble of different DNNs to pro- duce the threat model, FAT Wong et al. (2020) reduces the computational overhead of AT by utilizing FGSM attack with the random initialization LAS-AWP Jia et al. (2022) boosts AT with a learnable attack strategy. Sub-AT Li et al. (2022) constrains AT in a well-nesigned subspace, and many oth- Shafahi al. 2019); Andrrushchenko & Flammarion 2020); Kim et al. (202T); [Wang et al. (222)) However, prior AT's suffer from the dilemma of balancing the trade-off between standard accuracy and adversarial robustness. Besides, their improved robustness is vulnerable to sparse at- tacks. Although we adopt the threat model in FAT, by using the proposed adaptive ensemble, our method can be generalized to defend against sparse attacks. Meanwhile, it mitigates the standard accuracy degradation by employing two encoders for extracting visual representations respectively from clean images and adversarial examples.
3 OUR APPROACH
3.1 PROBLEM STATEMENT
We consider a set of N samples, i.e., x= {(x,, y4) | E 21. where x E #x is the input image and y E [C] denotes its label. Here, (H, W) represents the resolution of input images and C' is the number channels. For notational convenience, we let H x W x C. A classifier is function fo: Rd - [C), parameterized by a neural network. In this paper, we consider two types of inputs, i..., the clean image x sampled from the standard distribution Dat and the adversarial example ay sampled from the adversarial distribution Dav We assume Dad and Daay follow different distributions. The clean image cl itself or its augmented variant can be the input, while the adversarial example gu is a malicious version of æ within a small distance, that is, for some metric d, we have dlx x²)h)= < e, but "dy can mislead conventional classifiers. Parameterized by another neural network, a detector I to tell whether an input image is clean image or not, i.e., 96 : Rd -> {+1), where +1 and 1 indicate the clean image and the adversarial example, respectively. The binary indicator function (1.) is 1 if both the detector Js and the classifier fo make correct predictions.
this paper, we follow previous studies Madry et al. (2018); /Zhang et al. (2019) by referring standard accuracy Asii and adversarial robustness Aniv, as classification accuracy on clean images and on adversarial examples, respectively.
We start by defining standard accuracy. Given clean image, the incorrect prediction made either by the detector or by the classifier is counted as an error.
Definition 3.1 (Standard accuracy). Let f o g be a model with a classifier fe Rd + [C] and a detector 9s Rd + {=1]. Its standard accuracy is defined by the expected rate at which both the detector and the classifier make correct predictions on the clean image:
Asa (f oogg g) ==
E [anth (D))-DDi
[L(o0((r)))=1 A (o())))==)]
(I)
Similar to standard accuracy, adversarial robustness regards the incorrect prediction on an adversar- ial example made either by the detector or by the classifier as an error.
Definition 3.2 (Adversarial robustness). Let f fog be a model, including a detector gø : Rd + (+1}
and classifier sa IRd + [C]. Given the input (x,, y). the adversarial robustness at small distance i.e., d(x x: ad) < is defined as:
A S of
E. (anddr, Diai
[1 (aptl)))) fo((tt))=))) mny]
3

---

Under review as a conference paper at ICLR 2023
(a) Clean image "acoustic guitar"
(b) Guided Grad-CAM "acoustic guitar"
(c) Guided Backprop
"acoustic guitar"
(d) Adversarial example (e) Guided Grad-CAM "maze"
(I) Guided Backprop
"maze"
"maze"
Figure 1: Different visualizations on the clean image (Top) and the adversarial example (Bottom). Left: Original clean image and adversarial example with their predicted labels. Middle: Guided Grad-CAM visualization. Right: Guided Backpropagation visualization.
3.2 DETECTION
Parameterized by a neural network with parameters Φ,, the detector Os : IRd + {+1} whether the input is clean image not, where +1 and -1 indicate the clean image and the adversarial example, respectively. Mathematically.
} is to determine
+1, if a is a clean image (xxx)
(3)
-1.
otherwise.
Aiming to generalize the robust model to critical domains (e.g., autonomous driving), the input will not be rejected in this study. That is, the detector also outputs the estimated probability of p E [0, I] for the clean image and - p for the adversarial example
The design of our detector architecture is motivated by our empirical observation in that the ad- versarial perturbation is detectable after Guided Backpropagation visualization. Due to the small distance between clean image and its corresponding adversarial example, their difference is no- toriously imperceptible (see Figures Ta and ld), making it theoretically hard to detect adversarial examples Tramer (202). In empirical study, resort to Guided Backpropagation berg et al. (2015) to visualize the difference between the clean image and the adversarial example. Surprisingly y, we discovered that after Guided Backpropagation visualization on the adversarial ample, its adversarial perturbation is quite noticeable; See Figure Tc versus Figure Ti. i.e., visualiza- tion on the clean image versus on the adversarial example. Notably, our experiments also include the visualization comparison under Guided Grad-C.a Sraad-CAMSS3 Selvaraju et al. (2017). developed recently; see Figure [b versus Figure Tel However, Guided Grad-CAM exhibits inferior performance (compared to Guided Backpropagation) in terms of exposing adversarial perturbation. This empirical study motivates us to maximize the difference between clean images and adversarial examples by using Guided Backpropagation visualization.
(SprinageinnSSingenn... (Springen-
Figure illustrates our detector architecture. Given an input image æ Rd, we perform Guided Backpropagation on the original image, arriving at input variant x' E Rd Note that we employ the label predicted by a pre-trained model to be the target concept for Guided Backpropagation; hence, cround-trutie label is required during the detection. Following the standard Vision Trans- former (ViT) Dosovitskiy et al. (2021). we patchify the two inputs into two sets of image patches
Notably, approach is readily generalizable to the detection/rejectionn mechanism. We reject the input data if the detector identifies malicious input, i.e., d (x(x) -1.
4

---

Under review as a conference paper at ICLR 2023
Output
Exdgep
Ereader
MT
Verkent
Figure 2: Our detector architecture.
and embed them via linear projection, arriving at sets of patch embeddings, iee, E, E MXD and E,, E RMD respectively for the original input and its input variant. Here, M represents the number of patches and D indicates the hidden dimension. Driven by the above empirical obser- vation, we add two sets of patch embeddings together, expecting that the adversarial perturbation exposed by Guided Backpropagation can help differentiate adversarial examples from clean images. Similar to the standard ViT. we fill the position embeddings Epos RR x for remaining positional information. Hence, we have the input sequence E of the detection encoder (i.e., Transformer Encoder) as follows,
E [ + E Ep,, + E Epps + E E Eposy
(4)
where Ep, (or E denotes the i-th patch embedding in the original input (or its input variant produced by Guided Backpropagation). Following the ViT architecture in Masked Autoencoders (MAE) He et al. (2022), we perform the global average pooling on the full set of encoded patch embeddings, with the result fed into an MIP (i.e., multiple-layer perceptron) for telling whether the input is clean image or not.
Aiming further differentiate adversarial examples from clean images, propose novel loss function to train our detector, including a Cross-Entrope (CE) Loss Cee and a Soft-Nearest Neigh- bors (SNN) loss Lann Salakhutdinov & Hinton (007): Frosst et al. (2019), for jointly penalizing the detection error and the similarity level between the clean image and the adversarial example, i.e.,
(5) where (0. 1) is hyperparameter control the penalty degree of the two terms, and eln and adv denote the global representations, i.e., the global average pooling on the encoded representa- tions, for the clean image and the adversarial example, respectively.
Laet = (1 - \) (g (x), gdet) + (= 2ddv),
The SNN loss is variant of contrastive loss, allowing for the inclusion of multiple positive pairs. We regard members belonging to the same determined class (e.g., two clean images) as positive pairs, while members belonging to different determined classes (e.g., clean image and an adversarial ex- ample) as negative pairs. We consider a mini-batch of 2B samples, with one half being clean images, i.e., { (2) yt = and the other half of adversarial examples, i.e., {( dett= 1) R- the SNN loss at temperature is defined below:
B+1
2B ΣitinA j=1,..... 2B exp(-sim/ 2;, 22)/T)
Cynn
log
(6)
2B
Exph,y=1,...22 2B exp(-sim(2,) zx)/T)
where is the visual representations for the input D; and the similarity metric sim(:,) is measured by the cosine distance. The SNN loss enforces each point to be closer to its positive pairs than to its negative pairs. In other words, the SNN loss penalizes the similarity level between clean images and adversarial examples, making adversarial examples more discernible by our detector.
3.3 CLASSIFICATION
Inspired by self- self-supervised pre-training for vision tasks Chen al. (2020); Bao al. (2022) He ct al. (2022). we separate our adversarial training into two stages, f.e., pre-training and fine-tuning, for learning high-quality visual representations and fine-tuning robust classifier, respectively.
5

---

Under review as a conference paper at ICLR 2023
Doee
Haraed
Hadee
(a) Pre-training.
(b) Fine-tuning.
Figure 3: Our classifier architecture: (a) pre-training and (b) fine-tuning.
Pre-training. Our architecture for the pre-training is inspired by MAE He et al. (2022). Different from MAE, however, we utilize two encoders, denoted as the clean encoder and the adversarial en- coder, for learning visual representations from clean images and adversarial examples, respectively. The decoder aims to reconstruct the original inputs from the visual representations encoded by the two encoders. Figure Shoss the architecture of our pre-training. Given an input image æ Rd let z=ln and vddv denote its clean and adversarial variants, respectively, with the clean variant ob- tained by augmenting the original input. Regarding the clean variant aeen, we randomly mask out large proportion of image patches (e.g.. 75%) and then feed the subset of visible patches into the clean encoder. The masked tokens are inserted into corresponding positions after the encoder. Finally, the decoder reconstructs the clean variant geln from the full set of image patches, includ- ing encoded visible patches and masked tokens. The reconstruction of the adversarial variant aiily follows similar procedure, except that its visible patches are encoded by the adversarial encoder. Notably, the position of masked image patches in the adversarial variant ndv is the same as that in the clean variant ch in order minimize their visual representation difference during pre-training.
Denote geln and gadv respectively to be the global representations of the clean and adversarial vari- ants, ie., the global average pooling the input sequence for the decoder. Different from MAE, our design utilizes new loss function to learn visual representations by simultancously minimizing the reconstruction error and the visual representation difference, i.e..
Lene (1- free (x, +)+++ + La(2""n
where Ω E 10 is hyperparameter and æ is the reconstructed image. Leee and La denote the reconstruction loss and the contrastive loss, respectively. Given a set of input images, we first generate their adversarial variants, arriving at mini-batch of 2B samples, consisting of B clean variants {aedla B
]==] and adversarial variants {æ:"dv J2B B+1 We consider the form of contrastive loss in SimCLR Chen al. (2020), and define our contrastive loss at temperature follows:
J
Chen
exp(sim( 2T)/T)
(ii,j) = - -log
Lihh 1.....21....2 2B exp(sim ZZ 2x)/T)²
(8)
B
Ld
10(k, k B) + B,A)],
2B
where Σ; denotes visual representations for 2ith (or x:ddv) and the similarity level sim(-.) -) is mea- sured by the cosine distance. particular, we regard the clean and adversarial variants from the same input as the positive pairs, while the rest in the same batch are negative pairs. Therefore, our contrastive loss decreases when visual representations for the clean and the adversarial variants of the same input become more similar.
Fine-tuning. Figure our architecture during fine-tuning, which only keeps two tre-trained decoders with frozen weights for producing visual representations during adversarial training, and the decoders are dropped after pre-training. Different from MAE, which encodes the full set of image patches during fine-tuning, our approach randomly masks out a relatively small proportion of image patches (e.g., 45%), aiming to eliminate the potential adversarial effect if the input is an adversarial example.
Given an input image (a, ) where x E Rd is either a clean image or an adversarial example with the label yola E [C), we randomly mask the input image twice, arriving different masked

---

Under review as a conference paper at ICLR 2023
inputs. Two subsets of visible patches from the two masked imputs are fed into the clean and the adversarial encoders, respectively. The masked tokens are introduced onto their Corresponding po- sitions after the decoder, obtaining two full sets of visual representations, i.e., 2 and gndv which are partially encoded by the clean and the adversarial encoders, respectively. We then perform the global average pooling the adaptive ensemble of geln and zady, with the result fed into an MLP for classification.
Adaptive Ensemble. Although randomly masking an input image can eliminate the potential ad- versarial effect, this way inevitably hurts standard accuracy during fine-tuning. In this paper, we propose adaptive ensemble tackle this issue. That is. the global representation for an input image is derived from the sum of g and g with an adaptive factor p E [0, 1J, where z and gndv are visual representations encoded by the clean and the adversarial encoders, respectively, and p the probability of the input image being clean image estimated by our detector.
Let A be full set of image patches and V be a subset of A, including visible patches only. (v(-) is the indicator function for evaluating whether image patch is visible. Hence, for every image patch of A, we have,
1, if the i-th patch is visible for i 1,2,....
1 (1)
0, otherwise
where M is the number of image patches, i.e., |.]. For notational convenience, we let 1 in- dicate the visible patches fed into the clean encoder. Likewise, 1 1/th indicates the visible patches fed into the adversarial encoder. Let 2; be the visual representation of the i-th image patch, with i (1,2...,,M) M). Then, our adaptive ensemble is defined by:
p- (1) (i) Lgin + (1 - p) 1 (i) 2udV
(10)
max (p- 1sth
p) adv (i), c)
where the denominator serves to normalize the adaptive ensemble of 2 and 2nd and is small value to avoid divison by zero (i.e.. < 1e - 12 in this paper). The intuition underlying Eq. (10) is that if our detector has high confidence that the input is a clean image (i.e., p is large), the global representation & will be mostly encoded by the clean encoder. Otherwise, &; will be mainly encoded by the adversarial encoder. In addition, our pre-training encourages the similarity level of the clean and the adversarial variants from a given input (see Eq. 7 and Eq. (8). and two
different masked inputs exist upon fine-tuning, the invisible image patches in one masked input can be glimpsed from the other masked input.
EXPERIMENTS AND RESULTS
4.1 EXPERIMENTAL SETUP
Datasets. We conduct experiments three widely-used benchmarks. (i) CIFAR-10 Krizhevsky et al. (2009): 60, 000 32x32 RGB images of 10 classes. (ii) CIFAR-100 KtFARevooKKiihevskky Krizhevsky et al. (2009):
60, 000 32x32 RGB examples in 100 categories. (iii) Tiny-LmageNet Deng et al. (209); 120, 000 64x64 RGB images of 200 classes.
Compared Methods. We compare our approach with four detection approaches, i.e., Odds Roth et al. (2019), NIC Ma & Liu (211)) GAT Yin et al. (2020). and JTLA Raghuram et al. (2021).
Meanwhile, to exhibit how our approach boosts adversarial training (AT), we compare our method
with six AT counterparts: PGD-AT Madry et al. (2018). 8). TRADES Zhang et al. (2019), FAT Wong jet al. (2020), EAT Tramer et al. (218)), Sub-AT Li et al. 2022). and LAS-AWP P Jia et al. (2022). Hyperparameters for the baselines, if not specified, are set reported in their original literature.
Evaluation. We consider both dense attacks and sparse attacks for evaluating detection accuracy and adversarial robustness. In particular, we utilize (i) four dense attacks, i.e., FGSM Goodfellow et al. (2015). PGD Madry et al. (2018), DeepFool Deosavvi et al. (2016), and AutoAt- tack Croce & Hein (2020), and (ii) three sparse attacks, i.e., C& Lo Carlini & Wagner (017)) FoorseFool et al. (2019), and FMN Pintor et al. (202T). For notational convenience, we Tet PGD-20/50 denote the PGD attack with 20 or 50 steps.
7

---

Under review as a conference paper at ICLR 2023
Parameter Settings, Inspired by the recent success of Vision Transformer (ViT) Dosovitskiy al. (022), we use the ViT as the backbone network for our detector and classifier, with their ViT architectures respectively following DeiT Touvron et al. (202I) and MAE He et al. (2022). But we prune our model size as small as possible in order to conduct a fair comparison with baselines. Due to the page limit, defer the details of the model size and hyperparameters to Section A.1 in Appendix.
4.2
OVERALL PERFORMANCE
Table 1: Overall comparisons on CIFAR-IO, with the best results shown in bold
Method
Standard Accurey
Sparse
FGSM
PGD-20
DeepFool
AutoAttack
C&W Lo
SparseFool
PGD-AT TRADES
82.3
48.4
45.6
46.2
41.2
16.5
10.3
12.5
84.7
52.5
46.2
42.1
FAT
84.9
$1.9
45.9
48.6
40.2
10.4
13.3
12.7
83.5
52.8
50.1
47.9
47.1
Sub-AT
80.5
52.3
51.0
49.6
48.1
19.1
16.2
11.3
LAS-AWP
85.6
57.1
56.3
53.9
51.3
20.5
13.8
16.1
Ours
91.2
$7.5
56.6
54.6
49.4
54.8
52.5
51.4
Comparisons to Baselines. We first conduct extensive experiments on CIFAR-10 10 and compare our approach state-of-the-art adversarial training (AT) counterparts listed in Section [4.] in terms of standard accuracy, robustness against dense attacks, and robustness against sparse attacks. Table [] lists comparative results. It is observed that our approach achieves the best performance under all three scenarios. In particular, our approach achieves the standard accuracy of 91.2%, outperforming the best state-of-the-an (i.e., LAS-AWP) by 5.6%. This is contributed by employing two encoders to extract visual representations respectively from clean images and adversarial examples, able to significantly mitigate the adverse effect of adversarial training on standard accuracy. Besides, in terms of robustness against dense attacks, our approach outperforms its all competitors under every scenario except LAS-AWP on robustness against AutoAttack. This is because LAS-AWP employs an automatic attack strategy to produce the threat model, similar the attack strategy of Auto.Attack It should be noted that our approach significantly outperforms all compared counterparts under all sparse attacks, unlike prior ATs that tend to suffer from robustness overfitting upon dense attacks. Take LAS-AWP an example, its robustness degrades from 57. 17. under dense attack (i.e., FGSM attack) to 20.5% under sparse attack (i.e., C& Lo attack). By contrast, our approach exhibits only 2.7% robustness degradation under the same case, i.e., 57.5% under FGSM attack versus 54.8% under C&W Lo attack. This evidences that our proposed masked adaptive ensemble can boost adversarial training by generalizing robustness improvement to defend against sparse attacks.
More experiments for overall comparisons on CIFAR- and Timy-ImageNet are also conducted, with the results deferred to Section A.3 in Appendix to conserve space.
SabiT
Sab-T
20
10
2.0
240
200
30
CIAR-33
CIFAS-IE
Tay--aageei
CIFA--1
CIFAS-IBE
Tay Traghin
CIFAP-1
GIFAT-10E
Dilkkee
Robustness against PGD-50 Robustness against C&W Lo Figure 4: Illustration of the performance stability under different scales of datasets and different types of attacks.
(a) Standard Accuracy
Performance Stability. We next conduct experiments on CIFAR-10, CIFAR-100, and Tiny- ImageNet to evaluate the performance stability under different scales of datasets and different types of attacks, We compare approach with three baselines, i.e., EAT, Sub-AT, and LAS-AWP, Fig- illustrate the comparative results of standard accuracy, robustness against dense attack (i.e., PGD-50), and robustness against sparse attack (i.e., C&W Lo), respectively. We have three discoveries. First, as depicted in Figure 44l our approach suffers from the least standard accu- racy degradation of 35.5% (with standard accuracy ranging from 91. 11.2% CIFAR-10 to 55.7% on
ures 4a 4b and 4c
8

---

Under review as a conference paper at ICLR 2023
Tiny ImageNet), outperforming EAT (i.e., 39.7% degradation), Sub-AT (i.e., 44.0% degradation), and LAS-AWP (i.e. 41.0% degradation). Second, under the strong dense attack (i.e., PGD-50), approach and LAS-AWP achieve similar robustmess CIFAR-I0, but our method outperforms LAS-AWP by 2.9% on CIFAR-100 and by 9.7% on Imay- as illustrated in Figure 4b) Third, as shown in Figure 4c] all competitors suffer from poor robustness (S 20.5%) under the C&W Lo attack, where our approach still exhibits decent performance, with its robustness equaling 54.8%, 36.8%, and 28.9% on CIFAR-IO, CIFAR-I00, and Tiny-ImageNet, respectively. These re- sults validate that our approach is more stable when upscaling to large datasets and when defending against different types of attacks.
4.3 EVALUATING OUR DETECTOR
Table 2: Comparisons of detection accuracy on CIFAR- 10 under dense and sparse attacks
Dense Attack
Sparse Attack
Methed
FGSM
PGD-20
DeepFool
AutoAttack
C&W Lo
SparseFool
FMN
Odds
96.9
93.4
90.1
90.6
72.5
69.2
68.5
NIC
96.8
97.2
91.9
92.1
74.3
76.5
72.8
GAT
95.4
92.3
92.4
JTI.A
97.5
95.1
93.1
93.5
77.7
74.9
69.5
Ours
99.6
99.1
97.9
98.4
96.4
98.5
97.1
In this section, we conduct experiments on CIFAR-10. 10, and compare our detector with four detec- tion baselines, i.e., Odds, NIC, GAT, and JTLA. Table 2 lists the detection accuracy values under both dense and sparse attacks. We observed that our detector always achieves superior detection performance under all scenarios, i.e., more than 97. 97.1%. outperforming all competitors. Specifically, our approach achieves the best detection accuracy of 99.6% under the dense attack (i.e., FGSM) and of 98.5% under the sparse attack (i.e., SparseFool). Besides, state-of-the ineffective to sniff sparse attack-generated adversarial examples, with much inferior detection ac- curacy. For example, their best detection accuracy under sparse attacks is only 77.7%, i.e.. JTLA against C&W Lo. In sharp contrast, our approach under C& W Lo yields the detection accuracy of 96.4%, drastically outperforming the best counterpart JTLA by 18.7%. The statistical evidence exhibits that our two new designs for the detector, iee, our propose loss (i.e., (5 and the Guided Backpropagation- based input variant, are effective for exposing adversarial perturbation, boosting the detector to far better defend against sparse attacks.
counterparts
Due to the page limit, we defer ablation studies on our classifier (including the impacts of different masking ratios and of our adaptive ensemble) and on our detector (including the effects of Guided Backpropagation and of the SNN loss) to Section A.4 and Section in the Appendix, respectively.
CONCLUSION
This article has proposed novel framework, including detector and classifier, defend against adversarial attacks. With our newly developed adaptive ensemble to bridge the detector and the clas- sifier, our approach can boost adversarial training to defend against both dense and sparse attacks, and can also achieve a better trade-off' between standard accuracy and robustness. Our key idea includes applying the Guided Backpproagation expose adversarial perturbations for better detec- tion and employing two decoders to extract visual representations respectively for clean images and adversarial examples so as to reduce the negative effect of adversarial training on standard aceu- racy. Meanwhile, our adaptive ensemble allows to eliminate potential adversarial effects when encountering adversarial examples by masking out a random subset of image patches across input data. Extensive experiments have been conducted for evaluation, with results demonstrating that solutions significantly outperform their state-of-the-at counterparts in terms detection accuracy, standard accuracy, and robustness.
Since our approach lies in one-step adversarial training. which typically suffers from "catastrophic overfitting" Wong et al. (2020), where improved robustness may suddenly drop to 0% under the strong PGD attack. We defer the extension of our approach for addressing this possible "catastrophic overfitting" to future work.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. In NeurlPS, 2020.
Hangbo Bao, Li Dong, and Furu Wei. BEiT: BERT pre-training of image transformers. In ICLR, 2022.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In Symposium on Security and Privacy (SP), 2017.
Ting Chen, Simon Korbllihh, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. In ICML, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Xiaoyi Dong, Dongdong Chen, Jianmin Bao, Chuan Qin, Lu Yuan, Weiming Zhang, Nenghai Yu, and Dong Chen. Greedyfool: Distortion-aware sparse adversarial attack. In NeurlPS, 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
Nicholas Frosst, Nicolas Papernot, and Geoffrey E. Hinton. Analyzing and improving representa- tions with the soft nearest neighbor loss. In ICML, 2019.
Ian J. Goodfellow. Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, An- drew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint uwXiv:1706.02677, 2017.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, 2022.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In CVPR, 2021.
Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, and Xiaochun Cao. Las-at: Adversarial training with learnable attack strategy. In CVPR, 2022.
Hoki Kim, Woojin Lee, and Jaewook Lee. Understanding catastrophic overfitting in single-step adversarial training. In AAAI, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In ICLR, 2017.
Kimin Lec, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS, 2018
Tao Li, Yingwen Wu, Sizhe Chen, Kun Fang, and Xiaolin Huang. Subspace adversarial training. In CVPR, 2022.
Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In ICLR, 2017.
10

---

Under review as a conference paper at ICLR 2023
Iyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019.
Shiqing Ma and Yingqi Liu. Nic: Detecting adversarial samples with neural network invariant checking. In Network and Distributed System Security Symposium (NDSS), 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In JCLR, 2018.
Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Sparsefool: few pixels make a big difference. In CVPR, 2019.
Seyed-Mohsen Moosavi-Dezfoole Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In CVPR, 2016.
Nicolas Papernot, Patrick D. McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Anan- thram Swami. The limitations of deep learning in adversarial settings. In European Symposium on Security and Privacy (FuroS&P) 2016.
Maura Pintor, Fabio Roli, Wieland Brendel, and Battista Biggio. Fast minimum-norm adversarial attacks through adaptive norm constraints. In NeurIPS, 2021.
Jayaram Raghuram, Varun Chandrasekaran, Somesh Jha, and Suman Banerjee. A general frame- work for detecting anomalous inputs to DNN classifiers. In ICML, 2021.
Kevin Roth, Yannic Kilcher, and Thomas Hofmann. The odds are odd: A statistical test for detecting adversarial examples. In ICML, 2019.
Ruslan Salakhutdinov and Geoffrey E. Hinton. Learning nonlinear embedding by preserving class neighbourhood structure. AISTATS, 2007.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In ICCV, 2017.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John P. Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In NeurIPS, 2019.
Fatemeh Sheikholeslami, Ali Lotfi, and J. Zico Kolter. Provably robust classification of adversarial examples with detection. In ICLR, 2021.
Jost Tobias Springenberg. Alexey Dosovitskiy, Thomas Brox, and Martin A. Riedmiller. Striving for simplicity: The all convolutional net. In /CLR Workshop, 2015.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In ICML, 2021.
Florian Tramèr. Detecting adversarial examples is (nearly) as hard as classifying them. In ICML, 2022.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian J. Goodfellow, Dan Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research (JMLR), 2008.
Haotao Wang, Aston Zhang, Shuai Zheng, Xingjian Shi, Mu Li, and Zhangyang Wang. Removing batch normalization boosts adversarial training. In ICML, 2022.
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In ICLR, 2020.
Puyudi Yang, Jianbo Chen, Cho-Jui Hsich, Jane-Ling Wang, and Michael I. Jordan. ML-LOO: detecting adversarial examples with feature attribution. In AAAI, 2020.

---

Under review as a conference paper at ICLR 2023
Chengyuan Yao, Pavol Bielik, Petar Tsankov, and Martin T. Vechev. Automated discovery of adap- tive attacks on adversarial defenses. In NeurIPS, 2021.
Xuwang Yin, Soheil Kolouri, and Gustavo K. Rohde. GAT: generative adversarial training for adversarial example detection and robust classification. In ICLR, 2020.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference (BMVC), 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019.
Mingkang Zhu, Tianlong Chen, and Zhangyang Wang. Sparse and imperceptible adversarial attack via a homotopy algorithm. In ICML, 2021.
A
APPENDIX
A.1
MODEL SIZE AND HYPERPARAMETER DETAILS
Table 3: Model Details used in our design
Model
Layer
Hidden Size
Head
MLP Size
Params
Detector
DeiT-Tiny
12
192
3
768
5.4M
Classifier
Enceder
12
384
6
1536
21.4M
Decoder
192
3
768
1.8M
Model Size. To conduct a fair comparison with existing studies, we develop our detector and clas- sifier small as possible, with Table 3 listing the model size details. Our architecture includes detector and a classifier with two encoders and one decoder, with 50M parameters in total. This similar to the model size of Wide ResNet Zagoruyko & Komodakis (2016) employed by most previous studies. For example, the Wide ResNet adopted by TRADES Zhang et al. (2019) contains 48.3M parameters.
Hyperparameters. For all our models, not specified, we use Adam W Loshchilov & Hutter (2019) with 31=0.9, $,=0.999, the weight decay of 0.05, and batch size of 1024. We follow the setting in Goyal et al. (2017) to train our detector for 100 epochs, with the base learning rate of le - 3, the linear warmup epochs of 5, and the cosine decay schedule Loshchilov classifier, we pre-train it for 400 epochs, with the base learning rate of le - 4, the linear warmup of 40 epochs, and masking ratio of 75%. After pre-training, we drop the decoder and freeze the weights on the two encoders. Then, we finetune the classifier for 100 epochs, with the base learning rate le - 3, the linear warmup of 5, and the cosine decay schedule, and a masking ratio of 45%. We grid-search À in
Hutter (017)). For our
The patch size is set to 2 (or 4) for CIFAR- 1ER---O//FIARRR 100 (or Tiny-ImageNoo (5) and Q in Eq 7 ) and empirically set 0.15 and to 0.35 for all datasets.
A.2 ONE-STEP LEAST-LIKELY ADVERSARIAL. TRAINING
This section supports the main paper by presenting the technical detail of one-step least-likely ad- versarial training.
Adversarial training (AT) improves the model's 's robustness against adversarial attacks by intention- ally feeding adversarial examples into the training set. Given model f with parameters 0, a dataset with samples, i.e., X {( 9:) i 21, .,,,,..., N}}, the cross-entropy loss function L, and threat model A, AT aims to solve the following inner- maximmzaiiii problem and outer minimizaiiooo problem:
min 0
max
C(fo +), y.). SEA
(11)
where the inner problem aims to find the worst-case training data for the given model, and the outer problem aims improve the model's performance such data. Recently, one-step Fast Adversarial
12

---

Under review as a conference paper at ICLR 2023
Table 4: Overall comparisons on CIFAR-100, with the best results shown in hold
Standard Accuracy
Dense Attack
Sparse Attack
Method
FGSM
PGD-20
DeepFool
AutoArtack
C&W Lo
SparseFool
FMN
PGD-AT TRADES
54.6
35.3
34.7
31.9
27.5
12.3
7.4
11.3
56.8
35.6
32.2
28.9
27.8
11.5
9.3
10.6
FAT
53.1
34.3
34,3
31.8
28.7
EAT
56.6
37.5
32.1
30.1
28.9
12.8
11.5
7.8
LAS-AWP
62.8
38.1
35.4
36.3
29.7
15.6
11.5
11.3
Ours.
69.6
40.4
38.3
38.6
37,5
36.8
35,9
34.3
Table 5: Overall comparisons Tiny-ImageNet, with the best results shown in bold
Standard Accuracy
Sparse Attack
Method
FGSM
PGD-20
DeepFool
AutuAttack
Lg
Spursefooll
FMN
PGD-AT TRADES
41.7
17.5
15.3
14.9
10.2
6.9
9.5
8.4
36.6
18.1
15.9
13.8
12.7
7.7
8.3
6.6
FAT
42.9
19.1
15.6
14.1
12.5
7.8
6.2
5,4
EAT
11.7
Sub-AT
36.5
20.3
18.2
13.2
11.1
8.7
9.3
6.2
LAS-AWP
44.6
26.5
22.5
18.3
17.5
Ours
55.7
34.2
32.2
31.7
29.6
28.9
26.8
26.1
Training (FAT) Wong et al. (020 is popular due its computational efficiency. FAT sets the threat model under a small and Too constraint c, i.e. = {6 ||0]|c S <). by performing Fast Gradient Sign Method (FGSM) Goodfellow et al. (2015) with the random initialization, i.e.,
8 = Uniform -c, + e sign( ((fo(x)), 3t)).
(12)
8 max (6, -c),
where Uniform denotes the uniform distribution and sign the sign function. Notably, the second row in Eq. (2) serves to project the perturbation ô back into the loc ball around the data æ,,
To find the worst-case adversarial examples, we extend FAT by performing the least-likely targeted attacks, inspired by prior studies Kurakin et al. (2017); Tramer et al. (2018). That is, given an input F;, we perform targeted FGSM by setting the targeted label as its least-likely class, i.e., v.¹ = arg min fx:;;, arriving at,
8 Uniform( EE, E) + sign Vo L(fo(xx)) v!)).
(13)
= max(minf 6, c),-)), -c),
Our one-step least-likely adversarial training is to utilize Eq. 13) to produce the threat model.
A.3 OVERALL COMPARSIONS ON CIFAR- 100 AND TINY-IMAGENET
This section supports the main paper by comparing our approach with adversarial training (AT) counterparts CIFAR-100 and Imay-em
Table and Table list the experimental results on CIFAR-100 and Tiny-ImageNet, respectively. From Table 4 discovered that our approach achieves the best performance terms of standard accuracy, robustness against dense attacks, and robustness against sparse attacks. In particular, our approach achieves the standard accuracy of 69.6%, outperforming the best competitor (i.e., LAS- AWP) by 6.8%. Besides, our approach achieves the best robustness of 40.4% under the dense attack (i.e.. FGSM) and of 36.8% under the sparse attack (i.e., C&W Ly). Moreover, existing ATs achieve very poor robustness against sparse attacks (less than 15.8)), while our approach, even in the worst case (i.e., under FMN attack), still maintains decent robustness of 34.3%.
We have three observations from Table 5 First, our approach outperforms all counterparts under all three scenarios on Tiny-ImaeeNet. particular, our approach achieves the standard accuracy of 55.7%, outperforming the best competitor (i.e., LAS-AWP) by 11.1%. Besides, compared to LAS- AWP under robustness against dense attacks, our method achieves the performance improvements ranging from 7.7% (under FGSM attack) 13.4% (under DeepFool attack). Third, approach still achieves decent robustness under sparse attacks (e.g., 28.9% under C&W Lp attack), while all competitors perform very poor when encountering sparse attacks (i.e., less than 10%).
13

---

Under review as a conference paper at ICLR 2023
Figure 5: Comparison of the reconstruction quality from masked adversarial examples by employing our approach with/without the contrastive loss, denoted our approach (w/ CL) and our approach (w/o CL), respectively. From left to right are the original adversarial example, the masked input, reconstruction by our approach (w/o CL), and reconstruction by our approach (w/ CL), respectively.
A.4 ABLATION STUDIES ON OUR CLASSIFIER
This section supports the main paper by conducting ablation studies on our classifier, including the impact of the contrastive loss reconstruction quality, as well as the effects of different masking ratios and our proposed adaptive ensemble on the standard accuracy and robustness.
Pre-training: Contrastive Loss. We qualitatively and quantitatively exhibit the impact of our proposed loss, i... Eq. on learning visual representations. We first present the qualitative evalua- tions. Specifically, we reconstruct masked adversarial examples and compare reconstruction quality by utilizing our approach with/without the contrastive loss (CL) in SimCLR Chen et al. (2020). Figure 5 jillustrates the qualitative results. For images on each row, from left to right, are original adversarial example, the masked input, the image generated by our approach without the CL (i.e., w/o CL), and the image reconstructed by our approach with the CL (i.e., w/ CL). We observed that when using the CI., our approach always achieves better reconstruction quality; See the 3rd (and 7th) column versus the 4th (and 8th) column. Besides, we discovered that our approach (w/o CL), in some cases, reconstructs adversarial examples with poor quality; See the 3rd and 7th columns in the last row. By contrast, our method (w/ CL) still achieves a high reconstruction quality on these examples; See the 4th and 8th columns in the last row. These empirical results demonstrate that our proposed loss can boost the performance when learning visual representations from adversarial examples.
Following MAE He et al. (2022), we quantitatively evaluate visual representations by using the lin- ear probing accuracy. Specifically, we consider the standard accuracy, the robustness under dense attack (i.e., PGD-50), and the robustness under sparse attack (i.e., C&W T.o). (abbeena the experimental results. We observed that by utilizing the contrastive loss, our approach achieves per- formance improvement of 2.6%. 11.3%, and 10.7% on the standard accuracy, the robustness against PGD-50, and the robustness against C&W Lg, respectively. These empirical results demonstrate the necessity and importance of proposed loss for learning high-quality visual representations.
Fine-tuning: Masking Ratio. In this section, we conduct experiments on CIFAR- 10 to explore how different masking ratios affect the performance of our approach during the finetuning. 12 groups of masking ratios are taken into account, ranging from 25% to 80%. Note that in the pre- training, we directly set the masking ratio to 75% by following MAE He et al. (2022): hence, no similar ablation study requires. We consider the trade-off between the standard accuracy and the robustness, including robustness against a dense attack (i.e., PGD-50) and against a sparse attack (i.e., C&W Lg).
Figures 6a and 6b illustrate the experimental results. From Figure 6a we observed that increasing the masking ratio will negatively affect the standard accuracy (i.e., the grey line) in all scenarios. In contrast, when the masking ratio is small (i.e., < 50%), a larger masking ratio benefits the robust- ness against the dense attack (i.e., the blue line). But when the masking ratio is greater than 50%,
14

---

Under review as a conference paper at ICLR 2023
Table 6: Ablation studies on our classifier, including (a) pre-training and (b) fine-tuning, as well as on our detector, including (c) Guided Backpropagation
Methed
Standard Accuracy
Robustness
Method
Standard Accuracy
Robustness
PGD-50
C&W
PGD-50
C&W Lo
CI.
75.8
37.1
49.3
w/ CL
78.4
48.4
47.2
w/ AE
91.2
55,9
54.8
(a) Pre-training: Contrastive Loss (CL)
(b) Fine-tunings Adaptive Ensemble (AE)
Method
PGD-20
wro
95.7
82.6
GB
99.1
96.4
(c) Detection: Guided Backpropagation (GB)
50
Maiking
Marking Ratio
(a) Under PGD-50 attack
(b) Under C&W Lo attack
Figure 6: Illustration of how different masking ratios in the finetuning affect the performance.
increasing the masking ratio hurts the robustness. This is because small subset of masked patches can eliminate the adversarial effect of adversarial attacks, while a large subset of masked patches would prevent our classifier from accurate classification. Clearly, our approach achieves the best trade-off on the masking ratio of 45% (or 40%). with the standard accuracy of 91.2% (or 91.3%) and the robustness against PGD-50 of 55.9% (or 35.1%).
Similarly, Figure 6b depicts the robustness against the sparse attack (i.e., the pink line) under dif- ferent masking ratios. We also include the standard accuracy (similar to Figure (a) for a better illustration of the trade-off. Obviously, when the masking ratio equals 45%, our approach achieves the best trade-off, with the standard accuracy of 91 21.2% and the robustness of 54.8%. Based on the above discussion, we can set our masking ratio 45% ensure the best trade-off between the standard accuracy and the robustness against both dense and sparse attacks.
Fine-tuning: Adaptive Ensemble. Here, we conduct experiments to show the impact of our adap- tive ensemble the standard accuracy and the robustness. Table 6b lists the experimental results with/without our adaptive ensemble. Note that we employ the naive average ensemble when con- ducting experiments without our adaptive ensemble. From Table 6b we observed that our adap- tive ensemble significantly benefits the standard accuracy, with 11.5% performance improvement. Mcanwhile, boosts adversarial robustness against PGD-50 by 6.6% and against C&W Lo by 5.4%. This is because the adaptive factor in Eq. (10) estimated by our detector can adaptively adjust the proportion of visual representations from clean and adversarial encoders.
A.5 ABLATION STUDIES ON OUR DETECTOR
This section supports the main paper by conducting ablation studies on our detector. We present the effects of Guided Backpropagation and proposed loss, i.e., E on the detection accuracy and visual representations for clean images and adversarial examples, respectively.
Guided Backpropagation on the Detection Accuracy. In this section, we empirically show how Guided Backpropagation (GB)-based variant affects the detection accuracy under a dense attack (i.e.. PGD-20) and a sparse attack (i.e., C&W Lg). Table 6c lists the experimental results with/without the Ga-based input variant CIFAR-10. We discovered that removing the GB-based input variant from our design results in a small detection accuracy degradation of 3.4% under the dense attack. In contrast, it incurs significant detection accuracy drop of 13.8% under the sparse attack. This
15

---

Under review as a conference paper at ICLR 2023
Ocsa
Clsa
Qun
Con
(a) w/o SNN loss (PGD-20 attack)
(b) w/ SNN loss (PGD-20 attack)
(c) w/ SNN loss (C&W Lo attack)
(d) w/ SNN loss
(C&W Lo attack)
Figure 7: t-SNE visualization on CIFAR- 10 by using our detector with/without SNN loss. For each experiment, we perform t-SNE visualization on 200 clean images and 200 adversarial examples generated either by PGD-20 attack, i.e,, (a) and (b), or by C&W Lo attack, i..., (c) and (d).
confirms that Guided Backpropagation can expose adversarial perturbations, making adversarial examples, especially those generated by sparse attacks, easier to be detected.
SNN Loss on Visual Representations. This section reveals the effect of our proposed loss, i.e., E5. on detecting adversarial examples. We consider how our detector with or without the Soft- Nearest Neighbors (SNN) loss affects the resulting representation space. particular, we employ t-SNE visualization van der Maaten & Hinton (2008) on 200 clean images randomly sampled from CIFAR-10 and 200 adversarial examples generated either by the dense attack (i.e., PGD-20) or by the sparse attack (C&W Lo). Figures [7 and 7b depict the results by using PGD-20 attack, while Figures 7c and present the results by employing C&W Lo attack. We observed that without the SNN loss, the representations for clean images and adversarial examples are highly entangled; Figures 7a and 77 In sharp contrast, by minimizing the SNN loss, the representations for clean images and adversarial examples are mutually isolated, shown in Figures 7b and 7 making adversarial examples detectable.
16


Reviews:
[{"review_id": "XGklSqupZm", "reviewer": "Reviewer_Ty1m", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new framework to improve robust accuracy while maintaining standard accuracy.\nSince the gradient visualization for the adversarial image looks different from the normal image by the authors’ empirical observation, the authors use the visualization generated by Guided Grad-CAM and the input image as a pair to learn an adversarial example detector.\nIt produces a probability for how likely an image is to be attacked.\nFor the classification task, the authors use two distinct branches to classify normal and adversarial images separately, and adaptively ensemble two branches’ results with the probability predicted by the adversarial detector.\nThey use a pre-training and fine-tuning receipt to ensure the normal image encoder and the adversarial image encoder produce a similar representation for the same image with and without attack.", "summary_of_the_review": "Interesting approach, but some important details are missing.", "strength_and_weaknesses": "The proposed approach is interesting. However, I have the following concerns and questions:\n1. The authors claim that the adversarial examples are noticeable with Guided Backpropagation visualization in their empirical observation. However, previous work ex: Fooling Neural Network Interpretations via Adversarial Model Manipulation https://arxiv.org/abs/1902.02041 have shown that those visualizations can be manipulated under the white box settings. It is possible for the attackers to control the region that the model focuses on.\n\n2. It is unclear how the authors generate adversarial images to pre-train the encoder without having the classifier ready. The adversarial example should come from attacking an existing model. Are those adversarial images generated from attacking some model else?\n\n3. How are the adversarial images generated during the evaluation? Is the full structure (detector, two branch image encoder) go under white box attack?\n\n4. The authors claim they beat the state of the art under multiple attacks. However, the epsilon budget for those attacks is not specified throughout the article.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "VgGhMS9J-3C", "reviewer": "Reviewer_ZoGc", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a framework for the detection and correct classification of adversarial examples for applications where these cannot be rejected. The focus is the union $L_{\\infty}$ and $L_0$ attacks. The entire architecture is based on vision transformers. The detector uses guided backpropagation to render adversarial features more salient. The classifier leverages masked autoencoders, with two encoders. The classifier is trained in two steps, first learning embeddings, then being fine-tuned for the classification task. Experiments are performed on CIFAR-10, CIFAR-100 and Tiny ImageNet.", "summary_of_the_review": "Interesting and novel idea for a classification and detection framework of adversarial samples. Unfortunately, the experimental evaluation is not strong enough to support the performance claims of the method. Moreover, the method is not set in the context of similar methods literature and their failures.", "strength_and_weaknesses": "Strengths:\n- Interesting idea to combine the recent masked auto encoders with guided backpropagation for an adversarial example detection framework.\n- The combination of ideas in the paper appears to be novel.\n- Good ablation study.\n\nWeaknesses:\n- Incomplete experimental evaluation, with weak attack baselines and failing to state important attack parameters.\n- Limited reproducibility: missing important parameters, code not provided.\n- Missing important references related to adversarial examples' detection.\n\nPlease see detailed points below.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Dl8IDCYrwuO", "reviewer": "Reviewer_uWvJ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to combine the masked patches from both adversarial examples and clean examples to boost the model's robustness against adversarial attacks. Specifically, to differentiate adversarial example from the clean example, it first train a detector based on the heatmap obtained by guided backpropogation. To make the dectector better, It then chooses the cross-entropy loss together with the SNN loss.  Then it proposes to have a two phases classifier to adversarially train the model. It first do a pre-training with a auto-encoder way to reconstruct image and get a better model initialization. It then adds a MLP after performing a global average pooling on the adaptive ensemble between adversarial features and clean features with a coefficient got by the detector. The proposed method is then tested on CIFAR10, CIFAR100 and tiny imagenet dataset in the experiment section.", "summary_of_the_review": "The paper proposes to use a combination of detector and adversarial training to improve the model's robustness against adversarial attacks. However, the proposed framework is only shown in the ViT models and there exists some problems in the experiment evaluation.", "strength_and_weaknesses": "Pros:\n1. The proposed method achieve a good result in the L_0 attack case without introducing additional training.\n\n\nCons:\n1. The illustration of proposed framework could be improved. The whole framework is composed by several large components however it is not easy to get the whole picture how to do the training and inference in the end. \n2. It is not clear how the detector is trained. Is it trained before the pre-training and fine-tuning? If it is, are the adversarial examples generated from other models?\n3. The proposed framework is based on the ViT in both detector and classifier by using the patchify operation. It is not clear whether it could be applied in other CNN models such as ResNet, VGG etc.\n4. The improvement on the L-inf attack (dense attack) is limited. The autoattack is the strongest attack among the attack tested and however the proposed method is actually worse than other baselines. Although the improvement over l_0 attack is significant, however, It is also not fair to compare because those models are trained adversarially with L-inf attacks. A fair comparison would those defenses including ensemble of different attacks.\n5. It is not clear whether the attack tested have already known the existence of the detector. It would be trivial to defense if the attacker doesn't know the existence of the detector, which could regarded as another way of obfuscated gradient problem. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3LGanxgHu5", "reviewer": "Reviewer_W9ju", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents two methods for two relevant tasks. One is an attack detection method inspired by guided backpropagation. The other is adversarial training using pre-training and adaptive ensemble techniques. The two methods are evaluated on CIFAR-10, CIFAR-100, and Tiny-Imagenet datasets. Experimental results demonstrate the effectiveness of the proposed methods.", "summary_of_the_review": "This paper presents two straightforward ideas for attack detection and defense. The experiments are not convincing, suffering from fairness issues and missing metrics. I think it is very hard to change the situation during the rebuttal process. So my recommendation is something amid a weak reject and a strong reject.", "strength_and_weaknesses": "Strengths\n\n-[insight] The motivation to turn visualization (guided backprop) into a detection method is good\n\n-[effectiveness] the experimental results demonstrate the effectivenss of the proposed method\n\nWeaknesses\n\n-[typesetting, minor] please use \\citep and \\citet macros accordingly in the manuscript.\n\n-[adaptive attack, major] This paper talks about both defense and detection. However, adaptive attack (On Adaptive Attacks to Adversarial Example Defenses, neurips2020) and attacks against detection methods are not discussed (Adversarial Examples Are Not Easily Detected:\nBypassing Ten Detection Methods, AISec'17). Specifically, the notation of of adversarial robustness (eq.2) is built upon both detection and defense. So as long as one of them is broken, the whole system is compromised. In other words, the worst-case performance of the combination of detection and defense is the weakest one among detection and defense. In that sense, the adaptive attack is more necessary to be discussed.\n\n-[clarity, minor] in section 3.2, it is illustrated that the visualization inspired the detection method. However, perturbations from different attacks may have different characteristics that result in a distinct pattern in the visualization. This section does not explicitly specify whether the observation is applicable to any (or what) kind of adversarial perturbation. In that sense, the foundation of the proposed method is not very well supported.\n\n-[fairness in comparison, major] in section 4.1 \"parameter settings\", we know that the baseline methods are still using CNNs instead of a transformer like the proposed method. Even if the transformer is trimmed to a lower number of parameters, it is still posing unfair comparisons to previous works. We know that architecture itself is already a big enough variable regarding adversarial robustness. Plus, a transformer is well-suited for patch-wise representations, which may make transformers naturally good against sparse attacks. In that sense, we cannot isolate the contribution of the transformer itself and the proposed defense method in the robustness gain. What if the previous defense methods use the same transformer backbone? That would reveal the true improvement. The current table 1 suffers from unfair comparison and is not convincing.\n\n-[missing evaluation, major] in section 4.3, only \"accuracy\" is reported for attack detection. However, it is widely known that for a \"detection\" tasks, reporting merely TPR (true positive rate) or accuracy without reporting any FPR (false positive rate) is not making an experiment convincing. See also (Adversarial Examples Are Not Easily Detected:\nBypassing Ten Detection Methods, AISec'17). A detector that achieves 99% TPR at a high FPR is not necessarily useful.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject