Adversarial Feature Desensitization
Authors: Pouya Bashivan
Reza Bayat
Adam Ibrahim
Kartik Ahuja
Mojtaba Faramarzi
Touraj Laleh
Blake Aaron Richards
Irina Rish
Venue: NeurIPS 2021 Poster (2021)
Keywords: adversarial robustness
adversarial learning
adversarial examples
domain adaptation

Abstract: Neural networks are known to be vulnerable to adversarial attacks -- slight but carefully constructed perturbations of the inputs which can drastically impair the network's performance. Many defense methods have been proposed for improving robustness of  deep networks by training them on adversarially perturbed inputs. However, these models often remain vulnerable to new types of attacks not seen during training, and even to slightly stronger versions of previously seen  attacks. In this work, we propose a novel approach to  adversarial robustness, which builds upon the insights from the domain adaptation field. Our method, called Adversarial Feature Desensitization (AFD), aims at learning  features that are invariant towards adversarial perturbations of the inputs. This is achieved through a game where we learn features that are both predictive and robust (insensitive to adversarial attacks), i.e. cannot be used to discriminate between natural and adversarial data. Empirical results on several benchmarks  demonstrate the effectiveness of the proposed approach against a wide range of attack types and attack strengths. Our code is available at https://github.com/BashivanLab/afd.

Adversarial Feature Desensitization
Pouya Bashivan 1.2.
Reza Bayat
Adam Ibrahim
Kartik Ahuja²
Mojtaba Faramarzi
Touraj Laleh
Blake Richards
Irina R
McGill University, Montreal, Canada
2 MILA, Université de Montréal, Montreal, Canada *Correspondence to: (bashivap, irriaann rishhmmila quebec
Abstract
Neural networks are known to be vulnerable to adversarial attacks slight but carefully constructed perturbations of the inputs which can drastically impair the network's performance. Many defense methods have been proposed for improving robustness of deep networks by training them on adversarially perturbed inputs. However, these models often remain vulnerable to new types of attacks not seen during training, and even to slightly stronger versions of previously seen attacks. In this work, we propose novel approach to adversarial robustness, which builds upon the insights from the domain adaptation field. Our method, called Adversarial Feature Desensitization (AFD), aims at learning features that are invariant towards adversarial perturbations of the inputs. This achieved through a game where we learn features that are both predictive and robust (insensitive to adversarial attacks), i.e. cannot be used to discriminate between natural and adversarial data. Empirical results several benchmarks demonstrate the effectiveness of the proposed approach against a wide range of attack types and attack strengths. Our code is available at whttts::/// /i thub. com/ Bashi vanLab/afd
Introduction
When training classifier, common assume that the training and test samples are drawn from the same underlying distribution. In adversarial machine learning, however, this assumption is intentionally violated by using the classifier itself perturb the samples from the original (natural) data distribution towards new distribution over which the classifier's 's error rate is increased [52]. As expected, when tested on such adversarially generated input distribution, the classifier severely underperforms. To date, various methods have been proposed to defend the neural networks against adversarial attacks [34] 2. additive noise patterns and corruptions [24 25 [4), and transformations 17.. Among these methods, two of the most successful adversarial defense methods to date are adversarial training (34), which trains the neural network with examples that are perturbed to maximize the loss the target model, and TRADES [57], which regularizes the classifier to push the decision boundary away from the data. While past adversarial defence methods have successfully improved the neural network robustness against adversarial examples, it has also been shown that these robust networks remain susceptible to even slightly larger adversarial perturbations or other forms of attacks 19 46 48).
In this paper, we propose to view the problem of adversarial robustness through the lens of domain adaptation, and to consider distributions of natural and adversarial images as distinct input domains
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

that a classifier is expected to perform well on. We then focus our attention on learning features that are invariant under such domain shifts. Building upon domain adaptation literature [4] we use the classification-based H HAA--divergence to quantify the distance between the natural and adversarial domains. The theory of domain adaptation allows us to formulate a bound on the adversarial classification error (i.e. the error under the distribution of adversarial examples) in terms of the classification error on natural images and the divergence between the natural and adversarial features.
We further propose an algorithm for minimizing the adversarial error using this bound. For this, we train a classifier and a domain discriminator to respectively minimize their losses on the label classification and domain discrimination tasks. The feature extractor is trained to minimize the label classifier's loss and maximise the discriminator's loss. In this way, the feature extractor network is encouraged to learn features that are both predictive for the classification task and insensitive to the adversarial attacks. The proposed setup is conceptually similar to prior work in adversarial domain adaptation [18] [53, where domain-invarian features are learned through an adversarial game between the domain discriminator and feature extractor network.
This setup is similar to the adversarial learning paradigm widely used in image generation and transformation 20 28 60]. unsupervised and semi-supervised learning (39], video prediction 31
active learning 47. and continual learning [16]. Some prior work have also considered adversarial
learning to tackle the problem of adversarial examples [54 36 g These methods used generative
models to learn the distribution of the adversarial images [34] 33). or to learn the distribution of imput
gradients [9 8. Unlike our method which learns a discriminator function between distributions of adversarial and natural features and updates the feature extractor to reduce the discriminability of those distributions.
The main contributions of this work are as follows:
We apply domain adaaaiatn theory to the problem of adversarial robustness; this allows to bound the adversarial error in terms of the error the natural inputs and the divergence between the feature (representation) distributions adversarial and natural domains.
Aiming to minimize this bound, we propose a method which learns adversarially robust features that both predictive and insensitive to adversarial attacks, i.e. cannot be used to discriminate between natural and adversarial data.
We empirically demonstrate the effectiveness of the proposed method in learning robust models against wide range of attack types and attack strengths, and show that our proposed approach often significantly outperforms most previous defense methods.
Related Work
There is an extensive literature on mitigating susceptibility to adversarial perturbations 34 57
13 59, 3, (22 77). Adversarial training (34) is one of the carliest successful attempts to improve robustness of the learned representations to potential perturbations the input pattern by solving min-max optimization problem. TRADES (57) adds a regularization term to the cross-entropy loss which penalizes the network for assigning different labels to natural images and their corresponding perturbed images. [41] proposed an additional regularization term (local linearity regularizer) that encourages the classification loss to behave linearly around the training examples. [55 51 proposed to regularize the flatness of the loss to improve adversarial robustness.
Our work is closely related to the domain adaptation literature in which adversarial optimization has recently gained much attention [8] 32 53 From this viewpoint one could consider the clean and perturbed inputs as two distinct domains for which a network aims to learn an invariant feature set. Although in our setting, i) the perturbed domain continuously evolves while the parameters of the feature network are tuned; ii) unlike the usual setting in domain adaataaiont problems, here we have access to the labels associated with some samples from the perturbed (target) domain. Recent [or regularized the network to have similar logit values response clean and perturbed inputs and showed that this additional term leads to better robust generalization to unseen perturbations. Related to this, Adversarial Logit Pairing (27 increases robustness by directly matching the logits for clean and adversarial inputs. JARN 9 Another line of work is on developing certified defenses which consist of methods with provable bounds over which the network is certified to operate robustly [58 56 10 While these approaches provide sense of guarantee about the proposed defenses, they

---

caturbe
Fa
Ci
Clsot
Cland
Calogiory
D
toweds
avayy
clean Perturodd
Adversarial Training
TRADES
AFD
(a)
(b)
Figure 1: (a) Overview of the proposed AFD approach; (b) Visual comparison of adversarial robustness methods (Adversarial training 34, TRADES [57] and AFD), The dashed red and dotted black lines correspond to the decision boundary of classes and the domain discriminator respectively.
are usually prohibitively expensive to train, drastically reduce the performance of the network on natural images, and the empirical robustness gained against standard attacks is low.
3 Our approach
We will now make a connection between the domain adaptation and adversarial robustness, and build upon this connection to develop an approach for improving the network's robustness against adversarial attacks.
3.1 Preliminaries
Let Fo(x) X' - 2, where X' C R", 2 C Rmm be feature extractor (e.g. a neural network with parameters 0) mapping the input
X into the feature vector (representation) Co 2 - V, where )' (l...., ......,KK K} are the class labels, be a classifier, with parameters ø (e.g., the last linear layer of neural network plus the softmax function, on top of the extracted features).
2, and let
Adversarial attack: Let क (xx denote a perturbation function (an adversarial attack) which, for given (z, y) E X x y. generates perturbed sample x' E B(x, e) within the e-neighborhood of z, B(x, <) = (x E X: : || æ|| < c). by solving the following maximization problem
max tEB(x,e)
((C((Fo
where C is the task classification loss function. In practice, however, the perturbed sample a' found by an attacker is typically an approximate rather than the exact solution to this maximization problem.
In order to characterize the distance between the natural and adversarial data distributions, the following notion distance between two probability distributions, defined in [4 [8], will be used later to make connection with domain adaptation theory.
Let H be set of binary classifiers (hypotheses), called hypothesis space: then the symmetric difference hypothesis space HAH defines the set of hypotheses that capture the disagreements between two hypotheses in H., as in [4]:
HAH-distance:
g E HAH = g(x) = h(x) @ N'(x)) for some h, N E H,
where @ denotes the XOR function. Then the HAH-distance 4 48 between two data distributions (domains) S and T, with respect the hypothesis space H. is defined as:
dHAn(S,T)= =
sup Pa-sshhas = 1 - Pa-T [b(x) = 1]1-
(3)
HEHAH
This equation turns into an inequation when the supremum is taken over the hypothesis space H. instead of HAH (18.
3.2 A Domain Adaptation View of Adversarial Robustness
domain is defined as a data distribution on the set of inputs X' [5]. In the adversarial robustness setting, we consider two domains the natural and the adversarial domains, corresponding respectively
3

---

to the source and target domains in domain adaptation. We denote by Dx and D'y the natural and adversarial distributions of input instances respectively and by Dz and D'z their corresponding induced distributions over the feature space 2. As in domain adaptation, we assume that f X - y a labeling function common to both domains. The expected classification error ez of the classifier Co over Dz is defined as the probability that the classifier Co disagrees with the function f:
ez(Co) = DI (y + Ca(z)],
(4)
where f: - y/ is a mapping from the features to the class label such that f(x) f (()())) We similarly define <zz as the expected error of Cs over Dz'. Using theorem 2 from 4 that relates the source and the target domain errors, we get an upper bound on the expected adversarial error c'2 as:
(2) S cz (h) +
2 ddHAH(D,, D'2) +c,
(5)
where e is a constant term w.r.t. h. Eq. 5 @esenttially gives a bound on the adversarial error e'z in terms of the natural error eg and divergence dHAh between the natural and adversarial domains with respect to their induced representation distributions D2 and D'z. In the next section, we will describe an algorithm for improving adversarial robustness of model by iteratively estimating and minimizing these two components of the error bound.
3.3 Adversarial Feature Desensitization
Based on Eq. [] the expected adversarial error could be reduced by jointly minimizing the nat- ural error and the divergence between the distributions of natural and adversarial representations dHAH( (Dz, D'z). While minimizing the natural error ex is straightforward, minimizing the cross- domain divergence requires us to estimate d D'z). domain discriminator De is closely related to estimating the dHan (Dz, 2. D' ). The domain discrimi- nator is a classifier trained to assign a label of to samples from D2, and -1 to samples from D'z- Namely, it is shown 188 that
As was shown before [88. training a
dHAH((Z, (Dz, S 2 supl NEH
(h) 1/,
(6)
where O'22,D's (h) PZ-D2 [x= 1] + + [M(x) = -1] combines the true positives and true negatives, and is thus maximized by the optimal domain discriminator h = De. Note that, if the domain distributions Dz and D'z are the same, then even the best choice of domain discriminator De will achieve chance-level accuracy, corresponding to O'Dz.D', (Dw) 1. Our approach will aim at minimizing this estimated distance dHAx(DZ D'z) ) by tuning the feature extractor network parameters 0 in the direction that pushes the distributions Dzz and D'z closer logether. In parallel, we train the domain discriminator to estimate and guide the progress of the feature extractor's tuning.
We now describe the proposed approach (see Algorithm which essentially involves simultancous training of the feature extractor Fg. the task classifier Co and the domain discriminator Di (see Figure la One iteration of the training procedure consists of the following three steps.
First, parameters of the feature extractor Fe and classifier Co are updated aiming to minimize the natural error ex using the cross-entropy loss on natural inputs:
Lo= = - 1 Σ Js log (softmax(C,(Fo(x)))))
where ū; is a one-hot encoding of the true label of the i-th sample 24-
Next, steps two and three essentially implement two-player minimax game similar that in Generative Adversarial Networks (GAN) (20. carried out between the feature extractor network Fa and the domain discriminator Des, with value function
e)][] + Eg(u) (SD) v)]]]. (8) Note that we will somewhat abuse the notation, assuming that Co and Dr below correspond to the logits (last-layer output) of the corresponding networks. Also, we will use class-conditionss discriminators. DeoFo (z. v)), ie. train different domain discriminator for different label values y.
(iee, D+) = Ep(x)

---

Algorithm 1: AFD training procedure
Input: Adversarial perturbation function (attack) и, feature extractor Fo; task classifier Cø. domain discriminator Dæ,, learning rates ot f3, and 7y.
repeat
input next mini-batch {1r U1), (Imm Vm)]
for m: x < T E)
Compute Le according to Eq.
Compute LD according to Eq.
Compute Lp according to Eq.
I0
(0, Φ) f (0. Φ)
aVe. PCC % update feature extractor and task classifier
sb + e BWLLD
% update domain discriminator
- VoLr until convergence;
% update feature extractor
where is the softplus function. In particular, parameters of the domain discriminator D3 updated to minimize the cross-entropy loss associated with discriminating natural and adversarial inputs, maximizing ahh) in Eq. 6
CD Σ ((-Dv(Fe(a)) w)) + S(D(((F(()) u))]]
(9)
while the parameters the feature extractor function Fg are adversarially updated to maximize domain discriminator's loss from Eq. 9
m
CF
m Σ S( (Fo 9:))
(10)
In Figure Db. we visually compare the learning dynamics in adversarial training, TRADES and AFD. Essentially, the adversarial training solves the classification problem by pushing the representation of adversarial examples from different classes away. TRADES regularizes the normal classification loss on the natural inputs with an additional term that encourages the representation of adversarial and natural images to match. Similar to TRADES, in AFD, the regular classification loss natural inputs is augmented but with an adversarial game which consists of training the domain discriminator that distinguishes between the adversarial and natural inputs for each class followed by updates to the feature extractor to make the representations for natural and adversarial examples to become indistinguishable from each other. Notably, because the parameter update for the feature extractor network is done to maximize the domain discriminator loss and not to decrease the loss for particular adversarial examples (as is done adversarial training or TRADES), potentially increases the network robustness against any perturbation that could be correctly classified using the same domain discriminator. This could potentially lead to broader form of generalization learned by the network.
Relation to Adversarial Training. Adversarial training minimizes the expected error on adversarial examples (the perturbed versions of the natural samples), generated by attacker in order to maximize the classification loss. The adversarial training procedure involves minimax optimization problem consisting of an inner maximization to find adversarial examples that maxi- mize the classification loss and an outer minimization to find model parameters that minimize the adversarial loss. From the domain adaptation point of view, the inner optimization of adversarial training is equal to a sampling procedure that generates samples from the target domain. Intuitively, direct training of the classifier on samples from the target domain would be the best way improve the accuracy in that domain (i.e. adversarial classification accuracy). However, it's important to note that the adversarial examples found through the inner optimization only approximately maximize the classification loss, and therefore the adversarial eтror associated with these samples only act as a lower bound on the adversarial error and therefore the outer loop of the adversarial training method essentially minimizes lower bound on the adversarial classification error. In contrast to this setup, our proposed method minimizes conservative upper bound on the adversarial error and therefore is more likely to generalize to a larger set of unseen attacks, and to stronger versions of previously seen attacks (i.e. ones that generate higher-loss samples the inner optimization loop).
Discussion:
5

---

4 Experiments
4.1 Experimental setup
Datasets. We validated our proposed method on several common datasets including MNIST (0) CIFARIO, CIFAR [29], and tiny-Imagene, (26). The inputs for all datasets were used in their original resolution except for tiny-Imagenet where the inputs were resized to 32 x 32 to allow the experiments to finish within reasonable time on two GPUs.
Adversarial attacks. To fairly assess the generalization ability of each defense method across attack types, we tested each network on 9 well-known adversarial attacks from the literature, using existing implementations from the Foolbox 42] and Advertorch [12] Python packages. Namely, we tested the models against different variations of the Projected Gradient Descent (PGD) [34] (Log+ Lg. L1), Fast Gradient Sign Method (FGSM) (2). Momentum Iterative Method (MIM) 14. Decoupled Direction
and Norm (DDN) [4], Deepfool 40 C&W (6, and Autokt [T] attacks. Also to assess the generalization in robustness across stronger adversarial attacks, for each attack we also varied the value across wide range and validated different models on each. Specific hyperparameters used for
each attack are listed in Table-A2
Feature extractor network F' and classifier Cg. We used the same network architecture, ResNet18 [23 for the feature extractor and classifier networks in experiments on all datasets and only increased the number of features for more challenging datasets. The number of base filters in the ResNet architecture was set to 16 for MNIST and 64 for other datasets. We used the activations before the last linear layer as the the output of the feature extractor network (Z) and the last linear layer as the classifier network Cg- We added an activation normalization layer to the output of feature extractor network to provide normalized features to both Co and D& networks.
We compared several variations of the domain discriminator architecture and evaluated its effect on robust classification on MNIST dataset (Table A5 Overall, we found that using deeper networks for domain discriminator and adding projection discriminator layer improves the robust classification accuracy. The number of hidden units in all layers of D were equal (64 for MNIST and 512 for other datasets). Following the common design principles in Generative Adversarial Networks literature, we used the spectral normalization (37] on all layers of De. In all experiments, the domain discriminator (Dv) consisted of three fully connected layers with Leaky ReI.U nonlinearity followed by a projection discriminator layer that incorporated the labels into the adversarial discriminator through dot product operation 38 Further details of training for each experiment are listed in TAb
Domain discriminator network De
Training parameters and baselines. All networks including baselines were trained on an adaptive version of PGD attack (II) that adaptively tunes the step size during the attack with virtually no computational overhead compared to standard PGD attack. We used e = 0.3, 0.031, and 0.016 for MNIST, CIFAR, and finy-Imagemt randomly split the CIFAR 10 train set into a train and validation sets (45000 and 5000 images in train and validation sets respectively). We then carried out a grid-search using the validatii sets and picked the learning rates with highest validation performance. Based on this analysis, we selected the learning rate 7 = 0.5 for tuning the feature extractor Fo, and a = B = 0.1 for tuning the parameters in domain discriminator De, and the task classifier Co-
datasets respectively. To find the best learning rates, we
In all experiments we trained two versions of the AFD model, one with losses Lp and LF according 10 to Eq. 9 I0 which we call AFD-DCGAN and another version where we substitute the losses with
those from the Wasserstein GAN (I dubbed AFD-WGAN (see Eq. and 12 in the Appendix). We mainly compared the performance of our proposed method with two prominent defense methods, adversarial training and TRADES. We used re-implementation of adversarial training (AT) method 34] and the official code for TRADES? ? (57) and denoted these results with i in the tables. All experiments were run on NVIDIA V100 GPUs. We used one GPU for experiments on MNIST and GPUs for other datasets.
4.2 Robust classification against nominal attacks
We first evaluated our method against adversarial attacks under similar settings to those used during training (e = 0.3. 0.031, and 0.015 for MNIST, CIFAR, and Tiny-Imagent datasets respectively).

---

Table I: Comparison of adversarial accuracy against various attacks on different datasets. For PGD attack we used c = 0.3, 0.031, and 0.015 for MNIST, CIFAR CIFARROOCIFARIOM, and Tiny-Imagenet datasets respectively and for C attack we used = 1 for all datasets. i indicates replicated results. NT: natural training; AT: adversarial training: AFD: adversarial feature desensitization; WB: white-box attack; BB: black-box attack where the adversarial examples were produced by running attack on the NT R model. Numbers reported with μu + a denote mean and std values over three independent runs with different random initialization. RST additionally uses 500K
unlabeled images during training.
Method
Dataset
Network
Clean
PGD (WB)
C&W, (WB)
AA.,, (WB)
PGD,, (BB)
C&W, (BB)
NTİ AT 14
RN18
98.84
62.43
50.82
96.48
RN18
99.35
95.66
92.78
89.99
98.92
98.95
MNIST
TRADES
99.14
98
98.57
AFD-DCGAN
RNI8
99.24
95.72
93.78
88,79
98.65
98.49
AFD-WGAN
RN18
99.14
97.68
97.68
90.12
98.59
98.71
AT TRADES 57
RN18
87.3
45.8
86.0
RN18
84.92
56.61
87.60
RI
WRN-32
82.72
75.75
RST 75
WRN-34-10
89.82
64, 86
WRN-28-8
86.83
JARN 9
CIFARIO
WRN-34-10
84.8
46,7
NTI
RN18
94,89
0.55
17.93
AT
RN18
85.92
40.07
40,27
36.14
85.14
85.84
TRADES
RNI8
81.94
53.3
40.24
43.48
80.82
81.74
AFD-DCGAN
RN18
86.82
44.35
50,93
34.46
85.73
86.68
AFD-WGAN
RN18
85.95
$9.38
62.43
37,33
84,74
85.79
NT!
RNI8
76.76
0.01
0.52
RN18
56.49
18.54
17.71
18.30
56.07
56.42
TRADES[S7]
CIFARIOD
RN18
60.32
25,11
20.52
21.10
59.62
60.29
AFD.DCGAN AFD-WGAN
RN18
60.95
18.06
21.47
16.31
60.31
60.86
RN18
58.87
22.35
25.33
18.00
58. 15
58,75
NTI A TRADES 57 AFD-WGAN
Tiny-IN
RN18
43,80
12.62
4.90
9.48
41.87
42,86
37.70
13.26
36.26
RNI8
47.70
11.49
5.90
9.45
43,5
44.69
Table [alleeecosrrsss the robust classification performance of AFD and several other defense methods against L.D C& L2 and AutoAttack white-box black-box attacks. The black-box attacks were carried out by constructing the adversarial examples using ResNet18 architecture trained on the natural inputs I ~ xx- Overall both versions of AFD (AFD-DCGAN and AFD-WGAN) were highly robust against all five tested attacks while maintaining higher "Clean" accuracy (on natural data) compared to strong baseline models like TRADES and Adversarial Training. AFD-WGAN was consistently at the top MNIST and CIFAR10 datasets. On CIFARI 100 and Tiny-Imagennd AFD performed better than or similar to Adversarial Training on all the attacks and performed better than TRADES most of the attacks, although it occasionally behind TRADES (on PGD-L and AA white-bor box attacks). Analysis of feature sensitivity showed that on MNIST and CIFAR1O datasets on which AFD outperformed the other baselines by larger margin, the features more insensitive to adversarial perturbations and over a larger range of attack strengths (Figure-AA) In addition these tests, also evaluated the AFD model against transfer black-box attacks from Adversarial Training and TRADES models which further demonstrated AFD's 's higher robustness to those attacks too AT3
significantly
4.3 Robust classification against stronger and unseen attacks
To evaluate how each network generalizes unseen domains of adversarial inputs (i.e. adversarial attacks generated with unseen forms of adversarial attacks), we additionally validated the classification robustness against a range of possible values for several widely used attacks that were not used during training. To fairly compare different models while considering both attack types and values, we computed the grea-under-the-oun 2). Table-2] summarizes the AUC values for all 9 attack methods on four tested datasets. Compared with baselines, we found that, AFD-trained networks consistently performed better on various datasets and on almost all the tested attacks even for substantially larger values (Figure 2, also see Figures A A3 in the appendix). These results show that compared to other baselines, AFD- FFDttriind networks are robust against wider range of attacks and attack strengths (e). This further
(AUC) for accuracy vs. epsilon for each attack (similar to
7

---

Inlopd
Rogt
Opge
1gem
25-
Infrim
decsfool
Figure 2: Adversarial accuracy for different methods against white-box attacks on CIFARIO dataset with ResNet18 architecture.
Table 2: AUC measures for different perturbations and methods on MNIST, CIFARIO, CIFARIO0, and tiny-imagene datasets. AUC values are normalized to have maximum allowable value of Evaluations on AT and TRADES were made on networks trained using reimplemented or official
code.
Dataset
Model
PGD,..
PGD
PGD
FGSM
MIM
DDN
DeepFool
C&W
AA
NT
0.16
0.06
0.07
0.3
0.19
0.09
0.21
0.57
0.28
AT
0.74
0.29
0.19
0.83
0.95
0.49
0.55
0.87
0.89
MNIST
TRADES
0.71
0.26
0.15
0.47
0.86
AFD-DCGAN
0.77
0.33
0.3
0.78
0.91
0.51
0.49
0.9
0.88
AFD-WGAN
0.92
0.54
0.55
0.9
0.98
0.68
0.63
0.94
0.90
NT
0.05
0.1
0.17
0.19
0.05
0.1
0.16
0.1
0.12
AT
0.28
0,2
0.44
0.33
0.31
0.26
0.29
0.31
CIFARIO
TRADES
0.32
0.22
0.5
0.32
0.33
0.18
0.28
0.25
AFD-DCGAN
0.34
0.54
0.43
0.4
0.31
0.4
0.43
0.47
0.22
AFD-WGAN
0.56
0.54
0.66
0.59
0.56
0.4
0.52
0.62
0,24
NT
0.03
0.08
0.1
0.07
0.03
0.08
0.06
0.08
0.09
0.13
0.1
0.24
0.14
0.12
0.15
CIFARIO0
TRADES
0.16
0.13
0.31
0.12
0.17
0.18
0.1
0.16
0.15
AFD-DCGAN
0.14
0.12
0.27
0.17
0.16
0.15
0.16
0.18
0.13
AFD-WGAN
0.18
0.16
0.31
0.22
0.19
0.16
0.19
0.23
0.13
NT
0.04
0.03
0.08
0.05
0.04
0.06
0.07
0.07
0.07
Tiny-IN
0.10
0.03
0.16
0.15
0.09
0.14
0.13
0.11
TRADES
0.10
0.03
0,16
0.07
0.09
0.15
0.11
0.09
0.16
AFD-WGAN
0.10
0.04
0.19
0.12
0.09
0.15
0.16
0.12
0.15
suggests that the features learned through AFD generalize better across various forms of attacks and sustain larger perturbations.
also observed that the AFD-WGAN performs better than AFD-DCGAN under most tested conditions. This is potentially due to: 1) WGAN's ability to avoid vanishing gradients when the discriminator becomes too good compared to the generator (the feature extractor function in our case) [5]; 2) WGAN's ability to avoid mode-collapses during training. In training GANs, mode collapses lead to the generator network to only output a limited set of patterns instead learning to produce a diverse set of natural-looking images that fool the discriminator. Under our setting, WGAN potentially leads to learning a feature extractor that can produce a more diverse set of features for perturbed inputs, instead of focusing subset of latent dimensions. This suggests that applying more advanced GAN training algorithms could potentially further improve the robust performance in AFD-type models.
4.4 Estimated HAH--distance and adversarial-vs-natuee generalization gap
As stated in Eq. the upper bound on the adversarial error can be stated in terms of the natural error, the divergence between the two domains, and constant term. In practice, this means that the smaller the divergence term dRAN is, the smaller the gap between the adversarial and natural errors (e'z - <x) can be. We empirically tested this prediction using the domain discriminator trained on CIFARIO 10 dataset using the PGD-T attack. Figure Bp shows that the estimated dæAH using the domain discriminator (i.e., using the corresponding empirical value of a Eq. (6) trained on PGD Les with 0.031 is closely related to the adversarial-vs vestriirl--nnuuraal generalization gap over

---

0.9
0.6
1.4
linfpgd
11pgd
0.8
0.5
1.2
12pgd
fgsm
dHANDD (7,)
0.7
0.4
1.0
deepfoo!
0.8
CW
0.6
ddn
0.3 IN
W 0.6
0.5
0.2
0.4
0.4
0.1
0.2
0.3
0.01
0.02
0.03
0.06
0.13
0.0
0.2
0.4
0.6
0.8
1.0 1.2 1.4
Epsilon
duBAAD D2)
(b) Figure 3: (a) Estimated dHAH distance (based on empirical value of Eq. 6) and generalization gap in adversarial and natural error é'z scatter plot of the estimated dRaw distance using the domain discriminator and the gap adversarial and natural eπrоr across different attack types and magnitudes (i.e. e). Colors correspond to different attack types, Each dot corresponds to one attack evaluated a particular epsilon value. Estimations of the HHΛh distance for all attacks and epsilons are made with the domain discriminator trained on PGD- with 0.031.
(a)
as function of epsilon for PGD Loo attack; (b)
different e values predicted by Eq. 5 Moreover, estimations from the same domain discriminator also predicts the gap in generalization error attained for other forms of attacks (even ones not seen during AFD training) and values with high accuracy (Figure-3p). This further supports the proposal that minimizing the estimated distance between the natural and adversarial representations can be an efficient way to improve the model robustness against various adversarial attacks.
4.5 Learning a sparse representation
Because the AFD method aims learn representation that is insensitive adversarial attacks, expected the learned representational space to potentially of lower dimensionality (i.e. less number of orthogonal features). To test this, compared the dimensionality of the learned representation using two measures. i) number of non-zero features over the test set within each dataset and ii) number of Principal Component Analysis (PCA) dimensions that explains more than 99% of the variance the representation computed over the test-set of each dataset. We found that the same network architecture (i.e. ResNet18), when trained with AFD method learns much sparser and lower dimensional representational space (Table A4) compared to the naturally trained, adversarial training and TRADES models. The representational spaces learned with AFD on MNIST, CIFAR1O, and CIFARION 100 datasets had only 6, 9, and 76 principal components respectively.
4.6
Adversarial and norm-based desensitization
To investigate whether the same level of robustness could achieved by encouraging the network to produce similar representations in response to natural and adversarial inputs, we ran an additional experiment on the MNIST dataset in which we added a regularization term to the classification loss to directly minimize the representation sensitivity S, "E F(z'))i, [[. during training. We observed that although this augmented loss led to learning robustness representations, it achieved modest levels of robustness (~ 80%) and showed only weak generalization to stronger and other unseen attacks (FigureA.)). This result suggests that more direct forms of enforcing representational similarity may not lead to the same form of robustness with generalization properties similar that achieved using an adversarial training with domain discriminator (e.g. as in AFD).
Conclusion and limitations
Decreasing the input-sensitivity features has long been desired in training neural networks [15] and has been suggested as way improve adversarial robustness 44 61]. In this work proposed an

---

algorithm to decrease the sensitivity of neural network representations using an adversarial learning paradigm that involves joint training of domain discriminator, a feature extractor, and a task classifier. Essentially, our proposed algorithm iteratively estimates a bound on the adversarial enror in terms of the natural error and classification-bann and adversarial features and then minimizes the adversarial error by concurrently reducing the natural error as well as the distance between the two feature distributions.
measure of distance between the distributions of natural
Limitations. The empirical results presented here suggest that AFD-trained models are robust against wide range of adversarial attacks (distributions) even compared to strong baselines like Adversarial Training and TRADES. However, it is not guaranteed that the model would remain robust against any unseen attacks that we have not tested or may be invented in the future- as is the case in domain adaptation literature and the lack of theoretical guarantees for cross cross-domain generalization. With regards to the computational cost, when measuring the average per-epoch training time on the CIFAR 10 dataset (using 2 NVIDIA V100 GPUs), we found that the AFD training time is 31 % longer than adversarial training and only 4% longer than TRADES. This shows that while AFD requires three SGD updates per batch, the additional computational cost is not significantly higher than many prior methods when considering that most of the computational cost is associated with generating the adversarial examples during training.
6
Acknowledgements
We would like to thank Isabela Albuquerque, Joao Monteiro, and Alexia Jolicoeur-Martinge Martineau for their valuable comments on the manuscript. Pouya Bashivan was partially supported by the Unifying AI and Neuroscience Québec (UNIQUE) Postdoctoral fellowship and NSERC Discovery grant RGPIN-2021 -03005.. Irina Rish acknowledges the support from Canada CIFAR AI Chair Program and from the Canada Excellence Research Chairs Program.
References
[1] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. ICML, 1:298-322 00-,,,1,22 2017.
[2] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. 35th International Conference Machine Learning, ICML 2018, 1:436-448, 2018.
[3] Yang Bai, Yuyuan Zeng, Yong Jiang, Shu-Tao Xia, Xingjun Ma, and Yisen Wang. Improving Adversarial Robustness via Channel-wise Activation Suppressing. In ICLR, pages 2021.
[4] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. A theory of learning from different domains. Machine Learning, 19(1-2):151-175, 2010.
[5] Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. In Advances in Neural Information Processing Systems, pages 137-144, 2007.
[6] Nicholas Carlini and David Wagner. Towards Evaluating the Robustness of Neural Networks. Proceedings IEEE Symposium on Security and Privacy, pages 39-57, 2017.
[7] Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. In Advances in Neural Information Processing Systems, pages 11 191190111 1201, 2019.
[8] Alvin Chan, Yi Tay, and Yew-Soon Ong. What it thinks is important is important: Robustness transfers through input gradients. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 332-341, 2020.
[9] Alvin Chan, Yi Tay, Yew Soon Ong, and Jie Fu. Jacobian Adversarially Regularized Networks for Robustness. ICLR, 2020.
10

---

[10] Jeremy Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via ran- domized smoothing. 36th International Conference on Machine Learning. ICML 2019, 2019- June:2323-2356 2019.
[11] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fre free attacks. arXiv preprint arXi::2003.01690, 2020.
[12] Gavin Weiguang Ding, Luyu Wang, and Xiaomeng Jin. AdverTorch v0.1: An adversarial robustness toolbox based on pytorch. arXiv preprint arXiv:1902.07623, 2019.
[13] Yinpeng Dong. Zhijie Deng, Tianyu Pang, Hang Su, and Jun Zhu. Adversarial Distributional Training for Robust Deep Learning, In Neural Information Processing Systems (NIPS), number NeurIPS, pages 1-14, 2020.
[14] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting Adversarial Attacks with Momentum. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 9185-9193, 2018.
[15] Harris Drucker and Yann Le Cun. Improving generalization performance using double back- propagation. IEEE Transactions on Neural Networks, 3(6):991-99)) 1992.
[16] Sayna Ebrahimi, Franziska Meier, Roberto Calandra, Trevor Darrell, and Marcus Rohrbach. Adversarial continual learning. arXiv preprint arX1vv2003.09553, 2020.
[17] Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. Exploring the landscape of spatial robustness. 36th International Conference on Machine Learning, ICML 2019, Mo-Ju:n:n21-22222 2019.
[18] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. 32nd International Conference Machine Learning, ICML 2015, 20):1180-189, 2015.
[19] Ian Goodfellow, Dan Boneh, and Patrick Medaniel. Ensemble Adversarial Training: Attacks and Defenses. In ICLR, pages 1-20, 2018.
[20] Ian Goodfellow, Jean Pouget-Abadic Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672-26888 2014.
[21] Ian J Goodffllow,, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:1412.6577 2014.
[22] Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against com-boonded adversarial examples. arXiv, (2017):1-
30, 2020.
[23] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and paitern recognition, pages 770-778, 2016.
[24] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In ICLR, pages 1-16, 2019.
[25] Dan Hendrycks, Mantas Mazeika, Sauray Kadavath, and Dawn Song. Using pre-training can improve model robustness and uncertainty. NeurlPS, pages 4815-4826. 2019.
[26] Qiuyuan Huang, Paul Smolensky, Xiaodong He, Li Deng, and Dapeng Wu. Tensor product generation networks for deep NLP modeling. arXiv, pages 1263-1273, 2017.
[27] Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing, arXiv preprint 1833v:1803.06333, 2018.
[28] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4401-4410, 2019.

---

[29] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[30] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):227--2322 1998.
[31] Alex X Lee, Richard Zhang, Frederik Ebert, Pieter Abbeel, Chelsea Finn, and Sergey Levine. Stochastic adversarial video prediction. arXiv preprint 180400000 2018.
[32] Hong Liu, Mingsheng Long, Jianmin Wang, and Michael Jordan. Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers. International Conference on Machine Learning, pages 4013-4022, 2019.
[33] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, (Noov: 2008.
[34] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.0660 2017.
[35] Michael Mathieu, Camille Couprie, and Yann LeCun. Deep multi-scale video prediction beyond mean square error. arXiv preprini 15X 2015.
[36] Alexander Matyasko and Lap-Pui Chau. Improved network robustness with adversary critic. In Advances in Neural Information Processing Systems, pages 10578-10587, 2018.
[37] Takeru Miyato, Toshiki Kataoka, Koyama Masanori, and Yoshida Yuichi. Spectral normalization for generative adversarial networks. ICLR, 2018.
[38] Takeru Miyato and Masanori Koyama. egans with projection discriminator. arXiv preprint arXiv: 1802.05677 2018.
[39] Takeru Miyato, Shin Ichi Maeda, Shin Ishii, and Masanori Koyama. Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Sune Sumi-Super Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1-16, 2018
[40] Seyed Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. DeepFool: Simple and Accurate Method to Fool Deep Neural Networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-Decem:2574-2555 2016.
[41] Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhus- sein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial Robustness Through Local Linearization. (NeurIPS):1-10 2020.
[42] Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: A python toolbox benchmark the robustness of machine learning models. In Reliable Machine Learning in the Wild Workshop. 34th International Conference on Machine Learning. 2017.
[43] Jerome Rony, Luiz G. Hafemann, Luiz S. Oliveira, Ismail Ben Ayed, Robert Sabourin, and Eric Granger. Decoupling direction and norm for efficient gradient-based 12 adversarial attacks and defenses. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019 9019-June:4317-4335 2019.
[44] Andrew Slavin Ros and Finale Doshi- Velez. Improving the adversarial robustness and inter- pretability of deep neural networks by regularizing their input gradients. 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, pages 1660-1669, 2018.
[45] Evgenia Rusak, Lukas Schott, Roland S. Zimmermann. Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. Increasing the robustness of DNNs against image corruptions by playing the Game of Noise. 2020.
[46] Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversar- ially robust neural network model on mnist. arXiv preprint arXiv:1805.09) 2018
12

---

[47] Samrath Sinha. Sayna Ebrahimi, and Trevor Darrell. Variational adversarial active learning.
Proceedings of the IEEE International Conference on Computer Vision, 00119-ctob::971-5900 2019.
[48] Chawin Sitawarin, Supriyo Chakraborty, and David Wagner. Improving adversarial robustness through progressive hardening. arXiv preprint arXiv:2003.093347, 2020.
[49] Chuanbiao Song, Kun He, Liwei Wang, and John Hoperoft. Improving the generalization of adversarial training with domain adaptation. In ICLR, pages 1-14, 2019.
[50]
Chuanbiao Song, He Kun, Lin Jiadong, John E Hopcroft, and Liwei Wang. Robust local features for improving the generalization of adversarial training. In ICLR, pages 1-12, 2020.
[51] Gaurang Sriramanan, Sravanti Addepalli, Arya Baburaj, and R. Venkatesh Babu. Guided Adversarial Attack for Evaluating and Enhancing Adversarial Defenses. Neural Information Processing Systems (NIPS), number NeurIPS, 2020.
[52] Christian Szegedy, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Joan Bruna, Rob Fergus, and Dumitru Erhan. Intriguing properties neural networks. pages 1-10, 2013.
[53] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. Proceedings 30th IEEE Conference on Computer Vision and Pattern Recognition, 2017.
CVPR 2017, 201 2917
[54] Huaxia Wang and Chun-Nam Yu. A direct approach to robust deep learning using ad adversarial networks. arXiv preprint arXiv: 2019.
[55] Dongxian Wu, Yisen Wang, and Xia Shu-Tao. Revisiting Loss Landscape for Adversarial Robustness. ICML, 2019.
[56] Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsich, and Liwei Wang. Macer: Attack-free free and scalable robust training via maximizing certified radius. arXiv preprint arXiv:2001.02336 2020.
[57] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. arXiv preprint arXiv: 1901008701 2019.
[58] Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, and Cho-Jui Hsieh. Towards Stable and Efficient Training of Verifiably Robust Neural Networks. pages 1-25, 2019.
[59] Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry-aware instance-reweighter adversarial training. In ICLR, pages 1-28, 2021.
[60] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference computer vision, pages 2223-2232, 2017.
[61] Sicheng Zhu, Xiao Zhang, and David Evans. Learning adversarially robust representations via worst-case mutual information maximization. ICML, 2020.
13


Reviews:
[{"review_id": "gpeSoJk7wbO", "reviewer": "Reviewer_R6D1", "summary": "This paper proposes a feature desensitization method for adversarial defense. The authors formulate robust representation learning problems from the domain adaptation perspective, the algorithm to solve which is based on two-player minmax methods (i.e., GAN). The evaluation results on multiple datasets demonstrate the advantage over baseline methods. The learned feature also shows to be spare.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well organized and written. I like the idea of investigating robust feature learning from the domain adaptation perspective. The intuition and insights are also nicely discussed with nice figures to give intuition on the difference of AFD from adversarial training and TRADES.\n\nThe proposed method is also feasible and seems to be sound. The evaluation also gives interesting results. \n\nEven though, the paper still posts several questions that should be carefully addressed:\n\n- All the evaluation is only performed on one network architecture, without proper justification why on single architecture, this is quite limited. I strongly recommend authors add more diverse architectures in the study, at least on CIFAR and Tiny Imagenet datasets.\n\n- It lacks a study of the efficiency of the proposed methods, please add the discussion and results regarding this. This could be very important to know what the cost needs for the gain.\n\n- Another concern is the selection of the feature extraction part and classification part. The setting in the paper is a reasonable choice, but it would be important to justify whether this is the only choice and the best one, which may not necessarily be the best from my perspective.\n\n- Besides the performance advantage, it is more important to show what new stuff AFD brings, making it different from adversarial training and TRADES. One way might be to show the inference results difference by AFD and the other two methods, and analyze why it is the case. Since there are already quite a few methods there, when proposing a new one, it is very important to justify \n\nwhat makes the newly proposed method unique from existing ones, besides the performance enhancement. \n\n- An important claim of this paper is that the learned feature is robust. It would be necessary to perform an evaluation to support this claim, e.g., the authors could calculate the pairwise feature-based distance before and after the attack?\n\nI think this paper is working in a promising direction and hope more supporting results could be provided to make it more comprehensive.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Please refer to the main review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7 hours"}, "extra_text": {}}, {"review_id": "XPHCOaDK3EX", "reviewer": "Reviewer_nZT8", "summary": "This paper applied domain adaption ideas into the adversarial settings. Based on it, they proposed Adversarial Feature Desensitization (AFD) by leveraging a discriminator network to minimize the  distance between adversarial feature and natural feature. They show that their method could improve the robustness  compared to TRADE and AT and could also generalize well to other attacks (with different epsilon).  ", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main drawbacks stopping me to accept this paper at this moment are as follows.  \n1. The computation cost of this method. Although the show their method on Tiny-ImageNet, but it is also important to show the results on large-scale ImageNet  dataset. Additionally, in Tiny-Imagenet, their performance is not as good as TRADE.  \n2. The effectiveness of their methods. In CIFAR-10, I do not know why AFD-DCGAN show poor performance. It is important to explain the reason. \n3. The evaluation  is not sufficient. To evaluate the robustness of their methods, it is also important to evaluate the performance with Autoattack. Additionally, I also suggest the author to evaluate their method by using stronger blackbox attack method (e.g.  SPSA) . ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "SVfaHRkdDRi", "reviewer": "Reviewer_inuZ", "summary": "This paper proposes a new defense mechanism against adversarial examples (AE). It first formulates an upper bound of the adversarial effect using a distribution distance from domain adaptation, and then designs a defense playing a game with 1) an domain discriminator the adversarial/natural domain and 2) a feature extractor that tries to make the adversarial/natural inputs indistinguishable. ", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Clarity**\nThe paper is well-written and motivated. The theoretical parts are easy to follow and are supported by running examples. Experiment description has included enough details for people to understand the setup with little doubts.\n\n**Originality**\nThe techniques are not ground-breaking new: GAN has been used in AE generation/detection; there's also work investigating AE from the adv. distribution perspective too. However, the paper has done a great job applying these concepts in a coherent way and ends up with a well-explained and easy-to-follow defense mechanism. It adds a new defense with SOTA performance, and I learned something new from the paper.\n\n**Quality and Significance** \nThe experiments illustrate 1) the performance advantage of the new mechanism, 2) the promised dimension reduction effect, and 3) the quality of the upper bound estimation. The experiments are well designed for a conference paper. \nThe theoretical part, as the authors have self-identified in the limitation, may not directly guarantees the performance due to the heuristic nature of the algorithm. Nevertheless, it provides some good insights.\nOverall, this paper is of good quality.\n\n**Questions and More Detailed Comments**\n\n1) The caption of Table 1 mentioned reporting both the mean and the std. However, only the mean is reported. Is it the std reported elsewhere?\n\n2) I feel the discussion of relation to AT is slightly over-stated. In particular, the actual AFD also requires an AE generation process, which may not be perfectly finding the \"strongest\" AE either. So instead of finding the upper bound, the AFD heuristic is getting a \"lower bound of the upper bound\", which in turn may not be necessarily better than the lower bound of AT. Maybe this part can be tuned down a bit.\n\n3) How well is the domain discriminator doing at the end of training, against seen/unseen attacks? How does the result compare with the existing SOTA AE detectors?\n\n3) If the submission is accepted, I highly encourage the authors to release the code and present them in a modular way so that other researchers can readily test this framework with more AE generation processes or GAN structure in the future. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitation and societal impact are sufficiently discussed.", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical concern.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "2Yt2O1I3um", "reviewer": "Reviewer_1Prj", "summary": "This paper proposes a new framework for adversarial defense. The core idea is to develop a GAN-style training strategy that encouraging the model to distinguish the features of natural and adversarial examples.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed framework is novel and the intuition behind is very interesting. I particularly like the comparison to adversarial training, which explains why the proposed approach works better.\n\nThe improvement over adversarial robustness is very good, and we should be cautious about the results. I encourage the authors to use AutoAttack (https://github.com/fra31/auto-attack) to test their models. The implementation should be very easy as long as you provide the proper API. Although the authors include results for many types of attacks, some of them are outdated and easy to be fooled, while attacks in AutoAttack are often considered comprehensive. It is also easy for readers to compare to other works as they maintain a leaderboard.\n\nI am also interested in the computational complexity of this framework. Essentially it also requires adversarial example augmentations, therefore the computational overhead should be similar. It raises question that recent works in adversarial training are able to use single-step attack for the training. Could similar thing be done for this framework as well? It will be interesting to see as it could help scale this framework to ImageNet.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper is very much on the border. One reviewer votes for acceptance, two are marginally above the threshold and one is marginally below with several concerns. After inspecting the paper by myself, I lean toward acceptance mainly because the paper manages to propose a technically sound and well explained defense mechanism from a novel point of view. This could potentially facilitate the development of the challenging problem of improving adversarial robustness.

Nevertheless, the concerns raised by reviewers, especially in higher computational cost and marginal improvements over AT and TRADES when evaluated by strong attacks, are still valid. I strongly encourage the authors to add the new results and properly discuss the possible limitations of the proposed method when preparing for the final version.
Decision: Accept (Poster)