Self-Supervised Adversarial Example Detection by Disentangled Representation
Authors: Zhaoxi Zhang
Leo Yu Zhang
Xufei Zheng
Shengshan Hu
Jinyu Tian
Jiantao Zhou
Venue: NeurIPS 2021 Submitted (2021)
Keywords: Adversarial examples
adversarial detection
deep neural network
disentangled representation

Abstract: Deep learning models are known to be vulnerable to adversarial examples that are elaborately designed for malicious purposes and are imperceptible to the human perceptual system.  Autoencoder, when trained solely over benign examples, has been widely used for (self-supervised) adversarial detection based on the assumption that adversarial examples yield larger reconstruction error.  However, because lacking adversarial examples in its training and the too strong generalization ability of autoencoder, this assumption does not always hold true in practice.  To alleviate this problem, we explore to detect adversarial examples by disentangled representations of images under the autoencoder structure.  By disentangling input images as class features and semantic features, we train an autoencoder, assisted by a discriminator network, over both correctly paired class/semantic features and incorrectly paired class/semantic features to reconstruct benign and counterexamples. This mimics the behavior of adversarial examples and can reduce the unnecessary generalization ability of autoencoder. We compare our method with the state-of-the-art self-supervised detection methods under different adversarial attacks and different victim models ($30$ attack settings), and it exhibits better performance in various measurements (AUC, FPR, TPR) for most attack settings. Ideally, AUC is $1$ and our method achieves $0.99+$ on CIFAR-10 for all attacks. Notably, different from other Autoencoder-based detectors, our method can provide resistance to the adaptive adversary.

Self-Supervised Adversarial Example Detection by Disentangled Representation
Anonymous Author(s)
Affiliation Address
email
Abstract
Deep learning models are known to be vulnerable to adversarial examples that are elaborately designed for malicious purposes and are imperceptible to the human perceptual system. Autoencoder, when trained solely over benign examples, has been widely used for (self-supervised) adversarial detection based the assump- tion that adversarial examples yield larger reconstruction error. However, because lacking adversarial examples its training and the too strong generalization ability of autoencoder, this assumption does not always hold true in practice. To alleviate this problem, we explore to detect adversarial examples by disentangled represen- tations of images under the autoencoder structure. By disentangling input images as class features and semantic features, we train an autoencoder, assisted by a discriminator network, over both correctly paired class/semantic features and in- correctly paired class/semantic features to reconstruct benign and counterexamples This mimics the behavior of adversarial examples and can reduce the unnecessary generalization ability of autoencoder. We compare our method with the state-of- the-art self-supervisee detection methods under different adversarial attacks and different victim models (30 attack settings), and it exhibits better performance various measurements (AUC, FPR, TPR) for most attack settings. Ideally. AUC is and our method achieves 0.99+ on CIFAR-10 for all attacks. Notably, different from other Autoencoder-baan detectors, method can provide resistance to the adaptive adversary.
10
11
12
14
16
18
20
Introduction
21
In 2013, the seminal work reported that, during model test time, deep neural networks can be easily fooled by adversarial attacks that add tiny perturbation to inputs. Since then, adversarial 24 attacks and defenses have drawn significant research attention (2-9). On the one hand, attackers are
22
25 persistently developing new strategies construct adversarial examples; on the other hand, defenders
26 are struggling to cope with all existing and forthcoming attacks [10]
Most of the existing defense methods 9 28 well when defending against adversarial attacks they were originally trained for. However, is widely regarded that supervised methods cannot generalize well to adversarial examples from (existing) unseen attacks, let alone examples from new attacks.
trained with supervision, and these methods work
30
Self-supervised based defense, in comparison with supervised defense, requires only benign examples 32 for its training. a typical example, the works in [12 utilize the encoder of an autoencoder (AE) to draw the manifold of benign examples and then the decoder network for reconstruction, as shown in Fig. [a). Since the manifold is learnt from benign examples only and the AE is trained to minimize Reconstruction Errors (RE) for benign examples, thus the encoding of adversarial example
31
34
36
will likely be cut-of-distributiin and the associated RE is larger.
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.

---

feature
Pade
benignniggn
Pore
Kado
Fude
En
De
De
pemurotte
benign
samantic feature
(a)
(b)
Figure 1: Different adversarial example detectors, Here, En stands for the encoder, De the decoder, V victim model. (a) AE based method draws a manifold (the light gray area) of benign examples: (b) DRR disentangles images as two features: class features which represent images' labels (vulnerable to adversarial perturbation) and semantic features (robust adversarial perturbation). When decoding class and semantic features, benign examples can be reconstructed faithfully but adversarial examples cannot.
It is soon realized that this is not always true because AE has very strong generalization ability [14) examples with various kinds of small perturbations can be reconstructed with small RE This is desirable if the perturbed examples perturbed versions of benign examples and the malicious perturbations can be also made very small in many attacks (i.e., now their encodings will reside in the light gray area of Fig. [a)). When this happens, all REs are mixed and it leads to high false negative or false positive rate (FNR/FPR) during detection. To refine the volume of the manifold drawn by AE and reduce its unnecessary generalization ability on adversarial examples, there exist a number of variants [5 as will be reviewed in detail in Sec. 2.2
38
39
benign. However, adversarial examples
just specific
40
42
43
44
45
46
As a better solution to this problem, we propose a sulf-supervised disentangled representation based reconstruction (DRR) method to detect adversarial examples. DRR possesses the ad advantage of supervised defense, even though it does not have access to any adversarial examples in training. This is achieved through mimicking the behavior of adversarial examples by encoding and decoding special class of examples (counterexamples in this work), which is the reconstruction of semantic feature from one example and class feature from uncorrelated example. The rationale is based on the very fact of adversarial examples: they cause misclassification without changing semantics (contained in its benign counterparts).
48
50
51
52
53
54 With this fact, we disentangle, with the help of the victim model and an auxiliary encoder, the representation of images two parts: deesndee and Mlass-indpennden (i.c., class feature and semantic feature). After disentangling, we then train a decoder to reconstruct benign examples by combining class/semantic features from the same benign image, well as counterexamples by combining class/semantic features from different benign images (i.e., images with different labels). During detection, as shown in Fig. [b). DRR will faithfully disentangle any benign image paired class/semantic features and its adversarial counterpart to unpaired class/semantic features, whose associated REs are significantly different as desired.
56
58
60
61
62
This paper makes the following contributions: 1) We use disentangled representation for adversarial detection, which makes it possible for the detector to mimic the behavior of adversarial examples in the self sepe-supe framework. 2) We design DRR via an AE structure, but reduces the unnecessary generalization capability of AE adversarial examples. 3) We achieve state-of the----rttttttt adversarial detection performance on MNIST, Fashion-MNN MNIST, and CIFAR- in most cases. Specifically, the Area Under Curve of Receiver Operation Characteristic (ideal AUC of ROC value is 1) is 0.99+
64
66
68
CIRFAR-10. 10.
2
Background and Related Works
70
2.1
Constructing Adversarial Examples
The existence of adversarial examples in deep neural networks is first pointed out by (T]. who find maliciously designed imperceptible perturbations can fool deep models to misclassify. Let F(.) be a general neural model and C(-) be the layers before softmax of the model, then evaluating the test example x is simply a softmax classification over the logits c = C(x)), i.e., F(x) = softmax(c). The (untargeted) adversarial example &adv derived from satisfies
Tady Sadv. F(Zadv) * F(x), and [|dadv]
(I)

---

71 where dadv is the adversarial perturbation, is the maximum magnitude of éadv under certain norm
72 compliance. Commonly used norm could be L1, L2. and Lint, so we also focus on detecting adversarial examples bounded by these norms.
79
In the current literature, Fast Gradient Sign Method (FGSM) is the first widely used method in
75 generating adversarial examples (2). Basic Iterative Method (BIM) [19] and Projected Gradient
78 Descent (PGD) (5] improve FGSM by iterating the building block of according to different criteria
77
to find the optimized perturbation. DeepFool (3 and CW 4 do not directly rely on gradient but they
78
instead optimize (the norm of) ōadv. which are usually more stealthy than FGSM and its variants.
79
Another line of research for constructing adversarial examples is called adaptive attacks (20). Under an adaptive attack, the attacker also has white-box access to possible defense mechanisms, and his goal is to find an optimal davv that solves Eq. (T) and circumvents the defense mechanisms simultaneously. All the attacks above have their adaptive versions by considering different. defenses.
80
81
88
84
2.2 Detecting Adversarial Examples
85
Adversarial detection an effective way to prevent adversarial examples, and it can be classified as supervised and unsupervised methods considering whether adversarial examples are needed for training. For the case of supervised detectors, their detecting capability depends on how to capture the differences between adversarial and benign examples. Techniques range from studying statistical
86
87
88
89
properties [6 training traditional machine learning classifiers [21[22] and deep classifiers [T It is widely accepted that supervised methods cannot generalize well to adversarial examples produced by unseen attacks. For example, a supervised detector trained with PGD adversarial
90
16 23
91
examples is likely to fail to detect examples produced by CW. However, it is also known that supervised detectors are robust to adaptive attacks, i.c., detector trained with PGD will likely detect examples produced by the adaptive PGD attack (1]
93
94
95 For the case of unsupervised detectors, their detecting capability depends on how to embed and represent benign examples to different manifold (other than the natural spatiotemporal domain) thus adversarial perturbations will be magnified with embedding (without even seeing them at all). MagNet, proposed in [12). firstly uses an autoencoder to draw the manifold of benign examples, and it is regarded that the distance between the adversarial and benign examples are large via embedding and reconstruction.
97
98
89
100
101
However, the embedding manifold induced by autoencoder is not always desirable for detection, since autoencoder has too strong generalization capability (14). Further to MagNet, the work [15)
102
103
trains a variant of autoencoder by adding logits of the victim model into the loss function to refine
the volume of the embedded manifold. The work (16) proposes to directly use parameters fixed
105
victim model as the encoder, and the victim models' logits as high-level representations/mmbdddig Different from the works above which treat the manifold of all benign examples as whole, [17] [8]
107
propose a class-conditional model to embed and reconstruct benign examples for even better refining the (embedding) manifold. However, it is reported in [17] that this method still only reacts to relatively large (adversarial) perturbation over simple datasets. Compared with supervised detectors, self supervieed detectors may generalize better to unseen attacks but they are vulnerable to adaptive
109
110
attacks [12] 16). For example, it is shown in 24 that the detector of [12 fails to resist adaptive attacks all.
111
112
113
2.3 Disentangled Representation
Disentangled representation is firstly advocated by InfoGAN in (25), which encourages the learning of interpretable and meaningful representations of inputs for manipulating specific features. The work in (26 disentangles speaker-related representation to synthesis voices of different speakers. The work in 27 uses disentangled representation to address pose discrepancy problem among face images.
115
117
118
119
This paper uses disentangled representations for adversarial detection. It is pointed out 28 that there are many features in an image, and in adversarial attack, not all features are equally easy to be manipulated. Specifically, the class an image belongs to, which does not depend on all semantic features it has, is a concrete example. In this concern, we disentangle representations of an image
120
121
122
3

---

rloo Ieaturn
class
Semmantic Ieature
Figure 2: Overview of DRR.
123
into semantic feature and class feature. Semantic features can be easily manipulated by the attackers (through adversarial perturbation) but the perturbed features are similar the original version (even after embedding if the perturbation is tiny enough). In contrast, class features cannot be directly manipulated (by attackers), but they are fragile adversarial perturbation and robust to natural perturbation of the semantic features.
124
125
128
127
128
3
DRR for Adversarial Detection
129
This section presents the details of how to train DRR and use DRR to detect adversarial examples. For detection, as discussed earlier and shown in Fig. (bb), an incoming test example x will be encoded and decoded as z', then the RE || =|I2 is compared to threshold value. If the RE is larger, then is considered as adversarial; otherwise, it is not. Ideally, this threshold should be universal (on given dataset) for all possible attacks (even for new attacks). However, for existing detectors, the threshold value related to the used attack methods. This makes it hard to determine a universal threshold for existing detectors and to compare with them. For this reason, use ROC curve of each attack strategy as the metric for evaluation in Sec. [ But is worth mentioning that it is easy to set universal threshold for DRR for all the considered attacks.
130
132
134
136
138
We then move to how to train DRR. As depicted in Fig. 2 assisted by the parameter-fixed victim model V. DRR first extracts class and semantic features from one benign example to train the encoder En and decoder De. Then DRR takes the class feature from another benign example zy (j * i and z;'s label is different to 2;)) and combines it with the semantic feature from to mimic the behavior of adversarial examples. The two encoding-decodiigg processes are further enhanced with the discriminator D. The details of how to design the loss functions of the modular networks are presented in detail below.
139
140
142
144
145
First and foremost, we examine how to disentangle an input image to semantic feature s and class feature e for our detection purpose. For 8, which represents the semantics of xr we use general encoder network En to derive it, i.e., = En(z). This is because it is widely accepted that high-dimensional input x resides
146
148
low-dimensional manifold, and it a commonly used method in the area of representation learning. For which represents the class feature of xr we choose to use the logits from the victim mode i.e., C = C(x). Aligned with the encoder-decoder structure and the discussion in Sec. 2.3 the rationale for this choice is two-sided: 1) Different from the one-hot encoding of the label x, C(x)) contains richer information for the subsequent decoding process; 2) Regardless of how the concrete adversarial example Iady is constructed. C(xadv) must be different enough from C(x) to induce the final erroneous classification.
149
150
152
154
With these features available, a decoder network De is then used to reconstruct x' from and e (of z). ie.,, x' = De(s, c). The natural requirement for En and De that, for generic image z; from the benign dataset, the reconstructed version ær should be similar to z;. Moreover, we need to reserve the generalization capability over benign examples. For this purpose, an Gaussian noise vector n,, each of the i.i.d. component follows (0, 8²), is used for data augmentation, i.e., d; I; + n;, Thus, the
We hereinafter abuse the notation V in Fig. to denote victim model without the final softmax layer as this will not cause ambiguity.

---

associated loss is
L = E,, (2,, E.),
(2)
where o = De (En( C(.:)), MSE is the mean squared error and E is the expectation.
Different from the normal AE based detection and its variants, also require the class features, after encoding and decoding of x; or its noisy version i;, are still aligned. So, another loss function associated with z; is
L2 Ex, [MSE(C, c)) + MSE(C, 4] where c; C(x,). C(z)), C(a.) and & = C(²)).
158
worth mentioning that use the layer normalized version of ca/c/ and ég to calculate this cost in the experiment to avoid numeric instability and ease the task of hyperparameter tuning. This finishes our discussion of how to train En and De with paired semantic feature and class feature from benign examples (the red box with solid line in Fig. -
157
158
159
180
161
We then move to the training of DRR with unpaired semantic feature and class feature, as depicted by the red box with dashed line in Fig. 2 As emphasized in Sec. [.2] the drawback of AE based detection is AE generalizes too well and the refinement of the manifold drawn by AE is not always effective, especially on attacks that directly optimize the norm of the adversarial perturbation. The solution is now straightforward since we can mimic the behavior of adversarial examples by constructing counterexamples from unpaired semantic feature and class feature to better refine the manifold drawn by En and De.
183
184
185
186
167
For another benign example zy with # we extract its class feature via Cj C(x,), To obtain counterexample, the semantic feature 8; from x; and the class feature Cj from x are then passed to De get o = De (En(z:;; C(ar)). Recall that our original purpose is to refine the manifold drawn by the encoder and decoder, so the loss function here should satisfy the following two requirements:
169
170
171
The decoded v'., should not converge to x (and thus x.) because convergence indicates that counterexamples and benign examples are mixed, which is contrary to our original detection purpose;
173
175
The decoded o',, should not be far away from x since out-of-distribution examples do not really improve the detection capability of autoencoder.
176
For these reasons, we choose to use a soft hinge function as the loss, i.e.,
Lg
Ex,
max(0, d -1 =2il12)
177 where d is a hyperparameter used to control the farthest allowable distance between counter and
178 benign examples. This soft hinge function (passively) meets the second requirements list above: its 179 gradient equals when 1=4,5 2/112 > d.
To further reduce the unnecessary generalization capability of the encoder-decoder network, we enforce class feature consistency over the counterexamples and their noisy versions. This achieved by using the following loss function:
14 Ex,
Σ J
MSE( Cj (g)
(5)
180 where C, = De(e(.))) with Kt, = 21 tm (the component of n. also follows the Gaussian distribution (0, ))
Inspired by 229]. we treat the overall encoder-decoder (though assisted with the parameter-fixed victim model V) as generator and then couple it with the last component of DRR, i.e., discriminator network D. (En+De) and D form the structure of generative adversarial networks (GAN), whose loss function is
LGAN
En, [log D(r;)) )]
+Ia,,, [log(1 - D DDDe(En(z;, V(x))))) LL,,, [lo((1 D(De(En(z.), ((z)))))
5

---

182
The goal of this loss is to encourage the decoded examples, either benign or counterexamples, to be 183 indistinguishable from the original imput. is worth mentioning that this GAN framework can be though they do not use disentangled representation, to refine the manifold drawn by encoder-decoder networks.
directly incorporated to all other encoder-decoder--fg like detectors [2 13 15 18
185
In summary, the loss function to train the encoder En, the decoder De, and the discriminator D is
LOss Σ
X,L AGAN
(6)
1=1
186
where X; 1,...
.4) and AGAN control the relative importance of each loss function. En, De, and D obtained by solving the minmax problem arg minlgn,e maxp-
187
188
The last observation we made is that even though the training of DRR mimics the behavior of adversarial examples, it does not really see any of them. And an empirical fact is that the victim model is normally confident on benign examples. In contrast, it is not mandatory for to be confident on adversarial examples (e.g.. max(softmax( ((tss
190
0.2 << max(softmaxx(((m)) leads to trivial failure of DRR as DRR relies on class feature c C(x) for reconstruction. As a remedy of this problem, we use the class feature sharpening trick here, ie.. is updated as
0.9). This
192
193
a c- (a 1) mean(c), if std(c) <;;; others,
c,
where sld is the standard deviation function, is the standard deviation of the training benign examples, and a, an empirical sharpening constant, set to 3.
195
Table I: Detailed settings the experiment.
MNIST Fashion-MNIST
CIFAR-10
Optimization method
Adam
Adam
Learning rate
0,0002
0,0002
Training dataset size
60K
50K
Testing dataset size
10K
10K
Victim model
8-layer CNN
VGG-16
Encoder output size Ag. Ag. AGAN d, 8
100,
100, 1, 3,
0.5. 0.3
0.35, 0.1
Experimental Results
196
In this section, we assess the performance of DRR on three datasets, MNIST (30). Mashion-MNIST
198
31) and CIFAR- 10 (32), by comparing it with other state-of-the-art detectors 12 15 16] against the adversarial attacks FGSM, BIM, PGD, DeepFool and CW (taken from Foolbox (33) and the
200
adaptive PGD under differnt norms (L,, L2 and Liat). As proof-of-concept, two representative
networks, 8-layer CNN and VGG-16 34 are used as the victim models. These settings generate (24 + 6) attacks and (24 x 5) + defenses, so only the representative results are reported here due to
202
space limit and the complete results (including the source code) are provided in the supplementary file.
204
We use a GeForce RTX 2080 to conduct the experiments.
205
4.1
Settings
DRR has three modules: encoder En, decoder De and discriminator D. It is suffice to say that the encoder En and the discriminator D have the same architecture as the victim model except the last fully connected layer: for CIFAR-10 10, there are 128 neurons in the last layer of En; for MNIST and Fashion- -NISS, only 4 neurons are used. The architecture the decoder De is simply a mirror En.
DRR:
207
209
Following the literature 23), the hyperparameters [A,] and AGAN of the Loss are determined by a binary search in [10 31003,1 10% and and 5 are determined empirically. The full details are listed in T
211
213
Methods for comparison: As mentioned above, three kinds of self-supervised detectors 112 16 are used for comparison. The structure of the encoder, the decoder for all these three methods are the same as of DRR. These methods can be classified as hidden vector based reconstruction (HVR) and high-level representation based reconstruction (HLR).
215

---

Table Test sccurncy of the victim models.
Table AUC of ROC of difrerent detectors over different datasets.
Data
Attack
Accuracy
Datu
Norm
HVR-P
HLR-P
HVR-L
HLR-L
DRR (ours)
benign
0.869
BIM Lue
0.01
0.4755
0,6340
0.4766
0.9039
0,9992
CIFARIO
PGD = 0.01
BIM laat e = 0.01
PGD La 0.1
BIM Li = 5
0.267
0.744
0.588
0.195
CIPARIO
PGD Lus 0.01
PGD Lg c = 0.1
BIM L, c = 5
0.4704
0.4800
0.4411
0.5996
0.5992
0.6199
0,4469
0.4364
0.4725
0.8568
0.8856
0.8515
0.9975
0,9990
0.9988
PGD la - 0.3
0.352
PGD La 0.3
0.4594
0,6105
0.4613
0.8742
0,9983
PGSM Lup c 0.05
0.092
FGSM Lur 0.05
0.6620
0.7000
0.6430
0.8207
0.9977
DeepFool Fre
0,054
DeepFonl lut
0.5051
0,6518
0.5038
0.8167
0.9954
CW L2
0.001
CW L2
0.5020
0.6562
0.5019
0.8298
0.9982
benign
0.993
BIM Lat 0.3
0.4711
0.8509
0.3972
0.8672
0.9939
BIM Lut 0.3
0.001
BIM Ln = 50
0,7572
0.8876
0.7391
0.8968
0.9714
BIM 4 50
0.012
Lut
0.5226
0.8291
0.8551
0.9970
MNISST
PGD Lue 0.3
PGD L2 = 1
PGD La
0.086
0.724
0.000
MNIST
PGD I2 < 2
PGD 1/2 c = 1
0.2313
0.1645
0,9948
0.9897
0.2149
0,2633
0.9886
0.9915
0.9977
0.9691
FGSM Lar
0.1
0.762
RGSM lue 0.1
0.0724
0.9352
0 1
0.9569
0.9367
DeepFool Lay
0.078
DeepFool Iut
0,3824
0,9653
0,4301
0.9748
0,9815
CW L2
0.000
CW L2
0,7211
0.9871
0.6947
0.9828
0,9750
benign
0.926
BIM l = 0.05
0.5255
0.8295
0,5389
0.8419
0.8880
BIM Lut c = 0.05
0.021
BIM [ 20
0.5420
0,8311
0.5481
0.8419
0.8944
BIM L 20
0.300
PGD Lur c = 0.05
0.5267
0.8439
0,5415
0.8555
0,9000
Foshion
PGD Lug - 0.05
PGD La =
0.240
0.008
Fishion
PGD 12
PGD L2 6 = 2
1
0.5614
0.6972
0.9654
0,9610
0.5728
0.7259
0.9662
0.9709
0,9858
0.9914
FGSM
PGD 122 c - 2
DeepFool Lax
0.05
0.095
0.327
0.052
FGSM lut 0.05
DocpFool Lust
0.6113
0.6137
0.9088
0,9201
0.6270
0.6298
0.9193
0.9273
0.9192
0.9378
CW L2
0.003
CW L2
0.5068
0.8981
0.5013
0.9058
0.9125
217
For [2]. is based on optimizing the pixel-level reconstruction error from the hidden vector outputted by the encoder. We term it HVR-P. For [15), it optimizes both pixel and logits reconstruction егтor and we term it as HVR-L. For 16), it takes the logits from the parameter- fexee victim model as a high-level representation of examples for decoding, and it has two variants, HLR-P and HLR-L, depending on whether the loss function includes the logits reconstruction error or not. We also incorporate and optimize the GAN loss when implementing all these detectors: HVR-P, HVR-L,
218
219
220
221
222
223
HLR-P, and HLR-L. Note that our implementations perform slightly better than the original methods in [5] 16). but this provides a fair environment for evaluating the effectiveness of disentangled representation.
224
225
226
4.2
Results Analyses
227
With the settings above, we first train victim models on the 3 datasets, and then generate adversarial examples with the 5 attacks (FGSM, BIM, PGD, DeepFool, and CW) under different norm compliance (c in Eq. (T). All these test examples, either benign or adversarial, are passed to the detector for inspection.
228
229
230
231
Preparing test examples: The two victim models, VGG- and 8-layer CNN, are trained on CIFAR- 10 and MNIST/Fashion MNIST, respectively. For each test example, we generate the corresponding adversarial versions with attacks under different norm compliance, and the test accuracy is listed in
232
233
Table 2 From this table, it is clear that the victim models achieve 0.869, 0.926, 0.993 classification accuracy over benign test examples. Also from this table, the attacks DeepFool and CW, which directly optimize c, are generally stronger than the other three. By increasing c, other attacks also obtains better attack result (i.e., worse accuracy), e.g., PGD with different L2 constraints in Table 2
235
236
237
238
However, a larger adversarial perturbation will make the attack get detected easier.
239
Visual inspection: We visually compare the reconstructed results of the benign and adversarial examples for all the detectors, some examples are shown in Fig. 3 Inspecting columns 3 and 4 of Fig. 3 a) and (b), it is clear that DRR, HLR-P and HLR-L generally have bigger reconstruction errors over adversarial examples, which validates that logits (of the victim model) do help to reduce undesirable generalization capability of the detectors.
241
243
244
We further box-plot the reconstruction errors of all benign and adversarial examples from CIFAR- 10, the result is shown in Fig. 4 is clear from this figure, under all attacks of CIFAR-10 10. the reconstruction errors of DRR are clearly separable. The reconstruction errors of HLR-P and HLR-L
245
246
247
are also generally separable, but it has higher FNR or FPR than that of DRR (depending on the concrete threshold of RE). This validates that the disentangled semantic and class features can better refine the volume of the embedded manifold and prevent undesirable generalization. Moreoever, as shown in this figure, it is easy to determine universal threshold for DRR to detect all considered
249
7

---

WAL
23R
7
4
4
4
5
1
5
Q
6
4
8
8
(a) MNIST
(b) CIFARIO
Figure Reconstructed images of each detection methods.
25
HUN MUMEP
HOW PUMIHINE
HUR MMM PIN
(a) BIM Lint 0.005
(b) CW La
(c) PGD Tant
(d)
FGSM
Tait
(c) DeepFool L2
0.005
0.01
Figure 4: Box box-plot of the reconstruction errors for CIFAR-10 (red box is for adversarial and blue box is for benign).
08
08
00 Fax.
Dott
Det
162.1
02
02
02
12
14
18
17
2.5
18
28
PM
(a) BIM Tinf 0.005
(b) CW L2
(c) PGD
Lint
(d) FGSM Tant
(c) DeepFool L2
0.005
0.01
Figure ROC curve of all detectors under various attacks on CIFAR-10.
attacks CIFAR-10 (and it is similar for the other two datasets), but it is hard to do for HVR-P, HVR-L, HLR-P, and HLR-L.
252
With benign samples and adversarial examples of CIFAR-10, the ROC curves of all the detectors are depicted in Fig. for qualitative analysis. From Fig. 5 it very clear that
ROC curve and AUC:
254
DRR can retain high true positive detection rate (TPR) while keeping the FPR rate low. If the size of the AUC ROC curve is larger, the detector is better and the ideal size can reach up to 1.
256
We use AUC of ROC to quantitatively study the effectiveness of all detectors over CIFAR-10, as well as extending this metric to MNIST and MaShion-MNIST. The results are tabulated Table 3 clear that, for CIFAR-10 10, the AUC of DRR is the largest over all different attacks, and the value is very close to the ideal case For MNIST and Mushion-MNIS. DRR outperforms other detectors in most cases. For the cases that DRR is inferior performances are comparable to each other.
258
280
the gap is tiny and we regard their
282
263
Defensing Against Adaptive Adversarial Attack
To further evaluate the performance of DRR, assume that the attacker can not only access the victim model but also knows the details of the detector. Under this adaptive assumption, the attacker's goal is to fool both the victim model and the detector. Following the most-widely used adaptive attack strategy [8 23] the attack now aims to solve
min ALRE(KaavV) LCE( 8) das
s.t. || =|I, <e, p E 21,2
284 where R and Lcr are respectively the attack's function for the detector (i.e., reconstruction error) and for the victim model (i.c., cross entropy), and F(x) is the true label of and is the

---

023
066
020
N0L 205
04
PG2
070
(a) L1
(b) c 10
(c) L2 0.1
(d) L2 0.3
(e) List 0.01
022
035
022
2004
5000
2052
093 501
652,
056
00. 307
0Z2
003 100
10
100 340
140
0.5
0.3
11
02
25
(1) L1 c= 10
(g) c1
6= 20
(h) L1 6=
(i)
L2
( Line 0.3
Figure 6: Accuracy of victim model under the original PGD attack and the adaptive attack with different a. The first row is for CIFARIO, the second row is for MNIST.
266
parameter to control the relative importance of the two loss functions. It is worth mentioning that the
267
detectors 12 15 16 in comparison fail to resist the adaptive attack defined above The reason is, for
268
other based detectors, no matter high-level representation (i.e., logits) is used or not, they fail
269
to capture true features of adversarial examples by certain. In contrast, the AE in DRR mimics the behavior of adversarial examples via disentangled representation.
270
We evaluate this adaptive attack strategy under the same setting of Table 1 and the results for 272
CIFAR-10 and MNIST are depicted Fig.[6 Observing the first row of Fig. 6 it is clear the accuracy of victim model under adaptive attacks is higher than that without our defense, a.k.a., it is harder for the attacker to succeed when the detector DRR is deploved. Moreover, if the attacker focuses more on circumventing the detector by increasing a of Eq. (8) (from 102 105), it will be harder for him to succeed in constructing real adversarial examples. However, if the attacker puts less focus on bypassing the detector, further experimental results verify that it makes him easier to get detected.
274
275
276
277
278
similar trend can be also observed when implementing the adaptive attack on MNIST (the second row of Fig. (). But from this figure, it is clear that now it is harder for the attacker to succeed in attacking. for example, the attacker only has about 10% attack success rate (under L2 and L2 norm) 102 (less attention on circumventing the detector). We regard this is because the semantic feature of MNIST is simple, and the network architecture used by DRR (Table [ captures
280
even with small
282
283
this feature very well and the defense is stronger (than that of CIFAR- 10 which has more complex semantic features). In this concern, it is reasonable to speculate that DRR can be extended to other complex datasets with an appropriate choice of model architecture to capture the complex semantic
284
285
286
features within the dataset, which we leave as future work.
287
5
Conclusion
286
In this study, we propose to make use of disentangled representation for self-supervieed adversarial examples detection. The proposed method DRR is based on the very nature of adversarial examples: misleading classification results without changing images' semantics (a lot). With disentangled class and semantic features, this nature inspires to construct counterexamples to better guide the training of DRR. Compared with previous self-supervised detectors, DRR generally performs better under various measurements over different datasets and different adversarial attack methods. Not surprisingly, compared with other AE-based adversarial detectors, DRR is also more robust to adaptive adversaries. This makes DRR a promising candidate, when combined with other proactive strategies, for the defense of adversarial attacks. Moreover, disentangled representation is a ubiquitous property for many other data formats, including natural language, voice, and video signals, DRR may be extended easily to provide adversarial defense to other domains.
290
292
294
296
298
We note that 15 considered weaker adaptive attack strategy (maximize the cross entropy after AE reconstruction but not directly optimize reconstruction error) and HVR-L is robust the weaker adaptive notion.

---

References
299
300
[1] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, "Intriguing properties of neural networks," ICLR, 2013.
302
[2] J. Goodfellow, J. Shlens, and C. Szegedy, "Explaining and harnessing adversarial examples," " ICLR, 2015.
304
[3] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard. "DeepFool: A simple and accurate method to fool deep neural networks," in CVPR, pp. 2574-2582, 2016.
305
306
[4] N. Carlini and D. Wagner, "Towards evaluating the robustness of neural networks," in Oakland, pp. 39-57,
307
2017.
308
[5] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, "Towards deep learning models resistant to adversarial attacks," in JCLR, 2018.
309
310
[6] K. Roth, Y. Kilcher, and T. Hofmann, "The odds are odd: A statistical test for detecting adversarial examples," in ICML, pp. 5498-5507, 2019.
311
K. Grosse, P. Manoharan, N. Papernot, M. Backes, and P. McDaniel, "On the (statistical) detection of adversarial examples," arXiv preprint arXiv: 702006280, 2017.
313
[8] K. Lee, K. Lec, H. Lee, and J. Shin, "A simple unified framework for detecting outoof-distribution samples and adversarial attacks," in NeurIPS, pp. 7167-7177, 2018.
315
318
[9] D. Hendrycks and K. Cimpel, "Early methods for detecting adversarial images," ICLR, 2016.
317
[10] N. Carlini and D. Wagner, "Adversarial examples are not easily detected: Bypassing ten detection methods," in CCS, pp. 3-14, 2017.
318
319
[1I] Tian, J. Zhou, Y. Li, and Duan, "Detecting adversarial examples from sensitivity inconsistency of
320
spatial-transfomm domain," in AAAI, 2021.
321
[12] D. Meng and H. Chen, "Magnet: A two-pronged defense against adversarial examples," in CCS, pp. 135-
322
147, 2017.
323
[13] C. Chen, J. Liu, Y. Xie, X. Ban, C. Wu, Y. Tao, and H. Song, "Latent Regularized Generative Dual Adversarial Network For Abnormal Detection," in IJCAI, (Califormia), pp. 760-766, 2020.
324
[14] D. Gong, L. Liu, V. Le, B. Saha, M. R. Mansour, S. Venkatesh, and A. d. Hengel, "Memorizing normality
326
to detect anomaly:
aaggeete deep autoencoder for unsupervised anomaly detection," in CVPR, pp. 1705-1714, 2019.
327
[15] G. Vacanti and A. Van Looveren, "Adversarial detection and correction by matching prediction distributions," arXiv preprint 200 2020.
329
[16] B. Wojeik. P. Morawiecki, M. Smieja, T. Krzyžek, P. Spurek, and J. Tabor, "Adversarial examples detection and analysis with layer-wise autoencoders," " arXiv preprint arXiv:2006. 10013,, 2020.
331
332
[17] Y. Qin, N. Frosst, S. Sabour. C. Raffel, G. Cottrell, and G. Hinton, "Detecting and diagnosing adversarial images with class-conditional capsule reconstructions," in ICLR, 2019.
334 [18] L. Schott, J. Rauber, M. Bethge, and W. Brendel, "Towards the first adversarially robust neural network model on MNIST," in ICLR, 2018
336
[19] A. Kurakin, I. Goodfellow, and S. Bengio, "Adversarial machine learning at scale," ICLR, 2017.
337
[20] N. Carlini and D. Wagner, "Adversarial examples are not easily detected: Bypassing ten detection methods." in Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3-14, 2017.
338
339
[21] Lu, T. Issaranon, and D. Forsyth, "Safetynet: Detecting and rejecting adversarial examples robustly," in ICCV, pp. 446-454, 2017.
340
[22] R. Feinman, R. R. Curtin, S. Shintre, and A. B. Gardner, "Detecting adversarial samples from artifacts."
342
arXiv preprint arXiv:1703.0044 2017.
[23] X. Ma, B. I., Y. Wang, S. Frfani, S. Wijewickrema, M. Houle, G. Sehoenebeck, D. Song, and J. Bailey, "Characterizing adversarial subspaces using local intrinsic dimensionality." ICT.R, 2018.
344
345
[24] N. Carlini and D. Wagner. "Magnet and "efficient defenses against adversarial attacks" are not robust to
adversarial examples," arXiv preprint arXi:: 2017.
347
[25] x. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, "Infogan: Interpretable representation learning by information maximizing generative adversarial nets," in NeurIPS, pp. 2172- 2180, 2016.
348
350
[26] M. Sang, W. Xia. and J. H. Hansen, "Deaan: Disentangled embedding and adversarial adaptation network for robust speaker representation leaming," arXiv preprint arXiv:2012.06896 2020.
10

---

352
[27] L. Tran. X. Yin, and X. Liu, "Disentangled representation learning gan for pose-invariant face recognition." in CVPR, pp. 1415-1424, 2017.
354
[28] A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry, "Adversarial examples are not bugs, they are features," in Newr/PS, pp. 125 136, 2019.
355
358
[29] C. Xiao, Li, J. Zhu, W. He, M. Liu, and D. Song. "Generating adversarial examples with adversarial networks." in I/CAI. pp. 119-128, 2018.
357
[30 Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324 1998.
359
380
[31] H. Xiao, K. Rasul, and R. Vollgraf. "Fashion- -MIiiIT-MMIITTT... A novel image dataset for benchmarking machine learning algorithms." arXiv preprint arXiv:1708.07744 077447 2017.
362
[32] A. Krizhevsky, G. Hinton, et al.; "Learning multiple layers of features from tiny images," 2009.
[33] J. Rauber, W. Brendel, and M. Bethge, "Foolbox: A python toolbox to benchmark the robustness of machine learning models," in ICML, 2017.
364
365
[34] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," "
366
ICIR, 2015.
Checklist
387
388
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
370
(h) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
374
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
378
(b) Did you include complete proofs of all theoretical results? [N/A]
377
3. If you ran experiments...
378
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Please refer to Sec. for the details.
(c) Did you report error hars (e.g., with respect to the random seed after running experiments multiple times)? [Yes]
383
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] Please refer Sec. for the details.
385
386
4. If you are using existing assets (e.g., code, data, models) curating/releasing new assets...
387
(a) your work uses existing assets, did you cite the creators? [Yes]
388
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as URL? [Yes] Code is available the supplemental material.
390
(d) Did you discuss whether and how consent was obtained from people whose data you're 're us- ing/curating? [N/A]
382
383
(c) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content? [N/A]
394
395
5. If you used crowdsourcing or conducted research with human subjects...
396
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
387
[N/A]
396
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
389
400
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
401
11


Reviews:
[{"review_id": "YzhgDLqAsiN", "reviewer": "Reviewer_kDjj", "summary": "The paper proposes a novel adversarial example detection strategy based on disentangled latent representations. The main idea is that latents are separated into class features and semantic features. Detection then uses the fact that adversarial samples preserve semantics while at the same time changing class according to some victim model. Training is being performed through a sophisticated combination of losses accounting for begin examples and counterexamples. Experimental results on detecting various kinds of attacks on popular datasets show clear improvements over previous methods. The paper also presents an evaluation of an adaptive attack.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- Originality: The paper presents a novel combination of common ideas to learn disentangled representations for adversarial sample detection. While previous works in this area have investigated traditional (non-disentangling) autoencoders. Related work seems to be sufficiently citied and discussed.\n\n- Quality: The submission appears technically sound and well organized. The method and individual losses appear well motivated and the experimental evaluation is solid. \n\n- Clarity: \nThe paper is generally clearly written. The presence of a few missing connecting words and typos, these generally do not harm the reading flow. However, I found the methodological section a bit harder to follow at first. One suggestion might be to include an example similar to Figure 2 but using images from CIFAR-10 or MNIST instead of x_i or x_j. I also had the impression that the text in the adaptive evaluation was a hastily written and could need a bit of rework to increase clarity. \n\n- Significance: The experimental setup is sound and the presented results are noticeably better compared to competing approaches. I especially found the AUROC curves impressive. The approach is also reasonably resistant to the adaptive attack presented in the paper.\n\n- Questions:\n1) I would be curious to get further insights into the disentanglement quality. Would it be possible to add further results to the paper that visualize disentanglement quality?\n2) It is apparent from Figure 3 (b) that DRR does not reconstruct the benign sample well while 1-2 other methods do a reasonable job at this task. Granted that faithful reconstruction is not the main task, I wonder whether the authors could comment on this tradeoff between reconstruction quality of benign (!) samples and detectability of adversarial examples a bit more. At the same time, I understand that  the constructed loss is aimed at reducing the generalization capability of the autoencoder to more easily detect adversarial samples.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper includes an adaptive attack in which white-box access to the presented defense mechanism is granted. It would have been great if the paper would also discuss additional limitations of the proposed method (see questions above). ", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "EQviitamJ3", "reviewer": "Reviewer_aJ6q", "summary": "This paper proposes to detect adversary using learned disentangled auto-encoder. The model can disentangle the class feature and the semantic feature. Class features are easily changed by adversary, while the semantic features cannot. Existing Auto-Encoder cannot disentangle those two types of features, and also has strong generalization ability---high performance on both the adversary and the benign examples, making it hard to detect. Using the proposed disentangled AE, benign example can be reconstructed faithfully while adversary examples cannot. Thanks to the tailored AE, the paper state-of-the-art adversarial detection performance on MNIST, Fashion-MNIST, and CIFAR-10 in most cases.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n1. The paper is well motivated by the fact that AE generalize well even under adversary attacks, and propose a novel design to the AE, which disentangles the class features and semantic features, to allow detection of adversary.\n\n2. The paper evaluated on three datasets, MNIST, Fashion-MNIST, and CIAFR-10, over 5 different adversarial attacks. The gain is significant.\n\n3. It is nice to see the method is evaluated under adaptive attacks. The adaptive attacker attacks the detection metric in an end-to-end manner, which is a standard adaptive attack. However, it may be weak due to attacking an auto-encoding task that is high dimension, making the attack less effective, because gradient from multitask (one task for each pixel reconstruction) may cancel out [1].\n\n4. Overall, the paper is clear written.\n\nCons:\n1. Adaptive attack maybe weak. As I mentioned before, auto-encoding task is not a task that is easy to attack due to its high output dimensionality. Thus the author should consider a stronger adaptive attack that attack one component in the model that is less high dimension. For example, instead of attack End2Eend, we can attack the individual component of the defense model, if we have that knowledge.  What if the adaptive attacker attacks(optimizes, corrects) the V model that encode the class feature so that it is the same as the natural label? In this way, the detector should not longer working because class feature is the same as the benign.\n\n[1] Mao et al. Multitask Learning Strengthens Adversarial Robustness. ECCV 2020.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": " The authors adequately addressed the limitations and potential negative societal impact of their work", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "4B3XEUlW1Oh", "reviewer": "Reviewer_xJBn", "summary": "The paper propose a new defense mechanism against adversarial attacks based on an autoencoder architecture. The proposed architecture uses self-supervised learning to disentangle the semantic and class features, which enables the autoencoder to discriminate the clean and adversarial examples at test time. The approach was shown to outperforms other autoencoder approach in the experiments, when the attacker as access to only the victim model or both the victim model and autoencoder.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**:\n- The proposed approach builds on previous autoencoder approach to detect adversarial examples, however the author propose a new architecture and leverage self-supervised learning. \n0The related work seems to be adequately cited and the contributions is put into the context of the previous autoencoder approach.\n\n**Quality**:\n- The experiments clearly show that the proposed approach outperforms the other autoencoder approach. In particular, table 3 clearly shows that DRR has a significantly higher AUC on 3 datasets (CIFAR, MNIST, Fashion) and against different attackers. However, the experiments don't report any confidence interval, it would be nice to have error bars for the experiments. \n- The objective is also composed of many different losses and introduces many new hyperparameters, the role and importance of each losses is not clear, it would be nice to have some ablation studies that shows to which extent each loss terms and how the new hyperparameters affects the performance of the proposed solution. In particular, I don't understand the definition of $L_2$ the first term is a constant and $\\hat x_i$ is not defined. In eq. 4 how important is the hinge loss and the value of $d$. I would like to see an ablation study with $\\lambda_{\\text{GAN}} = 0$, $\\lambda_2 = 0$, $\\lambda_3 = 0$ or $\\lambda_4 = 0$, to see the influence of each loss on the performance of  DRR.\n\n**Clarity**:\n- The paper is overall well written and quite clear. However there is a problem with the labels of the axis and the legend of fig. 6. \n- In Figure 3, you show examples of adversarial example reconstructed that's quite interesting. I'm also very curious about what the reconstruction $x'_{i,j}$ looks like during training, and believe it would be interesting to also plot it, to show if indeed the network is able to disentangle the features.\n- You mention L210-211 that the hyperparameters are tuned using a binary search, it's not clear to me what you mean by that ? \n\n**Significance**:\n-  The proposed approach clearly outperforms other similar approach, however the approach introduces many losses and new hyperparameters without a lot of justification on why such loss are important. I believe this could prevent people from using this new method out-of the box, some ablation study could really help the paper have more impact.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "29AzRCKQnSW", "reviewer": "Reviewer_8oh2", "summary": "This paper presents a method to detect adversarial examples by disentangling the representation. This is done by training autoencoders over both correct and incorrect class/semantic features. The authors claim that the proposed method outperformed other SOTA methods for adversarial detection and even works against adaptive adversarial attacks.\n", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The core motivation of the paper is to distinguish robust features versus non-robust features using the DRR network. The assumption here is that the DRR could successfully disentangle the semantic features versus the class features. However, the establishment of such an assumption is still under question. There is no definition of the metric to measure disentanglment, we could show through statistical metrics such as Mutual information Gap (MIG), SAP score < Chen et al. 2019 Isolating Sources of Disentanglement in VAEs>. \n- Especially in part 3 of the paper, there are many hand-wavy explanations, e.g. line 148. The idea of encoder/decoder structure is quoted vaguely, but it is used as a base logic. In a word, this cannot convince me that DRR is able to disentangle the features. In representation learning, disentanglement is still a very challenging problem as is pointed out by seminal work such as <Locatello et al. 2018> whereas there is no evidence showing that the proposed method could generate any type of compact representation.  (In a compact representation, a given factor is associated with only one or a few code dimensions. <Ridgeway et al. 2018 Learning Deep Disentangled Embeddings\nWith the F-Statistic Loss> From the resulting images in the experiment section and in the appendix, I just could not tell whether the semantic features are different from the class features. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The other question is related to the size of the dataset in experiment, we are using very small datasets i.e. MNIST and CIFAR 10 to train a GAN basically, and I am not sure whether the conclusion would generalize to large-scale datasets such as ImageNet which is 20x larger. \n\nLast thing is about the adaptive attack which the authors added, why are the results on Cifar and MNIST different in terms of epsilon value? The description of Fig 6 is quite confusing, making me think PGD is a stronger attack than the adaptive attack, which shouldn’t be the case. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The manuscript introduces a defense against adversarial examples that relies on disentangling representations (here, disentanglement is not meant in the more common sense of feature disentanglement). On the methodology front, a number of concerns were raised about the ability of the approach to scale to larger datasets (where different classes are more likely to be difficult to disentangle). I also note that all experiments are performed on vision datasets. The current evaluation also fails to provide insights into the different hyperparameters introduced. An ablation study was suggested by the authors but not done in the current manuscript. Finally, the adaptive attack was not tailored to the proposed approach but rather simply evaluates an end-to-end attack. This is a good start but the authors did not provide a convincing argument for why it is sufficient in response to a comment raised in the reviews. In particular, given that the approach would involve a threshold should it be deployed in practice, I encourage the authors to provide evidence that a threshold offering sufficient accuracy on normal inputs while detecting adversarial inputs can be found - even in the presence of an adaptive adversary. 
Decision: Reject