Improved Detection of Adversarial Attacks via Penetration Distortion Maximization
Authors: Shai Rozenberg
Gal Elidan
Ran El-Yaniv
Venue:  (2020)
Keywords: Adversarial Examples
Adversarial Attacks
Adversarial Defense
White-Box threat models

Abstract: This paper is concerned with the defense of deep models against adversarial at-
tacks. We develop an adversarial detection method, which is inspired by the cer-
tificate defense approach, and captures the idea of separating class clusters in the
embedding space so as to increase the margin. The resulting defense is intuitive,
effective, scalable and can be integrated into any given neural classification model.
Our method demonstrates state-of-the-art detection performance under all threat
models.

Under review as a conference paper at ICLR 2020
IMPROVED DETECTION OF ADVERSARIAL ATTACKS VIA PENETRATION DISTORTION MAXIMIZATION
Anonymous authors Paper under double-blind review
ABSTRACT
This paper is concerned with the defense of deep models against adversarial at- tacks. We develop an adversarial detection method, which inspired by the cer- tificate defense approach, and captures the idea of separating class clusters in the embedding space to increase the margin. The resulting defense intuitive, ef- fective, scalable, and can be integrated into any given neural classification model. Our method demonstrates state-of state-of-the-art (detection) performance under all threat models.
INTRODUCTION
Defending machine learning models from adversarial attacks has become an increasingly pressing issue as deep neural networks become associated with more critical aspects of society. Adversarial
attacks can effectively fool deep models and force them to misclassify, using a slight but maliciously-
designed distortion that is typically invisible the human eye (Carlini & Wagner 2017c Athalye
et al. 2018 Szegedy et al.) 20133 Goodfellow et al., 2014 Kurakin et al. 2016). Despite numerous developments, defense mechanisms are still wanting.
Many interesting ideas have been proposed to construct defense mechanisms for adversarial exam- ples. Among these are adversarial training (Zuo et al. 2020 Yan al. 2018 Tramer et al. 2017 Madry et al. 2017). ensemble methods (Strauss et al. 2017)) and randomization (Dhillon et al.,
2018) to name few. These works consider both detection and resiliency. However, many these
defense ideas were found to be inadequate (Athalye al. 2018 Carlini et al. [019 Carlini & Wag- ner] 2017b) He al., (017). For example, adversarial training critically depends on the specific choice of adversarial attacks used to generate the adversarial training instances. As result, often this method cannot withstand attacks based on different strategies. (Engstrom et al. 2018).
A more formal approach to adversarial defense is the certification approach (Hein & An- driushchenko/ 2017), which is designed to provide a lower bound for the penetration distortion required to fool given network. Certified defense methods or "conservative". In exact methods no distortion smaller than the certification bound can penetrate the deep neural network (DNN) (Hein & Andriushchenko) 2017 Wong & Kolter 2017 Wong al.) 2018 Cohen et al., 2019). In "conservative" methods, the bound is merely relative metric for comparing DNN robustness adversarial examples (Tsuzuku
referred to as being either "exact"
2018 Zhang et al. 2019 Ding et al. 2018). Both exact and conservative methods have been criticized for being computationally expensive and unscalable (Tjeng al. 2018 Cohen al. 2019).
It is interesting to view adversarial attacks through activation geometry in embedding layers. A trained deep classification model tends to organize instances into clusters in the embedding space, according to class labels. Classes with clusters in close proximity to one another, provide excellent opportunities for attackers to fool the model. This geometry explains the tendency of untargeted attacks to alter the label of given image to that of adjacent class in the embedding space as demonstrated in Figure Ia Thus, we can modify the model to increase the margin between clusters, while lowering (or not increasing) the activation sensitivity in the embedding space to input changes, we can make the network more immune to attacks. This embedding sensitivity can be quantified through Lipschitz constant or directly via the Jacobian.
Inspired by certificate defense methods, we developed an adversarial detection method, that captures the above separation in embedding space intuition. Ideally, would like lower bound the distor-

---

Under review as a conference paper at ICLR 2020
tion, c, required by the adversary force DNN F to misclassify + where is an imput image. We propose an approximation to such a bound which, while not formal, motivates a useful strategy for creating defense methods. The bound, Z / which is similar to other known bounds, given in terms of 1, where 7 quantifies the "embedding margin" of the network, and Jr (1), the Jacobian of F with respect to x: (see details in Section 2 The embedding margin, for given inter- mediate layer, is the minimal distance (under any p-norm) between two instances belonging to two different classes. This approximate relation motivates strategy of penetration distortion maximiza- tion (PDM) whereby, we implicitly or explicitly maximize this lower bound without attempting to calculate it.
The notion of increasing DNN classification margin has been discussed in several contexts Liu et al. (2016); [Hoffer & Ailon (015); Wen et al. (2016). To apply the PDM approach we propose two procedures to increase the embedding margin. These two methods are complementary in the sense that we can benefit by applying them together. In conjunction, we use the reverse cross- entropy method of Pang et al. (2018), which tends to smooth the Jacobian. Our adversarial detection mechanism is constructed by training a resilient classifier using the above three procedures; we then apply standard kernel density estimation (KDE) the embedding layer (Feinman et al. 2017). We present an extensive empirical study focusing on the detection of adversarial examples under all threat models, in which we consider the FGSM, BIM, C&W and JSMA attacks. Our experimental procedure strictly adheres to the comprehensive evaluation desiderata proposed by Carlini et al. (2019). The results we obtain indicate that the proposed defense achieves state-of-the-art detection.
PENETRATION DISTORTION MAXIMIZATION
In this section, we explain the PDM strategy. Let F a neural classifier and let x hg be an image assumed to have class label c(z). Let e E Rgx be vector representing an adversarial distortion for image & such that the (successful) adversarial instance Fado - x + c whose label is different from namely, Code F(aado) * The attacker's goal is to find the smallest perturbation e such that misclassifies x,
min, ||||
s.t. c(x+c) c(x).
For a successful adversarial attack whose distortion required to be small, in the spirit (Ding et al. 2018: suzuku et al. 2018 Hein & Andriushchenko, 2017] Zhang et al. 2019), approxi- mate a prediction for x Fade using the first-order Taylor approximation
le|11 F(x) Jr (z)e,
F(xado) = F(x +
for vector-valued functions with (x(x) being the Jacobian of F. The same approximation applies to the output of any intermediate layer L. Denoting by Fz(z) the output of layer & we thus have,
FelZaar) x + Je(z)..
For layer f, we define its embedding margin,
A
arg min (
[F(xz) - F+(22)11
Thus,
[J(())|| [[F((x) Fe( m/
(2)
The Frobenius norm used here sub-muuluui uub-mutipllicative (proof can be found in Appendix A namely,
|+)(el)ll 2 [|
Combining (2) and 3 (and ignoring the approximation error) lower bound the norm of the distortion in terms of the embedding margin and the norm of the Jacobian,
||l|| 2 [|](())]|
(4)
2

---

explicitly maximize Under review as a conference paper at ICLR 2020
Airplane
Automobile
Bird
Dear
Dojg
Prog
Horse
ship
Ruck
Airplane
20
Automobile
Bird
Cat
Dog
Horse
-40
Ship
Truck
(a) CIFAR- 10: t-SNE of embedding Layer
sdversaral clsse
(b) Adversarial confusion histogram
Figure 1: Histogram of origin and target classes from C& W untargeted adversarial attack compared
embedding layer t-SNE
While the attacker's goal find small distortion that "penetrates" another class, our goal the defender is to create resilient model that forces larger distortion. The lower bound (4) motivates our penetration distortion maximization (PDM) method whereby the goal
the right side of 4 with respect to the embedding layer of the model F. To successfully apply this technique we must increase the embedding margin !) (while not increasing the norm the Jacobian) and/or smooth the network to decrease the nomm of the Jacobian []Je(z)]] We note that similar and stronger, formal bounds, in terms the Lipschitz constant, have been introduced by Tsuzuku et al. 2018 Zhang et al. (2019), Hein & Andriushchenko (2017) and Ding al, (2018).
INCREASING RESILIENCY USING PDM
In this section we show how we use PDM, which is applied to the final layer that captures the full embedding of the network (often referred to the "pre-logits"). We note that technically we can also apply PDM to any other layer in the model but defer such explorations to future work. The proposed approach consists of three components, which are described in this section. Two novel components are used to increase the margin, and the third is known technique that is responsible for reducing the norm of the Jacobian.
Our approach for increasing the embedding margin relies the observation that higher embed- ding layers of trained model, the embedding vectors (tensors) of instances tend to be structured in clusters according class labels. This can be seen, for example, in Figure Ta where observe the t-SNE visualization (Maaten & Hinton 2008) of the embedding layer of network trained for CIFAR-10. Moreover, observe that adversarial example created by untargeted attack of- ten obtains class label whose cluster is in close proximity with the cluster of the original class. In Figure Tb we present color matrix showing the adversarial label distribution obtained by the C&W attack (Carlini & Wagner 2017a). For example, the color for the Cat and Dog entry is bright red indicating a frequent label change from cats (original) dogs (adversarial), whose clusters are the closest. By increasing the margin between these clusters without increasing the norm of this layer's Jacobian ,we make harder for an adversary to alter the label using distortion of the same magnitude.
Adopting ideas from cluster analysis, the increase the embedding margin can be achieved by either increasing the distance between clusters reducing the variance of each cluster. Let Me DNe, 2f be the mean of each cluster, where N. is the number of samples from class and 2f is

---

Under review as a conference paper at ICLR 2020
the embedding of sample from class c, and let M be the number of classes. We thus have,
Cluster Variance
11
Mell2
No
c=1
1=1
M Σ
Cluster Distance
Ill
Mell2
i
To increase the margin, we would like to maximize the cluster distance and minimize the cluster variance, hence
Margin Maximization Objective = Cluster Variance - Cluster Distance.
A straightforward maximization of the cluster distance is problematic because the distance is poten- tially unbounded. However, we can proxy the distance using the angular distance between clusters. To this end, we use the cosine similarity. We now introduce two methods to optimize these compo- nents, We use a Siamese training procedure to maximize the cluster distance. The cluster variance minimized by including variance in the loss function.
3.1 STAMESE TRAINING
To explicitly increase the embedding margin, propose using Siamese training. We create Siamese network (Bromley al. 1994), where each sub-network is our classifier. The Siamese network has two input images denoted by z! 2 and three outputs: two classification outputs and an auxiliary output for the cosine similarity between each sub-network' ' embedding. We introduce an additional loss term to force embeddings from different classes samples have cosine similarity of 0 or otherwise. Formally,
29
if = ē else
SiameseLoss
2
0
3.2 REDUCE VARIANCE Loss
Inspired by (Szegdyy et al. (2016), include additional loss term that penalizes large variance for each class' cluster individually. We refer to this component as the "reduce variance loss" (RVL). Formally,
Notuaves
Je
2
Hell2
RVL
Oe
(5)
N
The variance is estimated per class on each mini-batch, then averaged and minimized as part of the learning process.
3.3 REVERSE CROSS ENTROPY
We the reverse cross-entropy loss introduced by Pang al. 2018 minimize the norm of the Jacobian. By labeling a sample with a "reverse" one-hot vector, we obtain
if
R
elsc,
(Nasses
and using reverse cross entropy loss
LACE RC log F(x).
Similar to label smoothing (Szegedy et al. 2016), this method smooths the classifier's gradients and prevents the network from becoming over-confident (Müller et al., 2019). Intuitively, the differen- tiation between two samples has tighter upper bound, given the reverse labels R than one-hot labeling. We tested the gradient L2 norm value on different layers. Comparing to the baseline model, the gradients were five times smaller when using the RCE training process.
4

---

Under review as a conference paper at ICLR 2020
3.4 PDM TRAINING
A simultaneous application of the three components described above, which can robustify clas- sification model, is obtained by training the model using an appropriate loss function as well as a specialized mini-batch construction procedure. A pseudo-code of the training procedure including the loss function appears in Algorithm under Appendix P The code is self-explanatory for the most part. We note that epoch begins by creating a Siamese counterpart for each image-label pair in given batch. With probability Q. the Siamese sample is selected from the same class, and its cosine similarity label set 1. Otherwise (probability
Q). the Siamese sample is selected
from different class, and its cosine similarity label is set to O. Notice that the Siamese and RVL (Equation 5) components of the loss function are computed from the embedding vectors of each mini-batch. The RCE component is calculated using the logits.
3.5 PDM VISUALIZATION
Using t-SNE to visualize the embedding space activation, Figure Figure@illlutrates the effect of each the components of our defense method. Figure (euure2ccdemoe demonstratess how well the RVL reduces the variance, while the Siamese training process made a more profound impact on the between-class distance as shown in Figure 2d
While I-SNE is useful for visualization purposes, the aggressive dimensionality reduction may lead to misleading conclusions. To obtain quantitative evidence, we calculated the Davies-Bouldin index (DBI) (Davies & Bouldin 1979). which scores clustering quality according to the distance between cluster centroids divided by the Euclidean distance between points within cluster (lower score means better clustering). The DBI of the combined method is indeed the lowest at 0.23. (See the other DBIs in the figure.)
EXPERIMENTS
Following (Pang et al. 2018 Meng & Chen 2017 Madry et al., 2017 Song et al. 2017: Dhillon et al. 2018 Samangouei et al. (2188) we evaluated our defense technique on the MNIST LeCun
al. 1998) and CIFAR- 10 (Krizhessky & Hinton 2009) datasets.
We adopted the detection method presented by Feinman et al. (2017), using a univariate Gaussian- based kernel density estimation (KDE), where the density estimated using 1000 training sample embeddings per class. An input image is deemed adversarial if the distance to the predicted class' report results as the area under the ROC curve (AUC). We use ResNet 56 (He et al. (0166) as our classifier, and compare our results two baselines: standard training of ResNet-56 (i.e., without any defense mechanism) and ResNet-56 equipped with the RCE defense, the current state-of-the model. The hyper- parem-parseeters used are listed in Appendix B
manifold exceeds predefined threshold. As the decision is threshold-dependens
In our study we used several attacks, which are described in Appendix E For the bounded adversarial attack algorithms, we used two versions of FGSM and BIM, one with small perturbation e 0.05, and another with a large perturbation two versions of C&W: lean version with zero confidence, and an extensive version with higher confidence value, denoted by C-Wh We used the Cleverhans implementation (Papernot et al.) 2018) for the attacks and applied them in an untargeted manner. A detailed description of the parameters used in the adversarial attacks appears in Appendix C A description of the threat models we consider in this paper appears in Appendix
0.1. For the unbounded attacks, we used JSMA and
4.1 PERFORMANCE ON NORMAL SAMPLES
We began by evaluating the performance of model normal samples, shown in Table [ While the RCE method of Pang et al. (2018) lowered the classifier's 's accuracy on CIFAR-10, using the Siamese training scheme and applying the reduce variance loss increased the accuracy for both CIFAR- 10 and MNIST. These results indicate that these independent value in training standard classifiers, regardless of the need for adversarial robustness.
procedures may be of
5

---

Under review as a conference paper at ICLR 2020
41-
-20
40
+50
-20
-20
Baseline DBI:0.98
(b) RCE DB1:0.31
-0
-40
-40
-50
-20
40
-40
20
(c) Reduce Variance DBI:0.28
(d) Siamese Training: DBI:0.32
airplane
automobile
bird
deer
dog
frog
-20
horse
ship
(f) Legend
(e) Combined Methods; DBI:0.23
Figure 2: (- visualization of the two margin increasing components PDM Com-
pared to the baseline, each method contributes to the increase of the margin; the combined method display the best clustering according to the Davies-Bouldin Index.
CE
RCE
Siamese RVL
PDM (ours)
CIFAR-10
93.62
93.3
94.37
93.81
MNIST
99.33
99.32
99.37
99.52
Table 1: Performance on normal samples
4.2 GRAY-BOX MODEL
We follow a strict definition of gray-box threat model as in (Pag et al.) 2018). where the attacker has full access to the trained model, but unaware of the detection mechanism. See Appendix Dor precise definitions of the threat-models we consider this paper. We evaluated the performance un- der the gray-box threat model by creating equally-sized groups of adversarial and normal examples.

---

Under review as a conference paper at ICLR 2020
We scored each example using (KDE-based) detection mechanism. The results are shown in Ta- ble[2] The PDM detection AUC results over MNIST are outstanding, showing that all the unbounded attacks (such as C&W) were detected perfectly. On the CIFAR-10 dataset PDM performance lagged behind in defending the two BIM attacks. However, it achieved excellent AUC results for the other attacks, including perfect score for the strong C& W high confidence attack. Figure R presents the resiliency of PDM and the baselines when attacked by FGSM and BIM. Consider, for example, Figure B(a) depicting the resiliency achieved against FGSM over the CIFAR-I0 dataset. The x-axis corresponds to the distortion step size (c) used by the adversary, which alters each pixel by #c. The y-axis measures the resiliency, namely how many perturbed instances were predicted correctly by the model. While the resiliency monotonically decreases as function of the step size, as expected, the resiliency exhibited by PDM (blue) consistently higher than the baselines. Similar behavior was observed for the BIM attack on this dataset. Over the MNIST dataset, PDM was more resilient than most methods, but not all step sizes.
Further investigation of the mediocre results obtained for the BIM attack revealed that the norm of the embedding layer gradients increased significantly for embedding vectors located in-between clusters. Since BIM makes sequence of small gradient steps starting inside clusters, it is able to move further away into the center of different class where it can no longer be detected using KDE. This observation was made by measuring the mean gradient norm after each BIM step. After several such steps we observed that the mean norm increased by an order of magnitude. This phenomenon does not occur when using solely RCE-trained model.
MNIST
CIFAR-10 10
Baseline
RCE
PDM
Baseline
RCE
PDM
FGSM-0005
0.981
0.983
0.988
0.958
0.898
0.967
FGSM-0.1
0.988
0.99
0.995
0.971
0.926
0.983
BIM-0.05
0.983
0.967
0.987
0.99
0.95
BIM-0.1
0.945
0.92
0.99
0.996
0.962
C&W
0.994
1
1
0.874
0.918
0.933
C&W-he
0.88
0.98
1
0.637
0.94
1
JSMA
0.995
1
1
0.952
0.96
0.973
Table 2: Detection AUC under the gray-box threat model.
4.3 WHITE-BOX MODEL
The white-box threat model (see Appendix D is perhaps the most interesting from the defender's viewpoint because no limitations are made regarding the information known to the attacker (Carlini 2019). For the white-box threat model, we applied the C&W modified attack (hereatter referred to as C-b) (Carlini & Wagner, 2017a). which has been shown to penetrate density estimation-based detection. To the best of our knowledge, this is the only known attack with this property. Following the evaluation procedure used by |Pan et al. (2018), we set the parameters of the C& R- attack such that all attacks would succeed fooling the model, and detection fully breached (i.e., their AUC score < 0.5). Then, we measured the average (minimal) required distortion that was able meet these criteria. Thus, a stronger defense should yield larger distortion. We note
that the distortion is quantified using the L2 norm, namely, d 1 IM the number of adversarial instances, and n is the number pixels per image.
||x; "rr=ad""lle lle where M is v
Baseline
RCE
PDM (ours)
MNIST
0.087
0.104
0.162
CIFARTO
0.008
0.019
0.026
Table 3: Distortion under the white-box threat model, scaled to [0,1], Our defense method requires 30% higher distortion on CIFAR-10 and 60% higher on MNIST
The woite-box results are presented in Table PDM, clearly outperforms the baselines by a wide margin by forcing 30% higher distortion than RCE CIFAR-10, and 60% on MNIST.
7

---

Under review as a conference paper at ICLR 2020
Baseline
0.4
Baseline
RCE
RCE
06-
0.3
PDM
PDM
Resllleny
0.5-
0.2
0.3-
0:5-
0.00
0.000
0,302
0.125
0.225
0200
Perturbation
Petturbation
(a) FGSM-CIFAR-10
(b) BIM-CIFAR-10
Baseline
Baseline
RCE PDM
RCE
PDM
Resillecy
Resillency
0.6
0.4
0.4
02
02
0.8
0.500
0.300
0.100
6.500
0.175
Perturbation
Perturbation
(c) -NIST
(d) BIM-MNIST
Figure 3: Resiliency under the FGSM and BIM adversarial attacks. Our method displays signifi- cantly higher resiliency
4.4 BLACK-BOY BBx MODEL
To evaluate our model in the black-bol setting, we follow (Papernot et al. [011 Carlini 2019) and create a proxy model, which is trained using input-output pairs probed from the defender's (target) model (i.e., the proxy model is trained via teacher-student distillation of the target model). The proxy model is then used by the attacker to generate adversarial examples under the white-box threat model (this black-box model variant is the most difficult, and was referred to by Pang et al. as "A white, black-box attack"). In our case, where detection is based KDE, we could only the C& W-wb because it is the only available attack known to penetrate KDE.
Following Pang et al. in this setting used ResNet32 He al. (2016) for the proxy model. Here again, we measured detection AUC rates as in the gray-box setting. The results appear in Table and show that PDM is consistently and significantly better than the baselines.
Baseline
RCE
PDM (ours)
MNIST
0.88
0.94
0.99
Cifar10
0.93
0.933
0.952
Table 4: Detection AUC under the black-box threat model
CONCLUDING REMARKS
We introduced powerful approach for the defense of deep models against adversarial attacks by building on procedures for margin maximization within a penetration distortion maximization framework and the RCE loss technique. Our empirical evaluation demonstrated state-of-tha oh sults defense against all threat models (with mixed results for the BIM attack). In addition, we provide geometric intuition attacks and defenses using t-SNE visualizations.
8

---

Under review as a conference paper at ICLR 2020
This work raises several interesting questions. First, it would be valuable to examine other meth- ods for margin maximization and Jacobian reduction. For example, recently Elsayed et al. (2018) proposed a sophisticated loss function that tends to maximize the embedding margin. Similarly, recent work by /Zhang et al. (2019) proposed an iterative technique to reduce the norm of the Jaco- bian. Finally, it would be very interesting explore ways increase the margin (and reduce the Jacobian) on shallower embedding layers where lower-level features are formed.
A Acknowledgment: The authors would like express their gratitude Tianyu Pang for his invalu- able help in the implementation and understanding of his work on the reverse cross-entropy method.
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses adversarial examples. arXiv preprint arXiv:1802.00422 8802.00420. 2018.
Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. Signature verifi- cation using a" siamese" time delay neural network. In Advances in neural information processing systems, pp. 737-744, 1994.
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3-14. ACM, 2017a.
Nicholas Carlini and David Wagner. Magnet and" efficient defenses against adversarial attacks" are not robust adversarial examples. arXiv preprint 171 2017b.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57. IEEE, 2017c
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, and Aleksander Madry. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705. 2019.
Jeremy M Cohen, Elan Rosenfeld, and 7ico Kolter. Certified adversarial robustness via randomized smoothing. arXiv preprint arXiv: /902.02918, 2019.
David L Davies and Donald W Bouldin. cluster separation measure. IEEE transactions on pattern analysis and machine intelligence, (2):224-227, 1979.
Guneet S Dhillon, Kamyar Arizzadenesheli, Zachary C Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. Stochastic activation pruning for robust adversarial de- fense. arXiv preprini arXiv: 2018.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Max-margin adversar- ial (mma) training: Direct input space margin maximization through adversarial training. arXiv preprint arXiv:1812.0266 2018
Gamaleldin Elsayed, Dilip Krishnan, Hossein Mobahi, Kevin Regan, and Samy Bengio. Large margin deep networks for classification. In Advances in neural information processing systems, pp. 842-852, 2018.
Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint a 2018
Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. Detecting adversarial samples from artifacts. arXiv preprint 1rXiv:1703.00410, 2017.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 14Xiv: 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.

---

Under review as a conference paper at ICLR 2020
Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song. Adversarial example Ensembles of weak defenses are not strong. In 17th {USENIX) Workshop on Offensive Technologies ( ([WOOT} 17), 2017.
defense:
Matthias Hein and Maksym Andriushchenko. Formal guarantees the robustness of a classifier against adversarial manipulation. Advances in Neural Information Processing Systems, pp. 2266-2276, 2017.
Elad Hoffer and Nir Ailon. Deep metric learning using triplet network. In International Workshop on Similarity-Baan Based Pattern Recognition, pp. 84-92. Springer, 2015.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, Citeseer, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint wwxiv::607.0253.. 2016.
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2278-2324, 1998.
Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. Large-margin softmax loss for convolu- tional neural networks. In ICML,, volume 2, pp. 2016.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, ((oov) 2008.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 2017.
Dongyu Meng and Hao Chen. Magnet: a two-pronged defense against adversarial examples. In Proceedings of the 2017 ACM SIGSAC Conference Computer and Communications Security, pp. 147- ACM, 2017.
Rafael Müller, Simon Kornblith, and Geoffrey Hinton. When does label smoothing help? arXiv preprint arXiv:1906.02629 2019.
Tianyu Pang, Chao Du, Yinpeng Dong, and Jun Zhu. Towards robust detection of adversarial exam- ples. In Advances Neural Information Processing Systems, pp. 3584-45944 2018.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In 2016 IEEE European Sympo- sium on Security and Privacy (EuroS&P), pp. 372-387. IEEE, 2016a.
Nicolas Paperno, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense to adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pp. 582-597. IEBE, 2016b.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pp. 506-519, ACM, 2017.
Nicolas Papernot, Fartash Faghri, Nicholas Carlini, Ian Goodfellow, Reuben Feinman, Alexey Ku- rakin, Cihang Xie, Yash Sharma, Tom Brown, Aurko Roy, Alexander Matyasko, Vahid Behzadan, Karen Hambardzumyan, Zhishuai Zhang, Yi-Lin Juang, Zhi Li, Ryan Sheatsley, Abhibhav Garg. Jonathan Uesato, Willi Gierke, Yinpeng Dong, David Berthelot, Paul Hendricks, Jonas Rauber, and Rujun Long. Technical report on the cleverhans v2.1.0 adversarial examples library. arXiv preprint arXiv:1610.0076. 2018
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-gan Protecting classifiers against adversarial attacks using generative models. arXiv preprint arXiv: 18805506605 2018.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. arXiv preprint arXiv: 1t1.///11.0700 2017.
10

---

Under review as a conference paper at ICLR 2020
Thilo Strauss, Markus Hanselmann, Andrej Junginger, and Holger Ulmer. ods as defense to adversarial perturbations against deep neural networks. (rXf::1709.03423, 2017.
Ensemble meth- arXiv preprint
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-2826, 2016.
Vincent Tjeng, Kai Y Xiao, and Russ Tedrake. Evaluating robustness of neural networks with mixed integer programming. 2018.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint (tt5::1770000000 2017.
Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Lipschitz-margin training: Scalable certifi- cation of perturbation invariance for deep neural networks. In Advances in Neural Information Processing Systems, pp. 6541-6550, 2018.
Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In European conference on computer vision, pp. 499-515. Springer, 2016.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. arXiv preprint arXiv:1711.008: 2017.
Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and J Zico Kolter. Scaling provable adversarial defenses. In Advances Neural Information Processing Systems, pp. 8400-8409, 2018
Ziang Yan, Yiwen Guo, and Changshui Zhang. Deep defense: Training dnns with improved adver- sarial robustness. In Advances in Neural Information Processing Systems, pp. 417-4114666 2018.
Huan Zhang. Pengchuan Zhang, and Cho-Jui Hsieh. Recurjac: An efficient recursive algorithm for bounding jacobian matrix of neural networks and its applications. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 5757-5764, 2019.
Fei Zuo, Bokai Yang, Xiaopeng Li, and Qiang Zeng. Exploiting the inherent limitation of 10 ad- versarial examples. In 22nd International Symposium Research in Attacks, Intrusions and Defenses ((RAID) 2019), 2020.

---

Under review as a conference paper at ICLR 2020
Appendix
A L2 NORM SUB-MULLTILICATTVIIY
Given A E IR NNNM and B E RKazt, we claim that the Frobenius norm of the multiplication of and less than or equal to the multiplication of each tensor's Frobenius norm. Proof:
N
M
K
|BB
m, bRij!
m2
j=1
1=1
N
M
L
K
lanmm
b ?
(Cauchy-Schward
m=
N
an.m,k
|bx.j.1
j=
1,,=1
ΣΣΣ
0n
ba
(6)
| BI
B CLASSIFIER HYPER-PARAMETER
Parameter
Value
Optimizer
SGD
ResNet Depth
56
Weight Regularization
L2 (0.002)
Batch Size
128
Initial Learning Rate
0.1
-oochs-CIFAR-10
400
Epochs-MNIST
80
Activation
Leaky-Relu (0.1)
C ADVERSARIAL ATTACK PARAMETERS
Attack
Parameter
Value
BIM
Iterations
JSMA
Max Iterations
100
CW
Max Iterations
10
Binary Search Steps
9
Confidence
0
CW-he
Max Iterations
1000
Binary Search Steps
9
Confidence
CW-wb
Max Iterations
5000/10000
Confidence
0/0
Binary Search Steps
3/3
Table 5: Adversarial attacks parameters
12

---

Under review as a conference paper at ICLR 2020
D THREAT MODELS
Rigorous analyses of adversarial defense systems requires precise specification of what the adversary knows and do. Conducting this type of specification called threat modeling (Kurakin al., 2016). In this paper we consider the three core models, which are differentiated by their knowledge of the classifier (target network) and the defense mechanism being used.
Black-box: In this weakest scenario, the adversary has knowledge of our system. It does not know the target network architecture, cannot access its gradients and does not know the defense methods. The adversary can, however, sample the targeted network for input-ouiput pairs and has access to the dataset used to train the target network.
Gray-box (a.k.a. oblivious): Here the adversary has full knowledge of the target network and can access its gradients and parameters, including the data being used in the training process, However it has no information on the defense mechanism.
White-box: In this most challenging scenario, we assume the adversary has full knowledge of the entire system be attacked, which includes the target network, the defense method, their parameters and the data used for training.
To ensure a clear distinction between threat models, Carlini al. 2019) recently proposed compre- hensive methodology for evaluating adversarial attacks and defenses, thus emphasizing the differ- ence between gray-box and white-box attacks. We believe that following the Carlini et al. guideline is crucial for advancing this line of research and therefore adhere to them strictly.
E ATTACK ALGORITHMS
Adversarial attack algorithms aim fool DNNs such that a given image is misclassified with min- imal perturbations. In untargeted attacks, the adversary aims to minimize the true class activation so that another arbitrary class will be predicted. In this paper, we focus on targeted attacks where the adversary aims to fool the classification in controlled manner such that specific class will be predicted instead of the correct one. Thus, targeted attacks are considered far more dangerous. For example, fooling an autonomous driving system to interpret a stop sign speed limit sign is far more dangerous than interpreting it as a yield sign.
In this section, we discuss the various attack algorithms we use to evaluate our defense method. We denote z, as the input and adversarial images, respectively, as the target label, F as the target model with loss function LF(z, f) and || z'1l as the pixel-wise perturbation between the adversarial and normal images. The general formulation, therefore, becomes,
minimize || x'11 s.l. F(x') =t x²
FGSM
Goodfellow et al. (2014) introduced the fast gradient sign method (FGSM), which optimizes the adversarial image by back-propagating the input through the attacked DNN, in accordance with the desired target. Formally, letting be fixed parameter, the adversarial example is,
a' = ssggn (x,
While not effective as other attack algorithms, this method has the advantage of being one of the fastest ones.
BIM
Kurakin et al. (0016) introduced the Basic Iterative Method (BIM), which performs the FGSM method iteratively, clipping the perturbation if needed. Formally,
2'v+1 I'y + esign (sign(VLs (2'y, 1)),
where e is a fixed parameter.
C&W
13

---

Under review as a conference paper at ICLR 2020
Carlini & Wagner (2017c) introduced the C& method, which operates by modifying the L-BFGS method as follows.
minimize || z'11 + cf(x')
where è is a hyperparameter and the loss function, f. is chosen such that f(x') <= 0 if is classified the target class; namely.
f(z') (a (())),
where is the target DNN logits, is the correct label and xi is a hyperparameter referring to the confidence of the attack. The higher the confidence. the higher the activation for the target class is, and therefore, the larger the perturbation.
Three different Euclidean norms are considered with this algorithm, Lo, L2, Loo. Following Pang et al. (2018), we conduct our evaluation using L2. This attack method is considered very et- fective and has had great success in overcoming various defense methods Carlini WWagner (2017c) Papernot et al. (2016b).
JSMA
Papernot et al. (2166a introduced the Jacobian-basee Saliency Map Attack (JSMA) which alters single pixel of x at each iteration maximize the saliency map. This method is known enforce large perturbations but on fewer pixels than other methods.
F PDM TRAINING ALGORITHM
Algorithm PDM Training
procedure PDM
2:
for batch = 1.... #batches do
3:
X, X,Y Y - get get. batcbatch))
4: initialize Xiammes = [. siamese 0.5 =
5: for = I.... batch_size size do
6:
q Bernouli(())
7:
if q = then y (
8:
9:
8 -
else
11:
y + random class + y/1
$
13:
random sample from class Y
Append xi, y, X Xsuueeee: Ysiamese;
15:
21, 22 + (x) Xiiamess
sample's embedding
P1.P2 P(X), F X
sinmese
17:
SL. butch size Σ
B2
S
D model logits
18:
RVL +- ()
19:
for c = 0, ... cclasses do
20:
Me = N. Σn= 2f
0e N. Σ || Pell2
RVL RVL Oc
23:
minimize -R² log (pI)-RYamml log (p2) + RVL SL
14


Reviews:
[{"review_id": "SkxDVJmVKH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nSummary\n========\nThis paper proposes a defense against adversarial examples that detects perturbed inputs using kernel density estimation. The paper uses a combination of known (and often known to be broken) techniques, and does not provide a fully convincing evaluation.\nI lean towards rejection of this paper.\n\nDetailed comments\n=================\nThe idea of increasing robustness by maximizing inter-class margins and minimizing intra-class variance is fairly natural, but the author's discussion of their approach (mainly in sections 1 and 2) is very hand-wavy and relies on a lot of general intuitions and unproven claims about neural networks.\n\nFor example, in the introduction, the authors claim:\n\n\"A trained deep classification model tends to organize instances into clusters in the embedding space, according to class labels. Classes with clusters in close proximity to one another, provide excellent opportunities for attackers to fool the model. This geometry explains the tendency of untargeted attacks to alter the label of a given image to a class adjacent in the embedding space as demonstrated in Figure 1a.\"\n\nFirst, a t-SNE representation is just a 2D projection of high-dimensional data that is useful for visualization purposes, and one should be careful when extrapolating insights about the actual data from it. For example, distances in the 2D projection do not necessarily correspond directly to distances in the embedding space. \nThe claim that untargeted attacks lead to a \"nearby\" cluster are hard to verify given just Figure 1. First, the colors of the labels between 1a and 1b do not seem to match (e.g., Dog is bright green in 1b but this color does not appear in 1a). If the other colors match, then this would seem to suggest that trucks (purple) often get altered to ships (orange). Yet, the two clusters are quite far apart in 1a. It seems hard to say something qualitative here. An actual experiment comparing distances in the embedding space and the tendency of untargeted attacks to move from one class to another would be helpful.\nThe color scheme in Figure 1b is also unclear. A color bar would help here at the very least.\n\nThese observations are then used to justify increasing cluster distance while minimizing cluster variance, but it would be nice to see a more formal argument relating these concepts to the embedding distance.\n\nThe technique proposed in Section 3.2. to reduce variance loss estimates each class' variance on each batch. Would  this still work for a dataset with a large number of classes (e.g., ImageNet)? For such a dataset, each class will be present less than once in expectation in each batch, which seems problematic.\n\nThe plots in Figure 2 don't give much of a sense of how the combination of the different proposed techniques is better than any individual technique. The evaluation compares PDM to RCE, but from Figure 2 one could guess that variance reduction alone (2c) performs very similarly to PDM (2e). An ablation study showing the contribution of each of the individual techniques would be helpful.\n\nThe evaluation section could be improved significantly. FGSM, JSMA, and to some extent BIM, are not recommended attacks for evaluating robustness. The gray-box and black-box threat model evaluations are also not the most interesting here. Instead, and following the recommendations of Carlini et al. (2019), the evaluation should:\n\n- Propose an adaptive attack objective, tailored for the proposed defense in a white-box setting. The authors do this to some extent, by re-using the attack objective from Carlini & Wagner 2017, which targets KDE. It would still be good to provide additional explanations about how the hyperparameters for this attack were set.\n- Optimize this objective using both gradient-based and gradient-free attacks\n- As the proposed defense is attack-agnostic, I also suggest trying it out on rotation-translation attacks, as the worst-case attack can always be found by brute-force search\n\nOther\n=====\n- The citations for adversarial training in the 2nd paragraph of the intro are unusual. Standard references here are for sure the first two below, and maybe some of the other three as is relevant to your work\n    - Szegedy et al. 2013: \"intriguing properties of neural networks\"\n    - Goodfellow et al. 2014: \"Explaining and harnessing adversarial examples\"\n    - Kurakin et al. 2016: \"Adversarial Machine Learning at Scale\"\n    - Madry et al. 2017: \"Towards deep learning models resistant to adversarial attacks\"\n    - Tramer et al. 2017: \"Ensemble Adversarial Training\"\n- The Taylor approximation in (1) does not seem to be well defined. The Jacobian of F is a matrix, so it isn't clear what evaluating that matrix at a point x means.\n- The \"greater yet similar\" symbol (e.g., in equation (4)) should be defined formally.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "S1ez9A-QKH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "After rebuttal: my rating remains the same.\nI have read other reviewers' comments and the response. Overall, the contribution of retraining and detection with previously explored kernel density is limited. \n\n=================\nSummary: \nThis paper proposes new regularization techniques to train DNNs, which after training, make the crafted adversarial examples more detectable. The general idea is to minimize the inter-class variance and maximize the intra-class distance, at some feature layer. This involves regularization terms: 1) SiameseLoss, an existing idea of contrastive learning known can increase inter-class margin; 2) reduce variance loss (RVL), a variance term on deep features, and 3) reverse cross entropy (RCE), a previously proposed term for detection purpose. The motivation behind seems intuitive and the empirical results demonstrate moderate improve in detection AUC, compared to one existing technique (e.g RCE).\n\nMy concerns:\n1. The proposed technique requires retraining the networks to get a few percents of detection improvement. This is a disadvantage compared to standard detection approaches such as [1] and [2] which do not need to retain the network. I am surprised that these standard detection methods were not even mentioned at all. Retraining with fixed loss becomes problematic when the networks have to be trained using their own loss functions due to application-specific reasons. Moreover, the detection performance reported in this paper is not better than the one reported in [2] (ResNet, CIFAR-10, 95.84%) which do not need retraining.\n\n2. There are already well-known margin-based loss functions, such as triplet loss [4], center loss [5], large-Margin softmax loss [6], and many others, which are not mentioned at all.\n\n3. In terms of retraining-based detection, higher AUCs have been reported in [3] for a neural fingerprinting method.\n\n4. Incorrect references to existing works. The second sentence in Intro paragraph 2: Metzen, et al, .... these are not adversarial training. Xu, et al. (feature squeezing) is not a randomization technique.\n\n5. The \"baseline\" method reported in Table 2, is confusing. RCE is also a baseline? You mean conventional cross entropy (CE) training?\n\n6. Some of the norms are not properly defined, which can be confusing in adversarial research. For example, from Equation (1) to (4). The \"Frobenius norm used here\" statement in Equation (3), don't know this F norm comes from.\n\n\n[1] Characterizing adversarial subspaces using local intrinsic dimensionality. ICLR, 2018\n[2] A simple unified framework for detecting out-of-distribution samples and adversarial attacks. NeurIPS, 2018\n[3] Detecting Adversarial Examples via Neural Fingerprinting. arXiv preprint arXiv:1803.03870, 2018\n[4] Facenet: A unified embedding for face recognition and clustering. CVPR, 2015.\n[5] A Discriminative Feature Learning Approach for Deep Face Recognition. ECCV, 2016.\n[6] Large-Margin Softmax Loss for Convolutional Neural Networks. ICML 2016.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "HJlIS4s1qB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Update after author response:\nI would like to thank the authors for the thoughtful response, and for addressing some of the concerns raised by the reviewers. The draft appears improved but my concerns about the novelty and interpretability of the work still stand, leading me to keep my assessment unchanged.\n---------------------------\n\nIn this paper, the authors propose a general defense method against adversarial attacks by maximizing an approximate bound on the magnitude of distortion needed to force a misclassification. The authors note that this maximization can be achieved by increasing the margin between class clusters and by reducing the norm of the Jacobian of intermediate layers. Subsequently, they either directly adopt or introduce simple modifications to existing techniques to affect these two factors, showing the robustness of the combined method to several adversarial attacks on MNIST and CIFAR-10 datasets. \n\nAs neural networks get deployed for increasingly critical applications, the issue of defense against adversarial attacks becomes progressively relevant. The paper does a good job of motivating a relatively simple approach to the problem based on an approximate bound, and pulls in from different existing methods to build a robust system. The strong points of the paper:\n1. The paper is clearly written, and the approach is sensible. \n2. Fairly thorough empirical investigation under different threat models.\n3. The proposed method performs consistently above the baselines for different experiments.\n\nHere are some of my concerns:\n1. The work is somewhat incremental and the novelty mostly lies in pulling a few different methods together that seem to work well in unison.\n2. The two methods used for increasing the margin don’t actually optimize that objective directly. The Siamese Loss uses cosine distance as proxy and the variance reduction doesn’t guarantee increase in margin which is sensitive to outliers. Any improvement achieved thus appears to be an ill-understood side-effect. \n3. The definition of cluster distance (page 3) looks erroneous. \n4. The authors note that the proposal doesn’t work very well for a specific kind of attack (BIM) but don’t have clear recommendations for improvement. The tentative explanation of why this happens is also somewhat loose. \n\nIn summary, I think the paper addresses an interesting problem even though the development is arguably incremental. However, since the unified approach is simple yet novel, and the results fairly promising, I am somewhat inclined to accept this paper.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: A defense against of adversarial attacks is presented, which builds mostly on combining known methods in a novel way. While the novelty is somewhat limited, this would be fine if the results were unequivocally good and other parts of the problematic. However, reviewers were not entirely convinced by the results, and had a number of minor complaints with various parts of the paper.

In sum, this paper is not currently at a stage where it can be accepted.
Decision: Reject