Ensemble-based Adversarial Defense Using Diversified Distance Mapping
Authors: Ehsan Kazemi
Mohamed E. Hussein
Wael AbdAlmageed
Venue:  (2021)
Keywords: adversarial machine learning
ensemble
mahalanobis distance

Abstract: We propose an ensemble-based defense against adversarial examples using distance map layers (DMLs). Similar to fully connected layers, DMLs can be used to output logits for a multi-class classification model. We show in this paper how DMLs can be deployed to prevent transferability  of attacks across ensemble members by adapting pairwise (almost) orthogonal covariance matrices. We also illustrate how DMLs provide an efficient way to regularize the Lipschitz constant of the ensemble's member models, which further boosts the resulting robustness. Through empirical evaluations across multiple datasets and attack models, we demonstrate that the ensembles based on DMLs can achieve high benign accuracy while exhibiting robustness against adversarial attacks using multiple white-box techniques along with AutoAttack.

Under review as a conference paper at ICLR 2021
ENSEMBLE-BASED ADVERSARIAL DEFENSE USING DIVER- SIFIED DISTANCE MAPPING
Anonymous authors Paper under double-blind review
ABSTRACT
We propose an ensemble-based defense against adversarial examples using dis- tance map layers (DMLs). Similar to fully connected layers, DMLs can be used output logits for multi-class classification model. We show in this paper how DMLs can be deployed to prevent transferability of attacks across ensemble members by adapting pairwise (almost) orthogonal covariance matrices. We also illustrate how DMLs provide an efficient way to regularize the Lipschitz constant of the ensemble's 's member models, which further boosts the resulting robustness. Through empirical evaluations across multiple datasets and attack models, we demonstrate that the ensembles based on DMLs can achieve high benign accuracy while exhibiting robustness against adversarial attacks using multiple white-box techniques along with Attack.
INTRODUCTION
Ongoing research has provided defenses against adversarial examples, which are crafted from cor- rectly classified inputs with imperceptible perturbations. Despite the success of ensemble learning as a mechanism for reducing prediction errors and improving generalization by combining predic- tions of multiple models performing the same task (Russakovsky et al. 2015 SSai & Rokach), early research has shown the ineffectiveness of mutiple ensemble-based defenses against adversarial examples and even has gone further to suggest that ensembles are only as robust as their weak components (He et al. 2017). If this claim is true, it basically defeats the purpose of using an ensemble, which is building strong model out of weaker ones. Nevertheless, research continued to investigate the usage of ensembles as defense mechanism (Pang et al. 2019 Verma & Swami Sen al.) However, recent attempts have been quickly shown ineffective (Tramer et al. CCrce & Hein 2020). We believe that one of the primary reasons for the weakness of ensemble defenses is the inter-model attack transferability. It was shown that even fundamentally different models could exhibit high attack transferability rate (Papernot et al. 2017 Kurakin et al. (218)). This phenomenon hindered the consideration of ensemble learning as strong defense mechanism on own. The reason is that if the member models are not robust, and they exhibit high attack transferability rate, attacks generated from one model can attack the rest, and hence, attack the entire ensemble.
In this work, we show that it is possible to circumvent the problem above and instill diversity among ensemble members via the employment of specially initialized and optimized distance-man map--ayyrpass (DMLs), and hence throttle the inter-model attack transferability. Moreover, we demonstrate that DMLs provide spontaneous regularization of the Lipsehitz constant, and therefore further boost the robustness.
The rest of this paper is organized as follows: We first discuss an overview of relevant recent works on ensemble defenses against adversarial examples and background information. Then we introduce a distance map layer based on Mahalanobis distance, and also explain the threat models considered in this work. We describe the creation of ensemble of DML-based individual models. Afterwards, we introduce a randomized version of DML. Finally, we evaluate the robustness our ensemble model over MNIST, CIFAR-10 10 and RESISC-45 datasets.

---

Under review as a conference paper at ICLR 2021
2 BACKGROUND AND RELATED WORK
Not long ago, researchers revealed the vulnerability of machine learning models, particularly deep neural networks (DNNs), against adversarial examples (Szegedy et al. 2014). These models can provide incorrect predictions on examples that are slightly perturbed from correctly classified ones. The process of generating adversarial examples from natural ones is called adversarial evasion allacks (Biggio et al.). Evasion attacks can be categorized into black-box, gray-box, and white-box attacks. In the black-box setting, the attacker does not have access to the model's s parameters and the potential defense of the model (Papernot et al. 2017: Brendel et al.) 2018). In the gray-box setting, although the attacker does not have access to the model's 's parameter, it is aware of the defense applied in the model. In the white-box evasion, the attacker has access to both the parameters and the defense applied in the model. Therefore, the attacker can apply gradient-based attack techniques 2017|Madry Madry al. 2018), or it can even design custom made adaptive attack based on its knowledge about the model and the defense (Tramèr et al.). From the defense perspective, white-box attacks are certainly the most challenging type of evasion attacks. Our proposed defense targets white-box attacks. Specifically, we focus on Lp restricted perturbations of images, and, as it is typical in many research works, we consider the Los perturbations.
(Goodfellow et al.] 2014/Carlini & Wagner] [017]
Since the discovery of adversarial attacks, a significant research activity has been devoted to de- veloping appropriate defenses (Madry et al., 2018 Zhang et al.). However, most of the proposed defenses have been soon defeated (Tramer et al. Croce & Hein 2020). An intuitive mechanism to defend against adversarial examples is training ensembles of other models in order to enhance their defenses (Pang al.) 2019 |eema & Swami [Sen et al.). However, the efficacy of existing ensemble approaches is often faced with skepticism due to the transferability phenomenon (Papernot et al., 2017). which makes the ensemble model believed to be most as robust as its strongest constituent model (He et al. 2017). In this paper, we promote the diversity among individual networks from a new perspective via distance map layers. Our construction is based on the assumption that defending against transferred adversarial example is effective, even if the individual models are vulnerable to direct attacks. Our approach is orthogonal to the previous approaches and can be combined with other generic weak or strong defenses to further enhance the ensemble's 's resistance to adversaries.
Although there is no strict illustration on what is intuitively defined as diversity, in this work, we define the diversity as the highest dissimilarity between the learned features of different classes across the ensemble members. Diversity greater when the prediction errors of individual members are highly uncorrelated (Liu & Yao 1999a b) Dietterich 2000 Liu et al. 2019). This property may lead an adversarial perturbation to fail to fool the majority of networks the ensemble. Topologically, diversity can be depicted as the variability in the shapes of the decision boundaries and inter-class neighborhood relationships in the embedding space. Changing the shape of the decision boundaries in the embedding space implies different loss shapes. With diversity in the shape of the losses corresponding to different models, the gradient from one model would not be a good approximation (transferable) to the other model. Note that due to the high capacity of DNNs, changing the topology of the embedding space can have no or negligible effect on the model's accuracy. Using this perspective on model diversity, we will illustrate that DML-based ensembles could be designed to achieve the requirement of diversity in prediction errors through injecting dissimilarities only between the DMLs of ensemble members. Previous works increase the diversity over training data via promoting the diversity of prediction errors of ensemble members
(Liu & Yao, 1999a b
et al. 2019)
3 TECHNICAL APPROACH
In this section, we elaborate on our model with distance map layer (DML). We start by recalling the definition of Mahalanobis distance between two x1, E R":
(x),x)) = v(xi - x2) M - x2).
(1)
where M is the inverse of the covariance matrix (referred to as the M matrix for the rest of the paper), which is a positive semi-definite matrix.
Lemma 1. The Mahalanobis distance is k- continmous with = v2 1|12, where M = PTT) P, and P is a triangular matrix.
2

---

Under review as a conference paper at ICLR 2021
A proof for this lemma can be found in (Zantedeschi al. 2016). We now introduce mapping function, based on the Mahalanobis distance, that can be used as a differentiable layer in a neural network. The function maps an input vector to output vector by calculating Mahalanobis distances between the input and the learned centers of the function. Equation shows the element of the output vector corresponding center for input vector x. We call this function distance map layer (DML).
dM.c(x) = ()x
(2)
In this formula, M E RNNN and R.W are the learnable parameters of the DML. When the DML is used as the penultimate layer of a classification neural network (i.e. the layer producing logits), each center will correspond to one class. Essentially, the layer maps an input vector to vector of distances to class centers. Note that, in this usage, the negative of the distances has to be used to maintain the typical assumption of DNNs that the highest logit corresponds the most likely class. The classification loss function will encourage the absolute distance to the target class to be small while the distances to all other classes to be large, which induces compactness in the embedding space. This compactness has been used in prior work for classification problems (Wan et al. 2018) and for adversarial robustness of individual models (Pang et al., 2020).
Our setting is composed of combining a classification network G with DML h to obtain model characterized by Fxx)=
o Gxx) where h RMM RMM and RN - RM In other words, G is used as prior to map the input space to representation space, and h maps the representation space to distances from the corresponding class centers. In the next section, we demonstrate how a distribution shift that F shows with respect to G could be efficiently utilized to form diverse ensemble of models. Before that, we want to highlight an important feature about DML. If the network G is LG Lipschhtt continuous, and the DML h is Ln-Lipschitz continuous, then the network F is Lipschitz continuous with constant upper-bounded by LG LA- Thus, imposing small L, during the training process may lead to the regularization of Lipschitz constant for the network F. It shown in the appendix that this can also largely improve the certification bounds of network F compared to the model G.
3.1 ENHANCING DIVERSITY BETWEEN ENSEMBLE MEMBERS
The main part of the proposed ensemble model is the deployment of DMLs. The ensemble members stacked with DMLs are trained interactively to achieve the highest level of diversity. In particular, a DML ensemble is trained to satisfy these properties: 1) the ensemble members have the highest accuracy over benign samples and diverse prediction errors over adversarial samples; 2) the attacks generated on one ensemble member are not transferable to other ensemble members; and 3) the training procedure maintains the Lipschitz constant of an ensemble member model small. The first objective is important as it indicates that the ensemble could provide high accuracy even with the low accuracy of the individual members. The second objective guarantees a well-performing ensemble model under evasion attacks. That is an adversarial example crafted on an individual ensemble member model would normally be misclassified in that individual member, but not necessarily in the other ensemble members. This goal can be achieved in two folds: 1) by randomly choosing the centers and shuffling them to vary over classifiers in the formulation of DML; 2) by imposing the M matrices to be dissimilar and possibly orthogonal between the ensemble members. Increasing the cosine dissimilarity of DML's covariance matrices of ensemble members would lead to nontransferable adversarial perturbations (Adam et al., 2019). One approach to reduce the transferability across several models is to enforce various geometric relationship like orthogonality between the input- output gradient of the models. For two distance map layers f and g with corresponding M, and Mg inverse covariance matrices, we have shown in the appendix that the orthogonality of the M matrices, i... M, Mg 0 is a sufficient condition for the input-output gradients of f and to be orthogonal. Note that the explored sufficient condition is independent of the input Roughly speaking, the increase in the dissimilarity of the M matrices reduces the cosine similarity of input- gradients of the combined network by the chain rule formula. Therefore, practically, minimizing the cosine similarity of the M matrices across models enables reducing the rate of transferability of adversarial attacks across models in the ensemble.
3

---

Under review as a conference paper at ICLR 2021
3.2 DML PARAMETER INITIALIZATION AND TRAINING
In this section, we describe the procedure of simultaneous and interactive training of ensemble members towards the goal of enhancing diversity. The main building block of various ensemble members using DML is that the centers and the inverse covariance matrices of members are trained to have the highest level of dissimilaritics between the ensemble members. We aim to increase the failure independence by enhancing the disagreement diversity of different networks. We recall that these ensemble members are generated using a reference model by stacking a DML to the last hidden layer of the prior model. The ensemble model consists of the created DML-based models. To train the ensemble members, the parameters of each model learned to increase the dissimilarity of the M matrices (e.g., by decreasing the dissimilarity loss function). The DML's centers in the ensemble are shuffled across its members. In practice, we initialize the centers to be a permutation of the rows of an identity matrix. Applying the shuffled normal vectors as the centers of the DML would avoid the scaling of the gradient of the network and can decrease the risk of gradient masking. Shuffling the centers across the individual members may cause to diversity in adversarial directions for the members. In this way, the ensemble framework can potentially be used to detect adversarial examples. We discuss more about this mechanism in appendix. Dissimilar M matrices in the ensemble of models alternate the way in which each individual learner traverses the hypothesis space. We also make the M matrix partially learnable, i.e., only certain elements M are trainable and the rest are fixed during the training. This would make the classifiers have different prediction errors on different labels. In the following. we elaborate on how we split the learnable parameters of the M matrix across members to encourage diversity through diverse tasks.
3.3 META ENSEMBLE MODEL
We choose the trainable parameters in M such that there would not be any overlap between the indices of the learnable elements of different ensemble members. More specifically, let Ij denote the learnable parameters of the M matrix of the DML in the jth ensemble member. To promote the diversity in the topology of the DNNs, we select 1 such that 1 n 1 = 0 for j. E [K]. By allowing a classifier to learn specific elements in M. causes the model be more accurate on the classes with labels corresponding to those elements, e.g., the k ensemble model would show better benign accuracy on the labels in the set /t. On the other hand, during experiments, we realized that the classifiers are more vulnerable on those labels when exposed to the attackers. Since the classifiers are more susceptible on those labels, the predictions from ensemble members could be aggregated while excluding the predictions of each member classifier's specific labels, e.g., Ie for K have no pairwise intersections, neglecting
classifier k. Due to the fact that the sets Ix 's for the prediction of models those particular labels when aggregating the final prediction would not impose any artificial bias on the ensemble. We will refer to the aforementioned mechanism in the following as meta-model. In summary, we are training models which fundamentally diverse in their robustness to different adversarial attacks, and therefore it increases the varieties in the shape of decision boundaries and consequently adversarial directions in the ensemble.
We use a tailored loss to interactively train the ensemble members. Formally, the loss function is formulated
L= Σ L +y Σ Σ I M,ll + +B kE[K]
Σ |M
kE[K]
where K is the number of ensemble members, L is the number of classes, L is the classification loss (cross entropy) of htth ensemble member.
Minimizing the sum of losses from ensemble members increases the benign accuracy of the ensemble model, while the other terms in the loss formula (Equation (3)) manipulate the arrangement of the decision boundaries. The second term encourages the dissimilarity of covariance matrices and thus advocates diverse hypothesis for classifiers. The third term is a regularization parameter for the sum of L1 norm of DML's M matrices to achieve a DML with a low Lipschitz constant. The positive real numbers y and & balance these terms with the cross-entropy objective. We choose these parameters via pprammer tuning. For training, one needs to initialize the centers and the M matrices of the DML. in each classifier to have the most dissimilarity to the other ensemble members. Towards
4

---

Under review as a conference paper at ICLR 2021
having DML with low Lipschitz constant, consider diagonal M matrix, where the diagonal terms are initialized randomly from the uniform distribution between and 1. is possible to opt not to train the M matrices if there exist sufficient dissimilarities between paired matrices (i.e., orthogonality) via appropriate initialization, given that networks trained from different initialization be far apart in their internal representations.
Finally, a consensus method over the predictions from individual classífiers performed to obtain the output of the ensemble model. The consensus method could be any operation on the prediction outputs, e.g., majority voting, mean, median, sum. In this work, the ensemble output is based on the majority voting k-top predictions of the ensemble members.
RANDOMIZED DML AND UNCERTAINTY IN THE DML DNN
Considering the distance map layer f(x) vx- ( c), where M P P and P is lower-triangular matrix, we construct a randomized version of DMI. from the hase DMI. f. The randomized DML g, is the expectation of the isotropic Gaussian perturbation of the M
(x)) = BB-N(O.
[ c) (P (Pt))
(4)
where is a hyper-parameterr
Here, we apply the uncertainty of a Gaussian process to identify the low-confidence regions of the input space the DML layer. The Gaussian process assumes of priors the set of all functions that can map the input space to the output space. At training time, for any sample x with target classification y, only the function of the center corresponding to y will be considered. At inference time, the expectation of all the output of these functions are used as the prediction. As the M matrix determines the shape of decision boundaries, uncertainty estimation from randomized smoothing of M can be extracted during the inference time. We sample i.idd Gaussian samples ōr....,ōr NO0, I)), and use the following Monte-Carlo estimator for the expectation of the randomized DML g
I Σ - (P= 1=1
(xx) ~
(5)
T
In the experiments, show that the randomized DML provides better robust accuracy compared to DML when used in an ensemble.
5 EXPERIMENTS
In this section, we present our empirical studies to show the effectiveness our proposed method on decreasing the threat from adversarial attacks, while showing high benign accuracy. We perform our experiments on MNIST, CIFAR-10 and RESISC-45 (Cheng et al. 2017) detasets. Each of MNIST and CIFAR-10 10 has 10 classes and RESISC-45 has 45 classes. We implement the ensemble model based on the networks shown in Table [ In the experiments for each dataset, we consider an ensemble of five models. Our baseline model the ensemble of five models without DML trained with cross-entropy loss. The DML-based ensemble models are jointly trained based on the loss in Equation 3 We set 0.01 and y 0.01 in Equation (3) for training the ensemble models over all three datasets. Following the illustration of the ensemble meta- model, for each classifier, only two elements of the diagonal elements of the M matrix are trainable and the rest of the diagonal elements are set to the fixed value 0. For all the individual models in the ensemble and for all datasets, the center points DMLs are set to the columns of an identity matrix with different shuffling across members. The centers are fixed during the training procedure.
In Figure I we show the distribution of the positive CLEVER score values (Weng et al. 2018) for 1000 randomly selected images from the MNIST dataset for model formed by summing the outputs of ensemble members. We used the L2 radius of 2 to calculate the CLEVER scores. The results for the other datasets, which is similer to MNIST's. s, are provided in the appendix. The results shows that the number of samples with non-zero CLEVER score an order of magnitude higher than the counterpart for the baseline model. Moreover, for the baseline model, there exist sample that has
5

---

Under review as conference paper at ICLR 2021
Dataset
MNIST
CIFAR-10
RESISC-45
Model
LeNet (Lecun et al. 1998)
ResNet-18 (He et al. 2016)
Table 1: Base models for creating DML-based models for each dataset
30-
0.
2.8
2.5
1.
0.23
0.20
1.00
1.20
2.5
1.75
CLEATR 30010
(L/7871 soare
(a) Baseline
(b) DML-based ensemble
Figure 1: Distribution of non-zero CLEVER scores for baseline ensemble and DML-based ensemble for 1000 randomly selected images from the MNIST dataset. The CLEVER scores are computed on the L2-norm ball with radius 2.
score higher than 1.7 while for DML, the highest score is close to 2. This shows that the DML-based ensemble has an intrinsic boost in robustness compared to the baseline.
In Tables 3 show the benign and adversarial accuracy of ensemble models over the three datasets. The white-box attacks that we consider are FGSM (Goodfellow et al. 2014), BIM (Kurakin) et al. 2017), PGD (Madry et al., 2018), and Carlini & Wagner (CW) (Carlini & Wagner 2017). which are commonly used for evaluating defense models. The number of iterations for BIM and PGD set to 10, while for CW, it is the penalty (initial constant) parameters for CW are shown in the table. The attacks we consider in this section are untargeted attacks unless otherwise mentioned. In untargeted attacks adversaries constructed to cause the classifier to produce any incorrect label. Unlike the previous research on ensemble-basee based defenses (Sharif et al. 2019 Liu al. 2019) where the adversaries were crafted over target model, in this paper, the adversarial examples are generated over the ensemble model derived by summing the predictions. Crafting adversarial examples over the ensemble model would assess the true resistance of the ensemble as the generated adversaries contain the gradient information from all the individual members. The predicted labels for the benign and adversarial samples in the ensemble model are obtained using the majority votes of top-1 predictions of the individual classifiers. The results show remarkable performance enhancement over the baseline model. This could be explained by high non-tree tr--tassferrbblity of adversaries and diverse ensemble representation across the ensemble members, which hinders the process of finding an optimal adversary that could fool the majority of ensemble members to have the correct prediction changed.
200. The L.s budget for FGSM, PGD, and BIM; and
Table 2: MNIST white-box attack results. FGSM and PGD are applied with specific perturbation budget e. CW is used with the specified penalty (initial constant) parameter y.
FGSM
FGSM
BIM-10
PGD-10
CW
Training Model
Benign Acc
0.2
0.3
= 0.2
= 0.2
y
Baseline
98.79
23.81
3.12
0.00
0.00
0.00
DML Ensemble
93.66
68.55
39.62
63.08
62.92
85.00

---

Under review as conference paper at ICLR 2021
Table 3: White-box attack results for CIFAR-I0 and RESISC-45 datasets. FGSM and PGD are applied with specific perturbation budget c. CW is used with the specified penalty (initial constant) parameter y.
FGSM
FGSM
BIM-10
PGD-10
CW
Training Model
Benign Ace
0.015
0.031
6 0.015
0.015
Y =
CIFAR-10
Baseline
85.66
23.84
6.15
7.75
6.15
27.67
DML Ensemble
84.57
56.61
43.35
38.35
39.87
39.54
RESISC-45
Baseline 85.66
3.75
1.34
0.00
0.00
45.22
DML Ensemble
87.28
52.06
39.64
45.07
45.12
61.84
Table 4: Adversarial accuracy of all ensemble models when the attacks crafted Model 1 with PGD-10. The labels for the targeted attacks are selected randomly and different from the true labels. The size of perturbation is 0.2, 0.015, 0.015 for MNIST, CIFAR-10, 10, and RESISC-45, respectively.
Dataset
Accuracy
Ensemble
Model
Model
Model 3
Model 4
Model
MNIST
Benign
93.66
86.56
69.57
95.40
93.35
95.36
Untargeted
58.24
33.79
44.96
66.14
53.99
63.83
Targeted
83.07
58.24
58.03
84.53
76.49
83.64
CIFAR-10
Benign 84.57
82.08
82.02
81.30
81.42
81,64
Untargeted
61.35
40.93
64.61
64.15
63.40
63.84
Targeted
68.60
48.45
70.51
69.12
69.07
69.42
RESISC-45
Benign
87.29
75.33
78.33
79.67
79.51
79,89
Untargeted
74.02
30.80
68.44
69.42
71.07
69.31
Targeted
75.33
35.58
72.89
74.40
75.42
74.12
We examine if the adversaries generated other models in the ensemble. For this purpose, in Table 4 we provide the results for both targeted and untargeted attacks when the crafted attacks on Model are transferred the other constituent ensemble members and the ensemble model. It is observed that the crafted attacks exhibit low transferability rate. This could be illustrated by the visualisation of the loss function around sample from RESISC-45 dataset in Figure which shows that the shapes of the loss surfaces of the constituent models are diverse and therefore the input-output gradients of a model do not provide close approximation for the other models included in the ensemble.
ensemble member could be transferred the
In T we compare the distortion needed the ensemble model versus the individual members for the targeted attacks using adaptive CW-L2 attack (He et al. (017) for 100 randomly selected samples from each dataset. The results show that the perturbation required to generate adversaries individual member is multiple times lower than the distortion required to create adversaries the ensemble models.
Finally, in Table 6 compared the robust accuracy for DML-ensemble and randomized DML ensemble over CIFAR-10 datasets. The results for the other dataset could be found in appendix. The results from randomized DML-ensemble shows outperform adversarial accuracy.

---

Under review as conference paper at ICLR 2021
<3/4
Model
Model 2
(c) Model
Figure 2: Loss surface for individual classifiers around a sample from RESISC-45 dataset for
points x, rV V.LCE(As. y.) + with N(0, 0.05), e [0, 0.1., and e [-0.1.0.1]
Table 5: The required L2 distortion for adaptive CW-L2 attack (He al. 2017) with y for the combined and individual ensemble members of size 5 for 100 randomly selected images for each dataset. The number of iterations and stepsize are 1000, 0.1, respectively. x is denoting the confidence parameter for CW-L2 attack.
La distortion required for generating adversaries on each model
Dataset
Ensemble
Ensemble
Model
Model
Model
Model
Model
k
K == 0.1
1
2
5
MNIST
Adversarial accuracy
66.00
45.00
0.00
8.00
1.00
1.00
4.00
Average distortion
2.31
3.79
1.34
1.41
1.52
1.36
1.52
CIFAR-10
Adversarial accuracy
52.00
12.00
0.00
0.00
0.00
0.00
0.00
Average distortion
0.46
23.30
0.52
2.67
2.69
0.51
0.45
RESISC-45
Adversarial accuracy
73.00
71.00
0.00
0.00
0.00
0.00
0.00
Average distortion
3.60
6.24
1.47
1.63
2.06
2.36
2.53
8

---

Under review as conference paper at ICLR 2021
Table 6: Accuracy of white box attacks on DML-based ensemble and randomized DML (R-DML) ensemble (R-DMl-ensemble) for the perturbation budget specified for each dataset. The attacks are performed with random restarts.
Dataset
Benign Acc
FGSM
PGD-20
PGD-40
CIFAR-10 (e 8/255)
DML Ensemble
84.57
47.72
19.38
14.46
R-DML Ensemble
81.25
56.48
38.12
37.34
Table 7: MNIST AutoAttack (Croce & Hein 2020 results. The advensaries are crafted with the
specified perturbation budgets c.
AutoAttack
Training Model
Benign Acc
c 0.1
6= 0.2
6 0.3
DML Ensemble
93.66
88.47
73.46
49.44
6 AUTOATTACK ON THE DML- ENSEMBLE
this section, we evaluate our proposed model by the ensemble of parameter-free attacks, AutoAttack (Croce & Hein 2020). In (Croce & Hein 2020), several recently introduced defenses, including the ensemble model in (Pang al., (201),, were evaluated on AutoAttack, and most of them were shown to be, at best, less robust than they had been thought to according to their original papers. For instance for the ensemble defense proposed in (Pang al. 2019), it reported that the accuracy of the model dropped to zero under AutoAttack on CIFAR- 10. In Tables 7 we show the results for the evaluation of proposed model under AutoAttack. The results indicate that the DML-based ensemble models exhibit notable adversarial robustness.
7 CONCLUSION
In this paper, we introduces new Mahalanobis distance based ensemble models, in which the prop- erties of the distance function are leveraged to increase the diversity of the ensemble predictions and reduce the attack transferability rate across the individual members. The ensemble members initial- ized in way that induces diversity, and are jointly trained while encouraging orthogonality among the inverse covariance matrices of their distance functions, which is shown to reduce transferability. The proposed mechanism can be applied to pretrained networks as priors form an ensemble with enhanced properties, which makes our model complementary to other existing defenses. Our sults show that the proposed ensemble-based defense achieves high benign accuracy and adversarial accuracy across multiple attack algorithms, including AutoAttack, without using adversarial training.
Table 8: AutoAttack (Croce & Hein 2020) results for CIFAR-10 and RESISC-45 datasets. The adversaries are crafted with the specified perturbation budgets e.
AutoAttack
Training Model
Benign Acc
4/255
= 8/255
= 16/255
CIFAR-10
DML Ensemble
81.62
40.23
16.8
6.77
RESISC-45
DML Ensemble 87.28
63.20
48.53
26.28

---

Under review as a conference paper at ICLR 2021
REFERENCES
George Adam, Petr Smirnov, Benjamin Haibe- Haibe-Kains, and Anna Goldenberg. Reducing adversarial example transferability using gradient regularization. arXiv preprint arXiv 07980, 2019.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning test time. In Machine Learning and Knowledge Discovery in Databases, Lecture Notes Computer Science, pp. 387- 402. Springer.
W. Brendel, Jonas Rauber, and M. Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In International Conference on Learning Represen- tations, 2018
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium security and privacy (sp), pp. 39-57. IEEE, 2017.
Jinghui Chen, Yu Cheng, Zhe Gan, Quanquan Gu, and Jingjing Liu. Efficient robust training via backward smoothing. arXiv preprini 2rXiv:2010.0/278, 2020.
Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865-183,, 2017.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference Machine Learning, pp- 3310-13200 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks. arXiv preprint arXiv:2003.0/6690 2020.
Thomas G Dietterich. Ensemble methods in machine learning. In Internatioaa workshop multiple classifier systems, pp. 1-15. Springer, 2000.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples, arXiv preprint 1rXiv:1412.00 1...22..62222.. 2014.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. doi: 10. 10.11099CVP..2016..90
Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song. Adversarial example defense: Ensembles of weak defenses are strong. In 11th {USENIX) Workshop on Offensive Technologies (( (WOO)) 17), 2017.
A. Kurakin, Ian J. Goodfellow, and S. Bengio. Adversarial examples in the physical world. ArXiv, 2017.
Alexey Kurakin, Ian Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, Jun Zhu, Xiaolin Hu, Cihang Xie, et al. Adversarial attacks and defences competition. In The 11 Competition: Building Intelligent Systems, pp. 195-231. Springer, 2018.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE,
2324, 1998. doi: 10. 7299779
Ling Liu, Wenqi Wei, Ka-Ho Chow, Margaret Loper, Emre Gursoy, Stacey Truex, and Yanzhao Wu. Deep neural network ensembles against deception: Ensemble diversity, accuracy and robustness. In 2019 IEEE 16th International Conference on Mobile Ad Hoc and Sensor Systems (MASS), pp- 274-282. IEEE, 2019.
Yong Liu and Xin Yao. Ensemble learning via negative correlation. Neural networks, 12(10): 1399-1404, 1999a.
Yong Liu and Xin Yao. Simultancous training of negatively correlated neural networks in an ensemble. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 29(6): 716-725, 1999b.
10

---

Under review as a conference paper at ICLR 2021
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In ICMI., 2019.
Tianyu Pang, K. Xu, Y. Dong, Chao Du, Chen, and Zhu. Rethinking softmax cross-entropy loss for adversarial robustness. In International Conference Learning Representations, 2020.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedingy of the 2017 ACM on Asia conference on computer and communications security, pp. 506-519, 2017.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):11-252, 2015.
Omer Sagi and Lior Rokach. Ensemble learning: A survey. Wiley interdisciplinary reviews, 8(4). ISSN 1942-4795. doi: https://doi.org/ 10. 1002/widm. 1249.
Sanchari Sen, Balaraman Ravindran, and Anand Raghunathan. EMPIR: Ensembles of mixed preci- sion deep networks for increased robustness against adversarial attacks. International Confer- ence on Learning Representations, pp. 12.
Mahmood Sharif, Lujo Bauer, and Michael K Reiter. n-ml: Mitigating adversarial examples ensembles of topologically manipulated classifiers. arXiv preprint arXiv: 19/2.0909/00 2019.
Christian Szegedy, W. Zaremba, Ilya Sutskever, Joan Bruna, D. Erhan, Ian J. Goodfellow, and R. Fergus. Intriguing properties of neural networks. CoRR, abs/1312.6199, 2014.
Florian Tramèr, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In Advances in Neural Information Processing Systems (NeurIPS).
Gunjan Verma and Ananthram Swami. Error correcting output codes improve probability estimation and adversarial robustness of deep neural networks, In Advances in Neural Information Processing Systems (NeurIPS).
Weitao Wan, Yuanyi Zhong, Tianpeng Li. and J. Chen. Rethinking feature distribution for loss functions in image classification. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9117-9126, 2018.
2018.
Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. Evaluating the robustness of neural networks: An extreme value theory approach. arXiv preprint 1X0I 2018.
Valentina Zantedeschi, Rémi Emonet, and Mare Sebban. Lipschitz continuity of mahalanobis distances and bilinear forms. arXiv preprint arXiv:1604.01376 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pp. 7472-7482. PMLR. ISSN: 2640-3498.
A APPENDIX
A.I CERTIFIABLE DML
Distance-man map layer is nonlinear map composed of square root of quadratic function. We consider the distance-map function
h(x)) Vxx (x
11

---

Under review as a conference paper at ICLR 2021
where My is semi-defintte positive matrix with a Choleskey decomposition denoted by Mn M LM. We let 6"-1 denote the input to the layer h. Then, we obtain
h(x)= (1r-1- - -c) ( - ((rr- c) L 1 (6- -
10 - -c) $ ||
(6)
3 6M -1| | |A Bn
On the other hand,
h( cc' -
-0r-1 c) 2. LM (d--1-0) c)
= = c))l 2 -| -11 1 = ||Andd 11- -B
(7)
Thus, the certifiable bound for h is given by
||An -|| Bn h(p0-1) 1AAn0'111 Bb
If M, is diagonal matrix, with aM maxi,j [Mij] and BM mini, MMaj]
B | Bn s 1( s t |10- 1| Bn.
A.2 ORTHOGONALITY OF INPUT-OUTPUT GRADIENTS OF PAIRED DMLS
One approach to reduce the transferability across several models is to enforce various geometric relationship like orthogonality between the input-output gradient of the models. We consider two (squared) distance map layers f, and g defined by f(x) = (x c5)' M - cy) and g(x) (x (x-cg) (x where My and Mg are the M matrices, and c and cg centers, corresponding to f and g. We have
vxf(x)= M Mycs
f(x)
and
Mgx M2 vxg(x)=
g(x)
Thus, we obtain
Vxf(x) Vx8(x)= = f(x)g(x) (x' (x'My Max My MgCs My Mgx My + My My MgCg) where we used the fact that f and My symmetric matrices. Given M Mg 0, obtain vxf(x) (xg(x)= = 0.
Thus, the orthogonality of the M matrices for f and g is a sufficient condition for the input-output gradients of and g be orthogonal.
A.3 THE THREAT
FOR
To classify the input x using an ensemble of classifiers, the output of each ensemble member on input x is computed. The ensemble prediction is performed by applying a consensus method to integrate the predictions form all individual members. The consensus method could be like majority voting, mean, median, sum. If all the members can predict with uncorrelated error, then a simple averaging may reduce the prediction error by the number of members. However, when the errors are not independent, the reduction error would be smaller than this factor. In this work, the ensemble output obtained by using the majority voting of k-top predictions of the ensemble members. In the experiments, since the majority voting operator is non-differentiable, the white box adversaries are generated by the ensemble model which is obtained via summing of the predictions and attacking
12

---

Under review as a conference paper at ICLR 2021
the aggregation layer of the ensemble. Unlike the previous research (Liu al. 2019) where the attacks are generated on a target model which is either a constituent ensemble member or a model not included in the ensemble, the white box attacks in this work are generated on the ensemble layer. Generating the attacks on a target model might provide a false sense of robustness for the ensemble, while generating the adversaries by attacking the aggregation layer of the ensemble provides the strongest white-box attack having the gradient information from all the individual members. Thus, it can measure the real robustness of the ensemble model. We also investigate adaptive attacks (He et al., (017) by crafting adversaries on all the component models of the ensemble. We show that the perturbation which required to generate adversaries on individual member is by order of magnitudes lower than the distortion required to create adversaries on the ensemble models.
A.4 DML-BASED ENSEMBLE FOR ADVERSARY DETECTION
Given an inputx; DML--ased ensemble might be utilized for detecting adversaries. More particularly if the majority number of ensemble members which is determined by the threshold v do agree on class of the input class, the ensemble classifies the input as the majority class. Otherwise the input is identified as adversarial example. To elaborate, we let fx( is the output of the t classifier on the inputxx x and = arg max (ki(x). Then the output of ensemble classifier fens on x is given by
if [(k] arg (a fai(s) = =1, E [K]] fens = = adversary otherwise.
(8)
Evidently, increasing the threshold rises the probability of detecting adversaries, while the number of false positive cases might grows at the same time and therefore the benign accuracy of the ensemble model decreases. Practically, the threshold v in (8) is selected as L + If = K (i.e., the number of ensemble members), the majority framework is called unanimity framework were the ensemble classifies the input x only if all the classifiers agree on the class of input x. Unanimity framework poses security risk if the adversarial examples cause the misclassification to the same class over models. Otherwise a defender can use the disagreement between models as a mechanism to detect adversarial examples. An ensemble with diverse DML-based members is less likely to mislcassify the adversary inputxx to the same class. Thus, the detector becomes more powerful when the predictions from constituent members of ensemble are diverse. Due to the shuffled representation of neighbour classes' domain to the decision region of class x for the DML-based models, the adversaries for the sample x is misclassified diversely (i.e., diverse labels) across the ensemble members. So the likelihood of two classifiers to misclassify the ad adversary input to the same class would decrease and increases the probability to detect x. To insure the diversity in the centers form DML, layer of the ensemble members, we shuffle the centers of classes per each individual classifier. Note that the diversity does not necessarily change the shape of the decision boundaries and only alter the adversarial directions. In the other words, it could cause the classifiers to provide various prediction on a given input that could be leveraged to detect adversaries. It is important to note that, unlike previous approaches on ensemble model (Sharif et al. 2019), imposing this diversity does not imply any upper bound in the number of constituent classifiers in the ensemble as there are infinitely many ways to have the centers of classifiers shuffled. In previous research (Sharif et al.) 2019 Liu al.) 2019). the authors had to split the classifiers to plenty of ensembles in order to achieve diversity inside each ensemble.
A.4.1 UNANIMITY FRAMEWORK FOR ADVERSARIAL DETECTION
We might use the ensemble of DML-based networks to detect adversarial samples based on unanimity framework. In this setting, the ensemble provides the prediction for label of input x, only all the individual members agree on the label of the Otherwise, the ensemble renders the sample as an adversarial example. In Figure 3 we have shown the ROC plots and AUC scores for adversarial detection using unanimity framework over three datasets. For each dataset we considered testset of correctly classified benign samples and the corresponding adversarial samples crafted with white-bon attacks FGSM and PGD. The parameters for the attacks used are similar to the ones in Tables I0 The results show the remarkable performance of DaLed ensemble with unanimity framework as detection metric.
13

---

Under review as conference paper at ICLR 2021
BCCC
Cor
13
44
1801
(a) MNIST
(b) CIFAR-10
(c) RESISC-45
Figure 3: ROC curves for adversary detection using unanimity framework. The perturbation size e is 0.3, 8.0/255 and 8.0/255 for MNIST, CIFAR-10 10 and RESISC-45 datasets, respectively.
A.5 ATTACK DETECTION MECHANISM 1-D PROJECTION
Shuffling the centers in high dimensional space does not necessarily lead to diverse adversarial directions as high dimension the decision boundaries of one class to the other class could be numerous. To circumvent this issue promote the diversity of the adversarial direction by taking advantage of shuffled centers, we restrict the embedding space to a one dimensional centers, where the center location for each class per each model the classifier is just a shuffled representation of fixed vector providing the center locations. In Figure fit is observed that shuffling the centers in 1-D can significantly increase the rate of adversarial detection.
(a) MNIST
(b) CIFAR-I0
(c) RESISC-45
Figure 4: ROC curves for adversary detection using unanimity framework with I-D mapping in the embedding space. The perturbation size is 0.3, 8.0/255 and 8.0/255 for MNIST, CIFAR-10 and RESISC-45 datasets, respectively.
A.5.1 ADVERSARIAL DETECTION USING UNCERTAINTY METRIC
We define the uncertainty distribution of the model at sample points as the variance of outputs of models across ensemble members and for multiple inferences. The quantified uncertainty of the randomized-smoothinng DML-based network is utilized to detect adversarial examples. Since there is one output for each class in the network, the mean of the uncertainties for all classes indicates the uncertainty of the model at the sample x. In Figure we have shown the uncertainty distribution of FGSM and PGD adversarial examples versus normal examples. Apparently, the distribution from adversarial examples is distinguishable from normal and noisy samples. We use this intuition to train logistic-regressien model on the inputs of uncertainty of data to detect adversarial examples.
Table 9 we show the adversarial accuracy of the same ensemble models we considered in Tables 3 for the randomized smoothing DML layer indicated in equation 4 To compute the randomized
smoothing DML layer with Konte-Carlo approximation formulation (5). we set T = 20 and the
noises generated from the normal distribution (00, o²1) with that the adversarial accuracies of white box attacks for randomized smoothing DML outperform the corresponding results for ensemble of DML based models.
0.1. It is seen in Table
14

---

Under review as conference paper at ICLR 2021
100
300
200
0.8238)
100
9.915!
Nakyy delvesorial Data (Aus 123380
Nobyy Asyersania Data (6c) 0.20011
6/0
Uncertarity
Encoriarry
(a) FGSM
(b) PGD
Figure 5: Uncertainty distribution.
Table 9: Accuracy of extended white box attacks on DML based ensemble (DML-ensemble) and randomized smoothing DMI. ensemble (R-DMI-eensemble) the perturbation budget specified for each dataset. The attacks are performed with 5 random restarts.
Dataset
Benign Acc
FGSM PGD-20
PGD-40
MNIST 0.3))
DML-ensemble
93.66
42.04
36.18
35.37
R-DML-ensemble
93.50
44.98
42.07
41.73
CIFAR-10 8/255)
DMI.-ensemble 84.57
47.72
19.38
14.46
R- R-DMLeensemble 81.25
56.48
38.12
37.34
We could also detect adversaries using the uncertainty quantity measured using randomized DML. As shown, the benign, noisy and adversarial data are representing different uncertainty distribution. Based on this observation, we train bined benign and noisy data as the negative class and the uncertainty from adversarial data as positive class. We use the testset of correctly classified samples for benign samples, and crafting noisy, and adversarial examples. The adversaries are generated using from the FGSM and PGD attackers. The trained model classifies benign and adversarial examples. We show the ROC plots and AUC scores of the classifier in Figure 6 The ROC plots demonstrate that the uncertainty from randomized DML might be used indicator to detect if sample is adversary.
classifier from the uncertainties of a com-
(a) MNIST
(b) CIFAR-10
(c) RESISC-45
Figure 6: ROC curves for adversary detection using uncertainty distribution. The perturbation size is 0.3, 4.0/255 and 8.0/255 for MNIST, CIFAR- 10 and RESISC-45 datasets, respectively.
15

---

Under review as conference paper at ICLR 2021
150
60
CUCVEn score
CLEVER
(a) Baseline
ensemble
Figure 7: Distribution of local CLEVER score for baseline ensemble and DML-based ensemble for 1000 randomly selected images from CIFAR-10 dataset with a CLEVER score of higher than the threshold 0.2. The CLEVER scores are computed on the L2-norm ball with radius 2.0.
120
100
Castes
40
(a) Baseline
(b) DML-based ensemble
Figure 8: Distribution of local CLEVER score for baseline ensemble and DML-based ensemble for 1000 randomly selected images from RESISC-45 dataset with a CLEVER score of higher than the threshold 0.2. The CLEVER scores are computed on the L2-norm ball with radius 2.0.
A.6 CLEVER SCORE
In Figures() [ we depicted the histogram of CLEVER score (Weng et al.) 2018) of 1000 randomly selected images of the model derived by summing the prediction of individual constituent models in the ensemble for CIFAR-10 10 and RESÍSC-45 datasets. The results is shown for samples that have higher CLEVER score than the threshold 0.2. CLEVER score provides an upper bound for the radius of certified ball around each sample in L2 norm. The figures show that the scores from DML-based ensemble outperform of the counterpart of baseline model. For instance for CIFAR-10 dataset the number of samples with the CLEVER score greater than 1.0 is less than 15 samples, however for DML--aseed ensemble around 250 samples have the score of 2.0.
A.7
ORTHOGONALIIT OF INPUT-OUTPUTS GRADIENTS
Now we demonstrate the effect of pairwise orthogonality of the matrices for DMLs across the ensemble imposing by regularized y in (3) on orthogonality of the input-output gradients of constituent models. For this purpose we design an ablation study for a ensemble of two models. Once train the ensemble model with y = 0.0 and once the ensemble model is trained with y 1.0. The histograms for the cosine similarity input-output gradients of the two models the ensemble over three datasets is shown in Figures 9 I0 and [1] For each dataset we have shown the cosine similarity of models' gradients and also for the sign of gradients. Iterative attacks like PGD apply the sign of gradients in order to craft the attacks. The figures show that the gradients for the models were
16

---

Under review as conference paper at ICLR 2021
600
0.30
millarty
Coaine simsinty
(a) gradient
(h) gradient sign
Figure 9: Distribution of the cosine similarity of the gradients with respect to inputs for two models the DML--based ensemble trained with the regularizer term for imposing orthogonality of the M matrices (y 1.0) and the DML based ensemble trained without regularizer term (y = 0) for MNIST dataset.
1800
y- 0.00
y=200 0.20
on
3200
002
300
200
400
-0.2
-0.50
-0.25
9.25
0.50
0.75
150
-0.1
0.0
0.2
0.6
Lothne 5 milartt
6 stribrrty
(a) gradient
(b) gradient sign
Figure 10: Distribution of the cosine similarity of the gradients with respect to inputs for two models in the DML-based ensemble trained with the regularizer term for imposing orthogonality of the M matrices (y 1.0) and the DML based ensemble trained without regularizer term (y 0) for CIFAR-10 dataset.
trained with y = are more aligned compared to the ones with 1. The results RESISC-45 is not prominent as the cosine similarity of the gradients of ensemble models trained with y 0.0 already close to zero. The results from these figures confirm that counting on the regularizer in the loss function impose the orthogonality of the M matrices of paired DMLs would significantly reduce the cosine similarity between the gradients of loss function with respect to the input data for paired models in the ensemble.
A.8 EPFECTIVENESS OF R-DML IN ADVERSARIAL TRAINING
In this section we illustrate empirically that R-DML can provide better benign and robust accuracy when compared to standard DML for adversarial training. To better distinguish the performance of R-DML from the randomness obtained by augmenting data with Gaussian noise, addition we also perform experiments for DML where the input data is augmented with the isotropic Gaussian noise, i.e.,
(xx) = B5-N00, (x 5 cy 55 cy)
We call this layer augmented DML BAUG-DM),, the input to DML augmented with random noise. In Figure [2 we have shown the benign and robust validation accuracy. The validation set is equal to the testset. Higure 133 also show the same counterparts for DML-bbased models which obtained
17

---

Under review as a conference paper at ICLR 2021
y 5.00
V=000
200
130
52
0-
0.33
Coline simtarty
Coline samiinty
(a) gradient
(h) gradient sign
Figure 11: Distribution of the cosine similarity of the gradients with respect to inputs for two models the DML--bsed ensemble trained with the regularizer term for imposing orthogonality of the M matrices (y 1.0) and the DML based ensemble trained without regularizer term (y = 0) for
RESISC-45 dataset.
DML
340
DML
57
R-DML
AUG-DDL
335
R-DML
ALXG-DML
Valdletton sccuracy
335
Valkiitioo
0.0
323
320
Epoch
Epech
(a) Benign validation accuracy
(b) Robust validation accuracy
Figure 12: Benign and adversarial accuracy for adversarialy train models with FGSM attack without random initialization.
by combining DML with the models which are adversarialy trained. In other words for the models in Figure I3 we use prior models which are adversarialy trained. All the models we consider in this section were trained adversarialy with one-step FGSM attack with the stepsize 10.0/255 and without random initialization. The results Figure 12 show that R-DML marginally show better benign and robust accuracy, while the results from AUG-DML present similar performance as DML. It shows that the noise injected to DML by convolving Gaussian distribution with the matrix is more effective compared to the DML obtained by augmenting inputs to DMI. with random noise. Also the results from Figure 13 for R-DML show a significance improvement over DML and AUG-DML. It is important to note that with the higher number of epochs in contrary to DML, R-DML avoids over-fitting It is also seen that standard DML based network achieve the peak of benign and robust accuracy with only 2 epochs. although the results underperform the ones from R-DML after in the long run training. It shows the DMLs can be applied effectively with pretrained adversarialy trained priors to generate diversity across the model members.
A.9 RANDOMIZED SMOOTHING AND CERTIFIED ACCURACY
In this section we examine the randomized smoothing model obtained by convolving the base model with the isotropic Gaussian noise (Cohen et al. 2019). Our hypothesis is that the certified accuracy for smoothed classifier with R- R-DML is providing tighter certified accuracy versus the standard DML. The reason is that R- R-DML explores some points in the embedding space of the classifier for certifying the samples which might be overlooked in the standard DML. case when the input data augmented
18

---

Under review as conference paper at ICLR 2021
34
114
Valdeekn
Valldation
1.
DNL
DML
R.DML AUG-ONL
R.MML
NUG DML
Epoch
(a) Benign validation accuracy
(b) Robust validation accuracy
Figure 13: Benign and adversarial accuracy for adversarialy train models with FGSM attack without random initialization. The DML based models formed with priors which are adversarialy trained.
DNL
DEL
RE AUS-CAL
UHL
870
A.C-OML
05
64-
conffc 200r00Y
0:5
03
0.0
0.55
Ivliis
(a)
(b)
Figure 14: Certified accuracy for three types of DML-based model trained standardly. The standard deviation to compute certified accuracy is (a) or = 0.1 and (a) or = 0.25, The standard deviation for the Gaussian noise of R-DML and AUG-DML or 0.5.
with Gaussian noise. The reason for that is since we apply DML as a last hidden layer, augmenting features with random noise could add more control on exploration of embedding space. Given that, those embedding points may have different classification results in a tighter bound for the certified accuracy by DML. We also demonstrate that AUG-DML defined in the last section does not lead to dramatic change in the certified accuracy compared to R-DML., which supports the significance of applying R-DML. Similar ideas on inserting noise to the embedding space to explore overlooked regions based on data augmentation has been recently explored in for example (Chen et al. 22220). However in this work, the noise is backpropagated in the network to find the mapped location for the noise in the input space, which add additional computational burden to the training procedure. However, in R-DML the noise in embedding space freely added by convolving R-DMI. versus DMI. and AUG-DML. Note that these models are trained regularly and not through randomized smoothing training procedure. The random noise for training R-DMI. and AuG-DMI. which is applied is isotropic Gaussian noise with standard deviation of 0.5. The certified accuracy for models in this section is measured over 1000 randomly selected samples. Figure 14 shows that R-DML basically provides better certified accuracy compared to the other two DML based models.
the original sample. Thus, R-DMI
the M matrix. Figure we we have shown the certified accuracy for
In Figure 15
we have shown the certified accuracy of smoothed classifier based on standard DML. The figure also shows the certified accuracy of smoothed classifier, when the DML in the smoothed classifier is replaced by R-DML and AUG-DML. This figure shows that R-D R-DML provides tighter bound for the certified accuracy of smoothed classifier by DMI..
19

---

Under review as conference paper at ICLR 2021
6M8
DML 10%
D011
030
055
040
9EL1
(a) Ge 0.25, OR = 0.1
(b) 0ec = 0.5, (TR = 0.1
0.8
TM1
DML.
650
0.5
(c) Gr 0.25, OR 0.25
(d) 0c 0.5, O'R 0.25
Figure 15: Certified accuracy for smoothed classifier obtained by randomized smoothing of the classifier with
DML. The curves corresponding R-DML and AUG-DML show the certified accuracy when the DML layer replaced by R-DML and AUG-DML where the Gaussian noise is oR. ar denotes std of Gaussian noise used to compute certified accuracy.
Table 10: RESISC-45 extensive white box attack results by scaling the M matrix. FGSM and PGD applied with specific perturbation scale CW is used with the specified penalty parameter y.
FGSM
FGSM
BIM-10
PGD-10
CW
Training Model
Benign Acc
0.015
0.03
0.015
0.015
y
Baseline
85.66
3.75
1.34
0.00
0.00
45.22
DML Ensemble
88.71
44.40
33,24
23.93
25.76
32
In Figure 16 we have shown the certified accuracy for smoothed classifier with three types of DML layer. We then removed the DML layer and replaced it with R-DML measure the certified accuracy. seen that the model trained with AUG-DML does not not tolerate the noise from R-DML and failed provide certification for any sample. The smoothed classifier trained by R-DML could provide high certified accuracy.
A.9.1 EFFECT OF META-MODEL ON PROMOTING DIVERSITY
To show the effectiveness of group-splitting mechanism discussed earlier, we trained the RESISC- 45 dataset without this mechanism and only by scaling the M matrix to impose diversity across members. Figure I7 shows that under this experiment setting, no remarkable diversity in the shape of loss function across members created. The corresponding results for the robust accuracy and rate of transferability as presented Tables I0 model trained with metal description.
12 show lower performance compared to the
20

---

Under review as conference paper at ICLR 2021
D011
BELL
9EL1
(a) Te. 0.25, OR = 0.1
(b) (Tc = 0.5, (TR = 0.1
TM1
AUG
DML
64
(c) Gr 0.25, OR 0.25
(d) 0e 0.5, O'R 0.25
Figure 16: Certified accuracy for smoothed classifier obtained by randomized smoothing of the classifiers
with three types of DML: DML, R-DMI and AUG-DMI. To calculate the certified accuracy the DML layer for each smoothed classifier replaced with R-DMI. where or denotes denotes the std of Gaussian noise used to compute certified accuracy.
Gaussian noise for R-DMI., de
Table 11: Adversarial accuracy of models when the attacks are crafted on the target Model 1 with PGD-10. The labels form the targeted attacks are selected randomly and different from labels. The size of perturbation is0.015. The model trained without using meta-model and diversity imposed only be scaling the matrix per each ensemble member.
Dataset
Ensemble
Model
Model 2
Model
Model
Model 5
RESISC-45
Benign Acc
88.71
83.80
86.42
85,40
87.69
84.80
Untargeted
67.96
20.40
68.22
64.91
70.31
69.21
Targeted
79.16
21.36
77.73
74.96
79.40
77.24
A.9.2 DISTORTION-ACCURACY PLOTS
In Figure 18 the distortion-accuracyy plots are shown for FGSM, BIM and PGD white-box attacks and over all three datasets. It is observed that increasing the distortion scale leads to almost 0% accuracy the ensemble models which could be an indication that the DML-based models do not mask the gradient information, i.c., the attackers are applying interpretable gradients from the model.
In Figures 19 21
visualize samples of original images and the corresponding adversarial images generated from Table 5 for all three datasets. These figures show that the DML-based ensemble largely increases the required distortion such that the distortion is highly perceptible.
21

---

Under review as conference paper at ICLR 2021
The rate of required L2 distortion for adaptive CW-L22 attack (He et al. 2017) with y = for the combined and individual ensemble members of size 5 for 100 randomly selected images. The number of iterations and stepsize are 1000, 0. respectively. x is denoting the confidence parameter for CW-L2 attack. The model trained without using meta-model and diversity imposed only be scaling the M matrix per each ensemble member.
Table 12:
L distortion required for generating adversaries on each model
Dataset
Ensemble
Ensemble
Model
Model
Model
Model
Model
K ==
3
RESISC-45
Adversarial accuracy
70.00
2.00
0.00
0.00
0.00
0.00
0.00
Average distortion
3.69
182.90
2.77
2.72
2.73
2.82
3.32
(a) Model
(b) Model
(c) Model
Figure 17: Loss surface for three individual classifiers from the ensemble model without group-wise splitting mechanism. The loss depicted around the sample x, with the label ys from RESISC-45 dataset. The noise vector P is drawn from the normal distribution (O0, 0.05). The loss surfaces does not show significant diversity across members.
RCOM
EM
0212 131
8
92
(a) MNIST
(b) CIFAR-10
(C) RESISC-45
Figure 18: Distortion-accuracy curves for adversarial examples crafted by FGSM and PGD white-box aattacks over three datasets.
22

---

Under review as conference paper at ICLR 2021
Original
2
3
4
5
6
7
8
9
Ensemble
2
4
5
8
M M1
5
3
4
5
6
8
9
2
4
5
6
7
8
9
M2
3
3
4
5
6
8
M 3
9
3
4
5
6
8
9
M 5
2
3
4
5
6
8
Adversarial samples generated by CW-I.2 attack (x=0.0) on the original samples from MNIST dataset (first row), and on Ensemble model (2nd row), Model (3rd row), Model 2 (4th
Figure 19:
row), Model 3 (5th row), Model 4 (6th row), Model (7th row).
Ensemble Original
M 1
M
M 3
M
Figure 20: Adversarial samples generated by CW-L2 attack (x = 0.10) on the original samples from CIFAR- dataset (first row), and on Ensemble model (2nd row), Model (3rd row), Model (4th row), Model 3 (5th row), Model 4 (6th row), Model 5 (7th row).
23

---

Under review as conference paper at ICLR 2021
Ensemble Original
M 1
M2
M3
M4
M 5
Figure 21: Adversarial samples generated by CW-L2 attack (x = 0.10) on the original samples from RESISC-45 dataset (first row), and on Ensemble model (2nd row), Model (3rd row), Model (4th row), Model 3 (5th row), Model 4 (6th row), Model 5 (7th row).
24


Reviews:
[{"review_id": "wjhdxxCsNE", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper aims to improve the robustness of ensembles of neural networks to adversarial attacks. If the models in the ensemble are susceptible to similar attacks, then the ensemble is also vulnerable. This has been shown to be true in practice (He et al.). The authors propose appending distance map layers (DMLs) to the learned representations (i.e., in place of logits) of the member models. The DMLs measure distance to randomly initialized centers using a learned Mahalanobis distance metric. A fraction of the distance matrix, M, is learned such that member M's are approximately mutually orthogonal (thereby decorrelating predictions), and the entries have low L1-norm. The member models' top predictions for each sample are then aggregated by majority vote. The introduction of DMLs is shown to improve robustness to adversarial attacks without a large degradation in accuracy on benign samples. Randomizing the distance metric a bit helps with robustness as expected but hurts benign accuracy also as expected.\n\nThe idea of learning orthogonal distance matrices is interesting. I would have liked to see experiments comparing gamma=0 to gamma>0 (I could not find what value of gamma was used in the experiments) to better understand its impact. Even so, I don't understand how orthogonal matrices could be learned effectively with only two free parameters per 10 x 10 matrix.\n\nThe distance matrices are learned via backprop on equation (3). Alternatively, the ensemble approach taken appears very similar to a more conventional approach: 1) learn a metric space over the feature representation (i.e., chop off the penultimate layer of the DNN) assuming KNN (k=1); 2) take a majority vote over each model's prediction. See \"Distance Metric Learning for Large Margin Nearest Neighbor Classification\" '09 JMLR for how to accomplish 1).\n\nThe authors emphasize that a low Lipschitz constant is important for improving robustness, but I don't see how it matters if the ensemble is aggregating with majority vote over each member's top prediction.\n\nThis peculiarity aside, there are issues with the comparison of the Lipschitz constant for the DML-ensemble as a sum of member models vs the baseline ensemble. 8/10 entries on the diagonal of the distance matrices are fixed to 0.1, so I would expect the Lipschitz constant to be reduced by a factor of about 10 without any learning. This could also be easily accomplished by using the member models as originally intended and introducing a temperature into the softmax (divide logits by 10). The text claims that Figure 1 reveals the mean of the baseline is 3 orders of magnitude larger than the DML-ensemble; by inspection, this looks more like 1 order of magnitude and can be explained by the fact that most of the entries in the matrices are fixed to 0.1.\n\nWhat $\\kappa$ is used for the CW-$L_2$ attack on the individual models in Table 5? Also, only CIFAR-10 shows an order of magnitude difference and that's only if you look at $\\kappa=0.1$ despite what's written in the text \"order of magnitudes lower than the distortion required to create adversaries\".\n\nWhy aren't the results for targeted and untargeted the same for Model 1 in Table 4 given that the adversary targets Model 1? Am I misunderstanding the experimental setup?\n\nModel 2 and Model 3 look like they have similar gradients contrary to what's said in the text about Figure 2.\n\nOverall, the direction seems promising, but the paper needs further work.\n\n\nQuality:\nThere are some interesting contributions in this paper, but I'm not sure the ultimate model used in the experiments really reveals how important they are. Only learning two entries per distance matrix seems small and I don't see the learned orthogonality reported anywhere. The experimental results are surprising if only a total of 10 additional parameters are learned.\n\nClarity:\nThe paper is clear at a high level, however, I had several questions (see above and below) and there were many typos.\n\nOriginality:\nTo my knowledge, learning orthogonal distance matrices for each member in an ensemble is novel.\n\nSignificance:\nImproving adversarial robustness is an important area of research.\n\n\nMinor:\n\nIn the page 4 Errata note below, you probably mean \"\\emptyset\" $\\emptyset$, not $\\phi$?\n\nHow are the indices of I divided among the ensemble members? Does the 1st ensemble learn M1 = diag([a, b, fixed, fixed, ...]) and the 2nd ensemble member learn M2 = diag([fixed, fixed, c, d, ...]) and so on?\n\n\"set of priories\" - set of priors\n\nRemember to add parentheses around citations (\\citep).\n\nDoes I-covariance stand for inverse covariance matrices? Why introduce this new term? Maybe just stick with M if nothing new is being introduced.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Qq9eBM2HjUT", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper concerns on developing neural network ensembles that can avoid adversarial attacks. The authors propose a new concept of Distance Map Layers (DML) that can be used as the one just before the final layer in a neural network for classification. DML is mainly used to improve the diversity of predictions from the ensemble members, and is defined to be the Mahalanobis distance between an input vector and a output center which may correspond to a class. Each member in the ensemble will learn a different Mahalanobis measure, encoded in the inverse covariance matrix (M), as well as the centers. The training tries to ensure that any two different ensemble members should have inverse covariance matrices of small $L_1$ norms and pair-wise orthogonality, and their set of centers should be far from each other. The authors further propose a randomized DML in which a noise is added to M to help the ensemble to be more robust. To validate the effectiveness of their approach, 3 datasets, 3 network architectures, and 5 adversarial attacks are used in evaluation. Their extensive experiments demonstrate that the DML-based ensemble can perform significantly better than the one without DML, and the randomized DML can perform much better than DML. \n\nPros:\n- The proposed DML is intuitive and promising, especially randomized DML. The idea can be easily used in different contexts.\n- The experiments are extensive and really promising.\n\nCons:\n- The notations are inconsistent across the paper. For example, M is used for both dimensionality and inverse covariance matrix. The DML sometimes use squared root (in page 3), but sometimes square (pages 5, 10). Formula (1) is also confusing/wrong.\n- The overall presentation is hard to understand and misses many details. For example, there is no section index to refer and follow. Some appendices sometimes are referred without index, causing some difficulties.\n- Some formulas needs being revised carefully, e.g., (1), (2), (4), and the proof about the product of two Jacobians in appendix.\n- The use of citation should be revised. The authors (quite arbitrarily) mix citations with the main text, which is not a good way.\n- It is unclear why one needs some randomness for DML. The authors should provide some reasons behind it. Such a randomness causes some inconsistencies in the obtained Mahalanobis distance, e.g., when $M_f + \\delta_i$ is not positive semi-definite. What it means in this case? It is not supportive when the authors say “we apply the uncertainty of Gaussian process to identify the low-confidence regions of input space to the DML layer”. Some theory may be needed.\n- When working work with a binary classification problem, the proposed DML-based approach may not provide a good solution. The authors should discuss about this situation and make a comparison with other approaches.\n- How to choose/initialize the fixed parts of the inverse covariance matrices. The authors should provide an analysis about this point as it can significantly affect the performance of their method.\n- An analysis about the sensitivity of some parameters in their method is also needed, e.g., the effect of the randomness in initialization of the centers, and of $(\\lambda, \\gamma, \\beta, \\sigma)$. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "EJdofBpS6rJ", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Important: Formatting of section titles - please appropriately number section titles, it makes it very hard to refer to sections otherwise.\n\nThe authors address the issue of building robust neural networks by training them using an ensemble based loss function which is designed to promote a sense of orthogonality among ensembles.\n\nThe main concern in robust ML is that many attacks are transferable between models, leading to blackbox attacks. This brings the merit of ensemble based methods down -- even though ensembles lead to better accuracies when no attacks are present, attacks which apply to one of the members in the ensemble, can also weaken accuracy of other members in the ensemble and therefore ensembles cannot be stronger against adversarial attacks. \n\nIn this paper, the authors derive a new methodology to promote \"orthogonality\" between different ensembles - which should help reduce the problem of attack transferability. Note that this idea has already been introduced in [1]. \n\nThe main contribution in this paper is to introduce Distance Map Layers (DML) as an additional layer which maps different class vectors to a class based on their distance from a learnable centroid (each class corresponds to a centroid). This mapping is formulated using Mahalanobis distance, characterized by matrix M. They propose the notion of diversity among ensembles by ensuring that these centroids are at least alpha margin apart for a fixed class, between two different ensembles. This formulation can be seen as an extra regularization term, as can be seen in Eq.(3).\n\nIn the experiments section authors demonstrate how when ensembles are trained with this modified loss function, the ensemble accuracy is much better than that of its individual members when attacked by l_\\inf or l_2 type attacks. \n\nMy main concern is that this isn't the first paper to do ensemble diversity based adversarial training. One of the references also cited in the main text [1]. There are no experimental nor analytical comparisons made to this paper. Until those are made explicit, I'm not sure how the contributions of this paper have an edge over [1]. \n\nPlease fix typos:\nPage 2: \"The classification loss function will encourage the distance to the target class to be small while the distances to all other classes to be small, which induces compactness in the embedding space.\" - other classes to be *large*\nPage 4: \"we select In's such that Ij \\interct Ik \" -incomplete sentence ---  = 0. \nPage 6: Fig 1 caption: \"Distribution lo ...\" - Distribution of \n\nReferences:\n[1] \"Improving adversarial robustness via promoting ensemble diversity\", Pang et. al., ICML, 2019.", "extra_scores": {}, "extra_text": {}}, {"review_id": "4A2QeTBWkVi", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Post-rebuttal:\nMy concerns are not addressed, so I would like to keep my original score.\n-------------------------------\nThis paper proposes to use the distance map layers for ensemble-based defense. By randomly choosing the centers to vary over classifiers, and imposing the I-covariance matrices to be dissimilar and possibly orthogonal, the distance map layers can be used to reduce the rate of transferability of adversarial attacks across models in the ensemble. This paper also demonstrates that distance map layers provide spontaneous regularization of the Lipschitz constant, and therefore further boost the robustness. \n\nI like this paper, but I have some comments as follows:\n1. The matrix M in Eq(1) can be represented as M = L^T L. The Mahalanobis distance is equal to \n‖x_1-x_2 ‖_M =√((x_1-x_2 )^T LL^T (x_1-x_2))\nIt denotes that the distance map layers are equal to the fully connected layers. I wonder whether the similar modifications on the fully connected layers can achieve similar performance. If not, please give the reason.\n2. There are some proposed SOTA methods like [1], Please follow the experimental settings as [1] for a fair comparison. \n3. I suggest the authors to pass this paper through someone that can do the edits to fix the grammatic and typo errors that seem glaring. For example, in eq(1)  ‖x_1-x_2 ‖_M^2->‖x_1-x_2 ‖_M ; in line 4 of abstract, transferablility->transferability …\nThus, I tend to reject this paper.\n\n[1] Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In ICML, 2019.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a method to improve adversarial robustness by diversifying the ensemble. 

Novelty: As pointed out by several reviewers, promoting diversity of ensembles has been done in the literature, but there's still a moderate novelty in proposing the DML layer. 

Empirical validations: The original submission lacks many important comparisons (e.g., with [1]). Despite the authors implicitly compared their method with (Pang et al) via Auto-attack in the rebuttal, it will be better if the comparisons are conducted in a well-controlled way to confirm that the improved robustness comes from DML instead of other hyperparameter settings. Further, it is not clear whether the proposed method is robust to hyperparameters. 

Based on these, we recommend rejection but encourage the authors to improve their paper based on the comments. 
Decision: Reject