Ensemble-based Adversarial Defense Using Diversified Distance Mapping

We propose an ensemble-based defense against adversarial examples using distance map layers ( DMLs ) . Similar to fully connected layers , DMLs can be used to output logits for a multi-class classification model . We show in this paper how DMLs can be deployed to prevent transferability of attacks across ensemble members by adapting pairwise ( almost ) orthogonal covariance matrices . We also illustrate how DMLs provide an efficient way to regularize the Lipschitz constant of the ensemble ’ s member models , which further boosts the resulting robustness . Through empirical evaluations across multiple datasets and attack models , we demonstrate that the ensembles based on DMLs can achieve high benign accuracy while exhibiting robustness against adversarial attacks using multiple white-box techniques along with AutoAttack . 1 Introduction . Ongoing research has provided defenses against adversarial examples , which are crafted from correctly classified inputs with imperceptible perturbations . Despite the success of ensemble learning as a mechanism for reducing prediction errors and improving generalization by combining predictions of multiple models performing the same task ( Russakovsky et al. , 2015 ; Sagi & Rokach ) , early research has shown the ineffectiveness of mutiple ensemble-based defenses against adversarial examples and even has gone further to suggest that ensembles are only as robust as their weak components ( He et al. , 2017 ) . If this claim is true , it basically defeats the purpose of using an ensemble , which is building a strong model out of weaker ones . Nevertheless , research continued to investigate the usage of ensembles as a defense mechanism ( Pang et al. , 2019 ; Verma & Swami ; Sen et al. ) . However , recent attempts have been quickly shown ineffective ( Tramèr et al . ; Croce & Hein , 2020 ) . We believe that one of the primary reasons for the weakness of ensemble defenses is the inter-model attack transferability . It was shown that even fundamentally different models could exhibit high attack transferability rate ( Papernot et al. , 2017 ; Kurakin et al. , 2018 ) . This phenomenon hindered the consideration of ensemble learning as a strong defense mechanism on its own . The reason is that if the member models are not robust , and they exhibit high attack transferability rate , attacks generated from one model can attack the rest , and hence , attack the entire ensemble . In this work , we show that it is possible to circumvent the problem above and instill diversity among ensemble members via the employment of specially initialized and optimized distance-map-layers ( DMLs ) , and hence throttle the inter-model attack transferability . Moreover , we demonstrate that DMLs provide spontaneous regularization of the Lipschitz constant , and therefore further boost the robustness . The rest of this paper is organized as follows : We first discuss an overview of relevant recent works on ensemble defenses against adversarial examples and background information . Then we introduce a distance map layer based on Mahalanobis distance , and also explain the threat models considered in this work . We describe the creation of ensemble of DML-based individual models . Afterwards , we introduce a randomized version of DML . Finally , we evaluate the robustness of our ensemble model over MNIST , CIFAR-10 and RESISC-45 datasets . 2 Background and related work . Not long ago , researchers revealed the vulnerability of machine learning models , particularly deep neural networks ( DNNs ) , against adversarial examples ( Szegedy et al. , 2014 ) . These models can provide incorrect predictions on examples that are slightly perturbed from correctly classified ones . The process of generating adversarial examples from natural ones is called adversarial evasion attacks ( Biggio et al. ) . Evasion attacks can be categorized into black-box , gray-box , and white-box attacks . In the black-box setting , the attacker does not have access to the model ’ s parameters and the potential defense of the model ( Papernot et al. , 2017 ; Brendel et al. , 2018 ) . In the gray-box setting , although the attacker does not have access to the model ’ s parameter , it is aware of the defense applied in the model . In the white-box evasion , the attacker has access to both the parameters and the defense applied in the model . Therefore , the attacker can apply gradient-based attack techniques ( Goodfellow et al. , 2014 ; Carlini & Wagner , 2017 ; Madry et al. , 2018 ) , or it can even design a custom made adaptive attack based on its knowledge about the model and the defense ( Tramèr et al. ) . From the defense perspective , white-box attacks are certainly the most challenging type of evasion attacks . Our proposed defense targets white-box attacks . Specifically , we focus on ! ? restricted perturbations of images , and , as it is typical in many research works , we consider the ! ∞ perturbations . Since the discovery of adversarial attacks , a significant research activity has been devoted to developing appropriate defenses ( Madry et al. , 2018 ; Zhang et al. ) . However , most of the proposed defenses have been soon defeated ( Tramèr et al . ; Croce & Hein , 2020 ) . An intuitive mechanism to defend against adversarial examples is training ensembles of other models in order to enhance their defenses ( Pang et al. , 2019 ; Verma & Swami ; Sen et al. ) . However , the efficacy of existing ensemble approaches is often faced with skepticism due to the transferability phenomenon ( Papernot et al. , 2017 ) , which makes the ensemble model believed to be at most as robust as its strongest constituent model ( He et al. , 2017 ) . In this paper , we promote the diversity among individual networks from a new perspective via distance map layers . Our construction is based on the assumption that defending against transferred adversarial example is effective , even if the individual models are vulnerable to direct attacks . Our approach is orthogonal to the previous approaches and can be combined with other generic weak or strong defenses to further enhance the ensemble ’ s resistance to adversaries . Although there is no strict illustration on what is intuitively defined as diversity , in this work , we define the diversity as the highest dissimilarity between the learned features of different classes across the ensemble members . Diversity is greater when the prediction errors of individual members are highly uncorrelated ( Liu & Yao , 1999a ; b ; Dietterich , 2000 ; Liu et al. , 2019 ) . This property may lead an adversarial perturbation to fail to fool the majority of networks in the ensemble . Topologically , diversity can be depicted as the variability in the shapes of the decision boundaries and inter-class neighborhood relationships in the embedding space . Changing the shape of the decision boundaries in the embedding space implies different loss shapes . With diversity in the shape of the losses corresponding to different models , the gradient from one model would not be a good approximation ( transferable ) to the other model . Note that due to the high capacity of DNNs , changing the topology of the embedding space can have no or negligible effect on the model ’ s accuracy . Using this perspective on model diversity , we will illustrate that DML-based ensembles could be designed to achieve the requirement of diversity in prediction errors through injecting dissimilarities only between the DMLs of ensemble members . Previous works increase the diversity over training data via promoting the diversity of prediction errors of ensemble members ( Liu & Yao , 1999a ; b ; Liu et al. , 2019 ) . 3 Technical approach . In this section , we elaborate on our model with distance map layer ( DML ) . We start by recalling the definition of Mahalanobis distance between two point G1 , G2 ∈ '= : 3 '' ( G1 , G2 ) = √ ( G1 − G2 ) ) '' ( G1 − G2 ) , ( 1 ) where `` is the inverse of the covariance matrix ( referred to as the `` matrix for the rest of the paper ) , which is a positive semi-definite matrix . Lemma 1 . TheMahalanobis distance is : -Lipschitz continuouswith : = √ 2 ‖ % ‖2 , where '' = % ) % , and % is a triangular matrix . A proof for this lemma can be found in ( Zantedeschi et al. , 2016 ) . We now introduce a mapping function , based on the Mahalanobis distance , that can be used as a differentiable layer in a neural network . The function maps an input vector to an output vector by calculatingMahalanobis distances between the input and the learned centers of the function . Equation 2 shows the element of the output vector corresponding to center 2 for an input vector G. We call this function distance map layer ( DML ) . 3 '' ,2 ( G ) = √ ( G − 2 ) ) '' ( G − 2 ) . ( 2 ) In this formula , `` ∈ R # × # and 2 ∈ R # are the learnable parameters of the DML . When the DML is used as the penultimate layer of a classification neural network ( i.e . the layer producing logits ) , each center will correspond to one class . Essentially , the layer maps an input vector to a vector of distances to class centers . Note that , in this usage , the negative of the distances has to be used to maintain the typical assumption of DNNs that the highest logit corresponds to the most likely class . The classification loss function will encourage the absolute distance to the target class to be small while the distances to all other classes to be large , which induces compactness in the embedding space . This compactness has been used in prior work for classification problems ( Wan et al. , 2018 ) and for adversarial robustness of individual models ( Pang et al. , 2020 ) . Our setting is composed of combining a classification network with a DML ℎ to obtain a model characterized by ( G ) = ℎ ◦ ( G ) where ℎ : R '' → R '' and : R # → R '' . In other words , is used as a prior to map the input space to a representation space , and ℎ maps the representation space to distances from the corresponding class centers . In the next section , we demonstrate how a distribution shift that shows with respect to could be efficiently utilized to form a diverse ensemble of models . Before that , we want to highlight an important feature about DML . If the network is ! -Lipschitz continuous , and the DML ℎ is ! ℎ-Lipschitz continuous , then the network is Lipschitz continuous with constant upper-bounded by ! · ! ℎ . Thus , imposing small ! ℎ during the training process may lead to the regularization of Lipschitz constant for the network . It is shown in the appendix that this can also largely improve the certification bounds of network compared to the model . 3.1 Enhancing diversity between ensemble members . The main part of the proposed ensemble model is the deployment of DMLs . The ensemble members stacked with DMLs are trained interactively to achieve the highest level of diversity . In particular , a DML ensemble is trained to satisfy these properties : 1 ) the ensemble members have the highest accuracy over benign samples and diverse prediction errors over adversarial samples ; 2 ) the attacks generated on one ensemble member are not transferable to other ensemble members ; and 3 ) the training procedure maintains the Lipschitz constant of an ensemble member model small . The first objective is important as it indicates that the ensemble could provide high accuracy even with the low accuracy of the individual members . The second objective guarantees a well-performing ensemble model under evasion attacks . That is an adversarial example crafted on an individual ensemble member model would normally be misclassified in that individual member , but not necessarily in the other ensemblemembers . This goal can be achieved in two folds : 1 ) by randomly choosing the centers and shuffling them to vary over classifiers in the formulation of DML ; 2 ) by imposing the `` matrices to be dissimilar and possibly orthogonal between the ensemble members . Increasing the cosine dissimilarity of DML ’ s covariance matrices of ensemble members would lead to nontransferable adversarial perturbations ( Adam et al. , 2019 ) . One approach to reduce the transferability across several models is to enforce various geometric relationship like orthogonality between the inputoutput gradient of the models . For two distance map layers 5 and 6 with corresponding `` 5 and `` 6 inverse covariance matrices , we have shown in the appendix that the orthogonality of the '' matrices , i.e. , `` 5 `` 6 = 0 is a sufficient condition for the input-output gradients of 5 and 6 to be orthogonal . Note that the explored sufficient condition is independent of the input G. Roughly speaking , the increase in the dissimilarity of the `` matrices reduces the cosine similarity of input-output gradients of the combined network by the chain rule formula . Therefore , practically , minimizing the cosine similarity of the `` matrices across models enables reducing the rate of transferability of adversarial attacks across models in the ensemble .

Summary: This paper concerns on developing neural network ensembles that can avoid adversarial attacks. The authors propose a new concept of Distance Map Layers (DML) that can be used as the one just before the final layer in a neural network for classification. DML is mainly used to improve the diversity of predictions from the ensemble members, and is defined to be the Mahalanobis distance between an input vector and a output center which may correspond to a class. Each member in the ensemble will learn a different Mahalanobis measure, encoded in the inverse covariance matrix (M), as well as the centers. The training tries to ensure that any two different ensemble members should have inverse covariance matrices of small $L_1$ norms and pair-wise orthogonality, and their set of centers should be far from each other. The authors further propose a randomized DML in which a noise is added to M to help the ensemble to be more robust. To validate the effectiveness of their approach, 3 datasets, 3 network architectures, and 5 adversarial attacks are used in evaluation. Their extensive experiments demonstrate that the DML-based ensemble can perform significantly better than the one without DML, and the randomized DML can perform much better than DML.