Manifold-aware Training: Increase Adversarial Robustness with Feature Clustering

1 INTRODUCTION . 1.1 BACKGROUND . Convolutional neural networks ( CNNs ) are increasingly used in recent years due to their high adaptivity and flexibility . However , Szegedy et al . ( 2014 ) discovered that by maximizing the loss of a CNN model w.r.t the input data , one can find a small and imperceptible perturbation which causes misclassification errors of the CNN . The proposed method for constructing such perturbations was designated as the Fast Gradient Sign Method ( FGSM ) , while the corrupted data ( with perturbation ) were referred to as adversarial examples . Since that time , many algorithms for constructing such perturbations have been proposed , where these algorithms are referred to generally as adversarial attack methods ( e.g. , Madry et al . ( 2018 ) ; Carlini & Wagner ( 2017 ) ; Rony et al . ( 2019 ) ; Brendel et al . ( 2018 ) ; Chen et al . ( 2017 ) ; Alzantot et al . ( 2019 ) ; Ru et al . ( 2020 ) ; Al-Dujaili & O ’ Reilly ( 2020 ) ) . Among them , Projected Gradient Descent ( PGD ) ( Kurakin et al. , 2017 ) and Carlini & Wagner ( C & W ) ( Carlini & Wagner , 2017 ) attacks are the most widely-used methods . Specifically , PGD is a multi-step variant of FGSM , which exhibits higher attack success rate , and C & W attack leverages an objective function designed to jointly minimize the perturbation norm and likelihood of the input being correctly classified . The existence of adversarial examples implies an underlying inconsistency between the decision-making processes of CNN models and humans , and can be catastrophic in life-and-death applications , such as automated vehicles or medical diagnosis systems , in which unpredictable noise may cause the CNN to misclassify the inputs . Various countermeasures for thwarting adversarial attacks , known as adversarial defenses , have been proposed . One of the most common forms in adversarial defense is to augment the training dataset with adversarial examples so as to increase the generalization ability of the CNN toward these patterns . Such a technique is known as adversarial training ( Shaham et al. , 2015 ; Zhang et al. , 2019 ; Wang et al. , 2020 ) . However , while such methods can achieve state-of-the-art robustness in terms of robust accuracy ( i.e. , the accuracy of adversarial examples ) , training robust classifiers is a non-trivial task . For example , Nakkiran ( 2019 ) showed that significantly higher model capacity is required for robust training , while Schmidt et al . ( 2018 ) proved that robust training requires a significantly larger number of data instances than natural training . 1.2 MOTIVATION AND CONTRIBUTIONS . The study commences by observing the distribution properties of the features learned by existing training methods in order to obtain a better understanding of the adversarial example problem . In particular , the t-SNE dimension reduction method ( van der Maaten & Hinton , 2008 ) is used to visualize the extracted features , as illustrated in Figures 1 . The observation results reveal the following feature distribution properties : • Non-clustering : same-class features are not always clustered together ( i.e. , some points leave the clusters of their respective colors in Figure 1 ) , which is at odds with intuition , which expects that the representative ( for classification ) features of same-class samples should be similar to one another . • Confusing-distance : closeness between samples in the feature space does not imply resemblance of their prediction ( especially for adversarial examples , as there are many triangles colored differently than surrounding points in Figure 1 ( b ) ) . We confirm the validity of these observations through a numerical analysis of the matching rate between the closest cluster dominant class and the predicted class . Clustering analysis algorithms ( e.g. , Ward ’ s Hierarchical Clustering algorithm ( Ward , 1963 ) ) are leveraged to find the clusters formed by the CNN-learned features . Additionally , the match rate is defined as Ex [ 1dom ( C ( x ) ) =f ( x ) ] , where C ( x ) is the closest cluster to x ( C ( x ) = argminC dist ( C , x ) ) and dom ( C ) evaluates the dominant class of a cluster ( dom ( C ) = argmaxim ( C ) i , where m ( C ) ∈ NL produces a cluster mapping vector describing the number of members of each class prediction ) . Table 1 summarizes the analysis results and confirms that both properties exist . Intuitively , a good feature extractor for classification purpose should produce similar features for all samples within the same class . On the other hand , according to Tang et al . ( 2019 ) , the existence of adversarial examples results from a mismatch between features used by human and those used by CNNs . Therefore , one intuitive approach for increasing CNN robustness is simply to drive CNN-learned features toward human-used features . However , it is impossible to understand and predict human-used features with any absolute degree of certainty . Thus , an alternative approach is to force the CNN-learned features to have some expected properties that human-used features should also have . For example , as mentioned above , features for the classification of objects belonging to the same class should be similar to one another . Based on the above observations , the present study proposes a novel training process , designated as Manifold-Aware Training ( MAT ) , for learning features which are both representative and compact . The experimental results confirm that models trained with MAT exhibit significantly higher robustness than existing state-of-the-art models . It would be clear later that our idea is , in some sense , similar to Pang et al . ( 2020 ) in that the authors proposed the Max-Mahalanobis Center ( MMC ) loss , which minimizes the distance of features to their assigned preset class centers . By showing that robustness of the model using MMC with adversarial training is higher than that using simply adversarial training , the authors claimed that high feature compactness results in locally sufficient samples , which are beneficial for robust training according to Nakkiran ( 2019 ) . Their conclusion of feature compactness helps robustness matches our idea . The main contributions of our study can be summarized as follows : • A better understanding of the relationship between robustness and the distance between adversarial examples and clean samples in the MAT-learned feature space . • Our method improves the state-of-the-art performance from 57 % to 80 % for CIFAR10 and from 96 % to 99 % for MNIST in terms of robust accuracy . The main notations used in present study are summarized in Appendix A 2 RELATED WORKS . One family of adversarial defense methods , referred to as adversarial-training-based methods , augments the training data with adversarial examples . For example , Zhang et al . ( 2019 ) defined the so-called TRADES loss , which is based on the trade-off between the clean accuracy ( i.e. , the accuracy of clean images ) and the robustness accuracy and has the form min θ 1 N N∑ i=1 J ( x ( i ) , y ( i ) ; θ ) + max x′ ( i ) ∈B ( x ( i ) ) J ′ ( x ( i ) , x′ ( i ) ; θ ) /λ , ( 1 ) where B ( x ( i ) ) denotes the lp-bounded ball ( B ( x ) = { x′ | ‖x′ − x‖p ≤ } ) , and J ( · , · ; θ ) and J ′ ( · , · ; θ ) are implemented using the cross-entropy loss and the Kullback-Leibler divergence criterion , respectively . Ding et al . ( 2020 ) proposed the Max Margin Adversarial ( MMA ) training method , which minimizes the cross-entropy loss for misclassified samples and maximizes the margin between them and the nearest decision boundaries for correctly classified samples . The authors proved that by minimizing the cross-entropy loss of the samples on the decision boundaries , the margin can be maximized . Wang et al . ( 2020 ) found that misclassified samples bring more robustness when used in adversarial training than correctly classified ones . Inspired by this observation , the authors proposed an objective function that encouraged stability around misclassified samples and optimized the classification loss for adversarial examples . While the adversarial-training-based methods achieve state-of-the-art robust accuracy , they inherit some limitations ( Nakkiran , 2019 ; Schmidt et al. , 2018 ) . Some studies try to alleviate these limitations to further improve the CNN robustness . For examples , according to the work of Schmidt et al . ( 2018 ) , robust training requires significantly more samples than natural training . However , high sample density may make the training samples locally sufficient for robust training . Therefore , Pang et al . ( 2020 ) proposed the Max-Mahalanobis Center ( MMC ) loss as follows : LMMC ( φ ( x ) , y ) = 1 2 ‖φ ( x ) − µ ( y ) ‖22 , ( 2 ) where φ ( · ) is a CNN feature extractor , and µy denotes the pre-set class center for class y. Additionally , the authors analyzed the limitations of traditional softmax and cross-entropy ( SCE ) loss based training methods regarding the induced sample density in the feature space . The authors then proved that the feature density induced by the MMC loss is guaranteed to be high around the centers ( µ ( y ) ) . To our knowledge , however , improving the robustness of CNNs by manipulating the feature space attracts little attention . In view of this , we propose a novel defense method by training CNNs with loss functions designed to induce feature compactness and improve the state-of-the-art robustness significantly . 3 PROPOSED METHOD . The proposed Manifold-Aware Training ( MAT ) algorithm is introduced in Subsection 3.1 , and two additional loss functions to further improve the performance of MAT are described in Subsection 3.2 . Moreover , a transformation technique designed to architecturally transform MAT-trained CNNs into traditional CNNs for compatibility with existing techniques is presented in Subsection 3.3 . 3.1 MANIFOLD-AWARE TRAINING ( MAT ) . Recall the observed properties of features learned by traditional CNNs , i.e. , the non-clustering property and the confusing-distance property ( see Section 1 ) . This subsection introduces our proposed loss function to achieve intra-class feature compactness and inter-class feature dispersion . Specifically , the loss function minimizes the distance of CNN output features to their corresponding class centers while the class centers are far from one another . According to Pang et al . ( 2020 ) , fixing class centers during the training phase makes better performance than updating them while learning for another objective ( e.g. , intra-class feature compactness ) since the CNN can focus on one objective rather than seeking a trade-off between multiple objectives . To ensure the pre-defined class centers are far from one another ( to ensure inter-class dispersion ) , the distance between the samples is measured using the cosine-distance ( CD ( a , b ) = 1− cos ( a , b ) , where cos ( a , b ) = ( a ·b ) / ( ‖a‖2‖b‖2 ) ) and the class centers are generated using the algorithm proposed by Pang et al . ( 2018 ) , i.e. , the Max-Mahalanobis Distribution ( MMD ) centers . The generated center vectors are the vertices of an L-dimensional simplex such that the included angles between class center vectors are maximized , i.e. , { µ ( 1 ) , ... , µ ( L ) } = argmin µ max i 6=j µ ( i ) · µ ( j ) , ‖µ ( k ) ‖2 = C ∀ 1 ≤ k ≤ L. ( 3 ) Note that the details of the generation algorithm and examples of MMD centers are provided in Appendix B . Accordingly , the proposed loss function for intra-class compactness , i.e. , Feature To Center ( FTC ) loss , is defined as LFTC ( x , y ) = CD ( φ ( x ) , µ ( y ) ) . ( 4 ) In addition , in the inference phase , an input images x is predicted to belong to class ŷ as : ŷ = f ( x ) = argmin 1≤k≤L CD ( φ ( x ) , µ ( k ) ) = argmax 1≤k≤L cos ( φ ( x ) , µ ( k ) ) . ( 5 )

Summary: Results: To defend against adversarial attacks, this work experimentally analyzes the feature distribution of traditionally- trained CNNs for gaining more knowledge about adversarial examples. Two properties, i.e., the non-clustering property and confusing-distance property, of the feature distribution are identified by means of t-SNE visualization and clustering analysis (showing the limitations regarding representativeness) in Figure 1. The authors introduce a loss function which separates out cluster centers of CNN output features, setting them as far as possible - so that model accuracy is preserved while strengthening robustness. They test on two datasets: CIFAR10, MNIST, and show improvements in "robustness" of the model.