Non-robust Features through the Lens of Universal Perturbations

1 INTRODUCTION . Modern deep neural networks perform extremely well across many prediction tasks , but they largely remain vulnerable to adversarial examples ( Szegedy et al. , 2014 ) . Models ’ brittleness to these small , imperceptible perturbations highlights one alarming way in which models deviate from humans . Recent work gives evidence that this deviation is due to the presence of useful non-robust features in our datasets ( Ilyas et al. , 2019 ) . These are brittle features that are sensitive to small perturbations—too small to be noticeable to humans , yet capture enough predictive signal to generalize well on the underlying classification task . When models rely on non-robust features , they become vulnerable to adversarial examples , as even small perturbations can flip the features ’ signal . While prior work gives evidence of non-robust features in natural datasets , we lack a more finegrained understanding of their properties . In general , we do not understand well how models make decisions , so it is unclear how much we can understand about these features that are believed to be imperceptible . A number of works suggest that these features may exploit certain properties of the dataset that are misaligned with human perception ( Ilyas et al. , 2019 ) , such as high-frequency information ( Yin et al. , 2019 ) , but much remains unknown . In this work , we illustrate how we can isolate more human-aligned non-robust features by imposing additional constraints on adversarial perturbations . In particular , we revisit universal adversarial perturbations ( Moosavi-Dezfooli et al. , 2017a ) , i.e . adversarial perturbations that generalize across many inputs . Prior works have observed that these perturbations appear to be semantic ( Hayes & Danezis , 2019 ; Khrulkov & Oseledets , 2018 ; Liu et al. , 2019 ) . We demonstrate that universal perturbations possess additional human-aligned properties different from standard adversarial perturbations , and analyze the non-robust features leveraged by these perturbations . Concretely , our findings are : Universal perturbations have more human-aligned properties . We show that universality adversarial perturbations have additional human-aligned properties that distinguish them from standard adversarial perturbations ( e.g. , Figure 1 ) . In particular , ( 1 ) the most semantically identifiable local patches inside universal perturbations also contain the most signal ; and ( 2 ) universal perturbations are approximately spatially invariant , in that they are still effective after translations . Non-robust features can be semantically meaningful . We show that universal perturbations are primarily relying on non-robust features rather than robust ones . Specifically , we compare the sensitivity of natural and ( adversarially ) robust models to rescalings of these perturbations to demonstrate that universal perturbations likely rely on non-robust features . Together with our first finding , this shows that some non-robust features can be human-aligned . Universal perturbations contain less non-robust signal . We find that the non-robust features leveraged by universal perturbations have less predictive signal than those leveraged by standard adversarial perturbations , despite being more human-aligned . We measure both ( 1 ) generalizability to the original test set and ( 2 ) transferability of perturbations across independent models , following the methodology of Ilyas et al . ( 2019 ) . Under these metrics , universal perturbations consistently obtain non-trivial but substantially worse performance than standard adversarial perturbations . 2 PRELIMINARIES . We consider a standard classification task : given input-label samples ( x , y ) ∈ X × Y from a data distribution D , the goal is to to learn a classifier C : X → Y that generalizes to new data . Non-robust vs. robust features . Following Ilyas et al . ( 2019 ) , we introduce the following terminology . A useful feature for classification is a function that is ( positively ) correlated with the correct label in expectation . A feature is robustly useful if , even under adversarial perturbations ( within a specified set of valid perturbations ∆ ) the feature is still useful . Finally , a useful , non-robust feature is a feature that is useful but not robustly useful . These features are useful for classification in the standard setting , but can hurt accuracy in the adversarial setting ( since their correlation with the label can be reversed ) . For conciseness , throughout this paper we will refer to such features simply as non-robust features . Universal perturbations . A universal adversarial perturbation ( or just universal perturbation for short ) , as introduced in Moosavi-Dezfooli et al . ( 2017a ) , is a perturbation δ that causes the classifier C to predict the wrong label on a large fraction of inputs from D. We focus on targeted universal perturbations that fool C into predicting a specific ( usually incorrect ) target label t. Thus , a ( targeted ) universal perturbation δ ∈ ∆ satisfies P ( x , y ) ∼D [ C ( x+δ ) = t ] = ρ , where ρ is the attack success rate ( ASR ) of the universal perturbation . The only technical difference between universal perturbations and ( standard ) adversarial perturbations is the use of a single perturbation vector δ that is applied to all inputs . Thus , one can think of universality as a constraint for the perturbation δ to be inputindependent . Universal perturbations can leverage non-robust features in data , as we show here ; we refer to these simply as universal non-robust features . ` p perturbations . We study the case where ∆ is the set of ` p-bounded perturbations , i.e . ∆ = { δ ∈ Rd | ||δ||p ≤ } for p = 2 , ∞ . This is the most widely studied setting for research on adversarial examples and has proven to be an effective benchmark ( Carlini et al. , 2019 ) . Additionally , ` p-robustness appears to be aligned to a certain degree with the human visual system ( Tsipras et al. , 2019 ) . 2.1 COMPUTING UNIVERSAL PERTURBATIONS . We compute universal perturbations by using projected gradient descent ( PGD ) on the following optimization problem : min δ∈∆ E ( x , y ) ∼D [ L ( f ( x+ δ ) , t ) ] ( 1 ) where L is the standard cross-entropy loss function for classification , and f are the logits prior to classification . and t is the target label . While many different algorithms have been developed for computing universal perturbations with varying attack success rates , we do not consider them as achieving the highest rate is not necessary for our investigations . As observed in Moosavi-Dezfooli et al . ( 2017a ) , universal perturbations trained on only a fraction of the dataset still generalize to the rest of the dataset , so in practice it suffices to approximate the expectation in equation 1 using a relatively small batch of data drawn from D. We call the batch we optimize over the base set , and use K to refer to its size . We describe the selection of all hyperparameters in Appendix B . 2.2 EXPERIMENTAL SETTINGS . Datasets . We conduct our experiments on a subset of ImageNet ( Deng et al. , 2009 ) and CIFAR-10 ( Krizhevsky et al. , 2009 ) . The Mixed10 ImageNet dataset ( Engstrom et al. , 2019 ) , which we refer to as ImageNet-M10 , is formed by sub-selecting and grouping together semantically similar classes from ImageNet ; it provides a more computationally efficient alternative to the full dataset , while retaining some of ImageNet ’ s complexity ( see Appendix B.1 for more details about the dataset ) . In the main text , we focus our results on ImageNet-M10 , and in the Appendices we provide additional results on CIFAR-10 as well as visualizations from a full ImageNet model . Models . We use the standard ResNet-18 architecture ( He et al. , 2016 ) unless mentioned otherwise . 3 PROPERTIES OF UNIVERSAL PERTURBATIONS . We find that universal perturbations , in contrast to adversarial perturbations , are much more aligned with human perception . Beyond visualizations , we demonstrate this in the following ways : ( 1 ) local patches that are most semantically identifiable with the target class also contain the most predictive signal ; and ( 2 ) universal perturbations are approximately spatially invariant , whereas standard perturbations are not . 3.1 PERCEPTUAL ALIGNMENT . Visual comparison with standard adversarial perturbations . Standard ` p adversarial perturbations are often thought to be incomprehensible to humans ( Szegedy et al. , 2014 ; Ilyas et al. , 2019 ) . Even when magnified for visualization , these perturbations are not identifiable with their target class . In contrast , universal perturbations are visually more interpretable : when amplified , they contain local regions that we can identify with the target class ( see Figure 2 for example universal perturbations , and Figure 1 for a comparison to standard perturbations ) . Prior works have also observed that universal perturbations resemble their target class ( Hayes & Danezis , 2019 ) and contain meaningful texture patterns ( Khrulkov & Oseledets , 2018 ; Liu et al. , 2019 ) . Next , we go beyond simple visualizations by identifying two additional human-aligned properties of universal perturbations . Predictive signals are localized to most semantically identifiable patches . While universal perturbations contain parts that are semantically identifiable with the target class , they also contain less identifiable regions ; a priori , it is unclear which parts influencing the model . To investigate this , we randomly select different local patches of the perturbation , evaluate their ASR in isolation , and inspect them visually . For ` 2 perturbations , the patches vary widely in norm , so a possible concern is that the model is only reacting to the patches with the highest norm ; to correct for this , we linearly scale up all patches to have the same norm as the largest norm patch . Even after normalization , the patches with the highest ASR are more semantically identifiable with the target class ( Figure 3 ) . For ` ∞ perturbations , where all patches have similar norms ( so no further normalization is done ) , we also observe that their ASR is proportional to their visual saliency . This demonstrates that the model is indeed reacting primarily to the most salient parts of the perturbation . do g ( 2 ) ASR : 0.494 0.237 0.220 0.205 0.098 0.098 do g ( ) ASR : 0.597 0.167 0.163 0.159 0.101 0.101 Figure 3 : Local analysis of universal perturbations on ImageNet-M10 : 64 random 80×80 patches are isolated and evaluated for their ASR . First column shows the original perturbation for the target class ( top : ` 2 ; bottom : ` ∞ ) ; next five columns show three patches with the highest ASR , and two with the lowest . Top of each perturbation indicates its ASR on the test set . Patches are normalized . 3.2 SPATIAL INVARIANCE . Adversarial perturbations are not only unintelligible but also extremely brittle to translations , as we demonstrate . In contrast , we find that universal perturbations are translationally invariant to a large degree . As spatial invariance is one of the key properties of the human visual system ( Hubel & Wiesel , 1968 ) , this illustrates another way that universal perturbations are more human-aligned than standard perturbations . We quantify spatial invariance by measuring the ASR of translated perturbations . Specifically , we compute targeted standard adversarial perturbations on a sample of 256 images from the ImageNetM10 test set . Then , we evaluate the ASR of different translated copies of these perturbations over the test set ( copies of each perturbation are evaluated only on its corresponding image ) . For comparison , we take a precomputed set of universal perturbations of the same norm , and evaluate them across all images . Figure 16 shows that universal perturbations still achieve non-trivial ASR after translations of varying magnitudes . In contrast , standard adversarial perturbations achieve a chance-level 10 % ASR when shifted by more than eight pixels ( two grid cells in Figure 16 ) . This illustrates that the model reacts differently to universal perturbations than to standard adversarial perturbations .

Summary: This paper studies the link between non-robust features and universal adversarial perturbations. This paper shows that universal perturbation leverage non-robust features in data in a different way than standard adversarial attacks. Experiments are based on a universal version of projected gradient descent (PGD). The findings are that universal perturbations are more aligned with visual semantics and human perception that general adversarial attacks. Moreover, it is shown to be difficult to obtain generalisation or transferability between models based on universal signals, as opposed to standard adversarial samples. Generalization seems to decrease with the size of the set used for generating a given universal perturbation, while semantics of the features improve.