Perceptual Adversarial Robustness: Defense Against Unseen Threat Models

1 INTRODUCTION . Many modern machine learning algorithms are susceptible to adversarial examples : carefully crafted inputs designed to fool models into giving incorrect outputs ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ; Kurakin et al. , 2016a ; Xie et al. , 2017 ) . Much research has focused on increasing classifiers ’ robustness against adversarial attacks ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ; Zhang et al. , 2019a ) . However , existing adversarial defenses for image classifiers generally consider simple threat models . An adversarial threat model defines a set of perturbations that may be made to an image in order to produce an adversarial example . Common threat models include L2 and L∞ threat models , which constrain adversarial examples to be close to the original image in L2 or L∞ distances . Some work has proposed additional threat models which allow spatial perturbations ( Engstrom et al. , 2017 ; Wong et al. , 2019 ; Xiao et al. , 2018 ) , recoloring ( Hosseini and Poovendran , 2018 ; Laidlaw and Feizi , 2019 ; Bhattad et al. , 2019 ) , and other modifications ( Song et al. , 2018 ; Zeng et al. , 2019 ) of an image . There are multiple issues with these unrealistically constrained adversarial threat models . First , hardening against one threat model assumes that an adversary will only attempt attacks within that threat model . Although a classifier may be trained to be robust against L∞ attacks , for instance , an attacker could easily generate a spatial attack to fool the classifier . One possible solution is to train against multiple threat models simultaneously ( Jordan et al. , 2019 ; Laidlaw and Feizi , 2019 ; Maini et al. , 2019 ; Tramer and Boneh , 2019 ) . However , this generally results in a lower robustness against any one of the threat models when compared to hardening against that threat model alone . Furthermore , not all possible threat models may be known at training time , and adversarial defenses do not usually generalize well to unforeseen threat models ( Kang et al. , 2019 ) . The ideal solution to these drawbacks would be a defense that is robust against a wide , unconstrained threat model . We differentiate between two such threat models . The unrestricted adversarial threat model ( Brown et al. , 2018 ) encompasses any adversarial example that is labeled as one class by a classifier but a different class by humans . On the other hand , we define the perceptual adversarial threat model as including all perturbations of natural images that are imperceptible to a human . Most existing narrow threat models such as L2 , L∞ , etc . are near subsets of the perceptual threat model ( Figure 1 ) . Some other threat models , such as adversarial patch attacks ( Brown et al. , 2018 ) , may perceptibly alter an image without changing its true class and as such are contained in the unrestricted adversarial threat model . In this work , we focus on the perceptual threat model . The perceptual threat model can be formalized given the true perceptual distance d∗ ( x1 , x2 ) between images x1 and x2 , defined as how different two images appear to humans . For some threshold ∗ , which we call the perceptibility threshold , images x and x′ are indistinguishable from one another as long as d∗ ( x , x′ ) ≤ ∗ . Note that in general ∗ may depend on the specific input . Then , the perceptual threat model for a natural input x includes all adversarial examples x̃ which cause misclassification but are imperceptibly different from x , i.e . d∗ ( x , x̃ ) ≤ ∗ . The true perceptual distance d∗ ( · , · ) , however , can not be easily computed or optimized against . To solve this issue , we propose to use a neural perceptual distance , an approximation of the true perceptual distance between images using neural networks . Fortunately , there have been many surrogate perceptual distances proposed in the computer vision literature such as SSIM ( Wang et al. , 2004 ) . Recently , Zhang et al . ( 2018 ) discovered that comparing the internal activations of a convolutional neural network when two different images are passed through provides a measure , Learned Perceptual Image Patch Similarity ( LPIPS ) , that correlates well with human perception . We propose to use the LPIPS distance d ( · , · ) in place of the true perceptual distance d∗ ( · , · ) to formalize the neural perceptual threat model ( NPTM ) . We present adversarial attacks and defenses for the proposed NPTM . Generating adversarial examples bounded by the neural perceptual distance is difficult compared to generating Lp adversarial examples because of the complexity and non-convexness of the constraint . However , we develop two attacks for the NPTM , Perceptual Projected Gradient Descent ( PPGD ) and Lagrangian Perceptual Attack ( LPA ) ( see Section 4 for details ) . We find that LPA is by far the strongest adversarial attack at a given level of perceptibility ( see Figure 4 ) , reducing the most robust classifier studied to only 2.4 % accuracy on ImageNet-100 ( a subset of ImageNet ) while remaining imperceptible . LPA also finds adversarial examples outside of any of the other threat models studied ( see Figure 2 ) . Thus , even if a model is robust to many narrow threat models ( Lp , spatial , etc . ) , LPA can still cause serious errors . In addition to these attacks , which are suitable for evaluation of a classifier against the NPTM , we also develop Fast-LPA , a more efficient version of LPA that we use in Perceptual Adversarial Training ( PAT ) . Remarkably , using PAT to train a neural network classifier produces a single model with high robustness against a variety of imperceptible perturbations , including L∞ , L2 , spatial , recoloring , and JPEG attacks , on CIFAR-10 and ImageNet-100 ( Tables 2 and 3 ) . For example , PAT on ImageNet-100 gives 32.5 % accuracy against the union of these five attacks , whereas L∞ and L2 adversarial training give 0.5 % and 12.3 % accuracy , respectively ( Table 1 ) . PAT achieves more than double the accuracy against this union of five threat models despite not explicitly training against any of them . Thus , it generalizes well to unseen threat models . Does the LPIPS distance accurately reflect human perception when it is used to evaluate adversarial examples ? We performed a study on Amazon Mechanical Turk ( AMT ) to determine how perceptible 7 different types of adversarial perturbations such as L∞ , L2 , spatial , and recoloring attacks are at multiple threat-specific bounds . We find that LPIPS correlates well with human judgements across all the different adversarial perturbation types we examine . This indicates that the NPTM closely matches the true perceptual threat model and reinforces the utility of our perceptual attacks to measure adversarial robustness against an expansive threat model . Furthermore , this study allows calibration of a variety of attack bounds to a single perceptibility metric . We have released our dataset of adversarial examples along with the annotations made by participants for further study1 . 2 RELATED WORK . Adversarial robustness Adversarial robustness has been studied extensively for L2 or L∞ threat models ( Goodfellow et al. , 2015 ; Carlini and Wagner , 2017 ; Madry et al. , 2018 ) and non-Lp threat models such as spatial perturbations ( Engstrom et al. , 2017 ; Xiao et al. , 2018 ; Wong et al. , 2019 ) , recoloring of an image ( Hosseini and Poovendran , 2018 ; Laidlaw and Feizi , 2019 ; Bhattad et al. , 2019 ) , and perturbations in the frequency domain ( Kang et al. , 2019 ) . The most popular known adversarial defense for these threat models is adversarial training Kurakin et al . ( 2016b ) ; Madry et al . ( 2018 ) ; Zhang et al . ( 2019a ) where a neural network is trained to minimize the worst-case loss in a region around the input . Recent evaluation methodologies such as Unforeseen Attack Robustness ( UAR ) ( Kang et al. , 2019 ) and the Unrestricted Adversarial Examples challenge ( Brown et al. , 2018 ) have raised the problem of finding an adversarial defense which gives good robustness under more general threat models . Sharif et al . ( 2018 ) conduct a perceptual study showing that Lp threat models are a poor approximation of the perceptual threat model . Dunn et al . ( 2020 ) and Xu et al . ( 2020 ) have developed adversarial attacks that manipulate higher-level , semantic features . Jin and Rinard ( 2020 ) train with a manifold regularization term , which gives some robustness to unseen perturbation types . Stutz et al . ( 2020 ) also propose a method which gives robustness against unseen perturbation types , but requires rejecting ( abstaining on ) some inputs . Perceptual similarity Two basic similarity measures for images are the L2 distance and the Peak Signal-to-Noise Ratio ( PSNR ) . However , these similarity measures disagree with human vision on perturbations such as blurring and spatial transformations , which has motivated others including SSIM ( Wang et al. , 2004 ) , MS-SSIM ( Wang et al. , 2003 ) , CW-SSIM ( Sampat et al. , 2009 ) , HDR-VDP-2 ( Mantiuk et al. , 2011 ) and LPIPS ( Zhang et al. , 2018 ) . MAD competition ( Wang and Simoncelli , 2008 ) uses a constrained optimization technique related to our attacks to evaluate perceptual measures . Perceptual adversarial robustness Although LPIPS was previously proposed , it has mostly been used for development and evaluation of generative models ( Huang et al. , 2018 ; Karras et al. , 2019 ) . Jordan et al . ( 2019 ) first explored quantifying adversarial distortions with LPIPS distance . However , to the best of our knowledge , we are the first to apply a more accurate perceptual distance to the 1Code and data can be downloaded at https : //github.com/cassidylaidlaw/perceptual-advex . problem of improving adversarial robustness . As we show , adversarial defenses based on L2 or L∞ attacks are unable to generalize to a more diverse threat model . Our method , PAT , is the first adversarial training method we know of that can generalize to unforeseen threat models without rejecting inputs . 3 NEURAL PERCEPTUAL THREAT MODEL ( NPTM ) . Since the true perceptual distance between images can not be efficiently computed , we use approximations of it based on neural networks , i.e . neural perceptual distances . In this paper , we focus on the LPIPS distance ( Zhang et al. , 2018 ) while we note that other neural perceptual distances can also be used in our attacks and defenses . Let g : X → Y be a convolutional image classifier network defined on images x ∈ X . Let g have L layers , and let the internal activations ( outputs ) of the l-th layer of g ( x ) for an input x be denoted as gl ( x ) . Zhang et al . ( 2018 ) have found that normalizing and then comparing the internal activations of convolutional neural networks correlates well with human similarity judgements . Thus , the first step in calculating the LPIPS distance using the network g ( · ) is to normalize the internal activations across the channel dimension such that the L2 norm over channels at each pixel is one . Let ĝl ( x ) denote these channel-normalized activations at the l-th layer of the network . Next , the activations are normalized again by layer size and flattened into a single vector φ ( x ) , ( ĝ1 ( x ) √ w1h1 , . . . , ĝL ( x ) √ wLhL ) where wl and hl are the width and height of the activations in layer l , respectively . The function φ : X → A thus maps the inputs x ∈ X of the classifier g ( · ) to the resulting normalized , flattened internal activations φ ( x ) ∈ A , where A ⊆ Rm refers to the space of all possible resulting activations . The LPIPS distance d ( x1 , x2 ) between images x1 and x2 is then defined as : d ( x1 , x2 ) , ‖φ ( x1 ) − φ ( x2 ) ‖2 . ( 1 ) In the original LPIPS implementation , Zhang et al . ( 2018 ) learn weights to apply to the normalized activations based on a dataset of human perceptual judgements . However , they find that LPIPS is a good surrogate for human vision even without the additional learned weights ; this is the version we use since it avoids the need to collect such a dataset . Now let f : X → Y be a classifier which maps inputs x ∈ X to labels f ( x ) ∈ Y . f ( · ) could be the same as g ( · ) , or it could be a different network ; we experiment with both . For a given natural input x with the true label y , a neural perceptual adversarial example with a perceptibility bound is an input x̃ ∈ X such that x̃ must be perceptually similar to x but cause f to misclassify : f ( x̃ ) 6= y and d ( x , x̃ ) = ‖φ ( x ) − φ ( x̃ ) ‖2 ≤ . ( 2 )

Summary: This paper studies the adversarial robustness of deep neural networks against multiple and unforeseen threat models. Since there lacks a precise formalization of human perception, this paper adopts LPIPS, a metric that correlates well with human perception based on neural network activations. Then, two adversarial attack methods are proposed to generate adversarial examples under the metric. And an adversarial training method is also proposed. The experiments on various threat models validate the effectiveness of the proposed method.