Perceptual Adversarial Robustness: Defense Against Unseen Threat Models

1 INTRODUCTION . Many modern machine learning algorithms are susceptible to adversarial examples : carefully crafted inputs designed to fool models into giving incorrect outputs ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ; Kurakin et al. , 2016a ; Xie et al. , 2017 ) . Much research has focused on increasing classifiers ’ robustness against adversarial attacks ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ; Zhang et al. , 2019a ) . However , existing adversarial defenses for image classifiers generally consider simple threat models . An adversarial threat model defines a set of perturbations that may be made to an image in order to produce an adversarial example . Common threat models include L2 and L∞ threat models , which constrain adversarial examples to be close to the original image in L2 or L∞ distances . Some work has proposed additional threat models which allow spatial perturbations ( Engstrom et al. , 2017 ; Wong et al. , 2019 ; Xiao et al. , 2018 ) , recoloring ( Hosseini and Poovendran , 2018 ; Laidlaw and Feizi , 2019 ; Bhattad et al. , 2019 ) , and other modifications ( Song et al. , 2018 ; Zeng et al. , 2019 ) of an image . There are multiple issues with these unrealistically constrained adversarial threat models . First , hardening against one threat model assumes that an adversary will only attempt attacks within that threat model . Although a classifier may be trained to be robust against L∞ attacks , for instance , an attacker could easily generate a spatial attack to fool the classifier . One possible solution is to train against multiple threat models simultaneously ( Jordan et al. , 2019 ; Laidlaw and Feizi , 2019 ; Maini et al. , 2019 ; Tramer and Boneh , 2019 ) . However , this generally results in a lower robustness against any one of the threat models when compared to hardening against that threat model alone . Furthermore , not all possible threat models may be known at training time , and adversarial defenses do not usually generalize well to unforeseen threat models ( Kang et al. , 2019 ) . The ideal solution to these drawbacks would be a defense that is robust against a wide , unconstrained threat model . We differentiate between two such threat models . The unrestricted adversarial threat model ( Brown et al. , 2018 ) encompasses any adversarial example that is labeled as one class by a classifier but a different class by humans . On the other hand , we define the perceptual adversarial threat model as including all perturbations of natural images that are imperceptible to a human . Most existing narrow threat models such as L2 , L∞ , etc . are near subsets of the perceptual threat model ( Figure 1 ) . Some other threat models , such as adversarial patch attacks ( Brown et al. , 2018 ) , may perceptibly alter an image without changing its true class and as such are contained in the unrestricted adversarial threat model . In this work , we focus on the perceptual threat model . The perceptual threat model can be formalized given the true perceptual distance d∗ ( x1 , x2 ) between images x1 and x2 , defined as how different two images appear to humans . For some threshold ∗ , which we call the perceptibility threshold , images x and x′ are indistinguishable from one another as long as d∗ ( x , x′ ) ≤ ∗ . Note that in general ∗ may depend on the specific input . Then , the perceptual threat model for a natural input x includes all adversarial examples x̃ which cause misclassification but are imperceptibly different from x , i.e . d∗ ( x , x̃ ) ≤ ∗ . The true perceptual distance d∗ ( · , · ) , however , can not be easily computed or optimized against . To solve this issue , we propose to use a neural perceptual distance , an approximation of the true perceptual distance between images using neural networks . Fortunately , there have been many surrogate perceptual distances proposed in the computer vision literature such as SSIM ( Wang et al. , 2004 ) . Recently , Zhang et al . ( 2018 ) discovered that comparing the internal activations of a convolutional neural network when two different images are passed through provides a measure , Learned Perceptual Image Patch Similarity ( LPIPS ) , that correlates well with human perception . We propose to use the LPIPS distance d ( · , · ) in place of the true perceptual distance d∗ ( · , · ) to formalize the neural perceptual threat model ( NPTM ) . We present adversarial attacks and defenses for the proposed NPTM . Generating adversarial examples bounded by the neural perceptual distance is difficult compared to generating Lp adversarial examples because of the complexity and non-convexness of the constraint . However , we develop two attacks for the NPTM , Perceptual Projected Gradient Descent ( PPGD ) and Lagrangian Perceptual Attack ( LPA ) ( see Section 4 for details ) . We find that LPA is by far the strongest adversarial attack at a given level of perceptibility ( see Figure 4 ) , reducing the most robust classifier studied to only 2.4 % accuracy on ImageNet-100 ( a subset of ImageNet ) while remaining imperceptible . LPA also finds adversarial examples outside of any of the other threat models studied ( see Figure 2 ) . Thus , even if a model is robust to many narrow threat models ( Lp , spatial , etc . ) , LPA can still cause serious errors . In addition to these attacks , which are suitable for evaluation of a classifier against the NPTM , we also develop Fast-LPA , a more efficient version of LPA that we use in Perceptual Adversarial Training ( PAT ) . Remarkably , using PAT to train a neural network classifier produces a single model with high robustness against a variety of imperceptible perturbations , including L∞ , L2 , spatial , recoloring , and JPEG attacks , on CIFAR-10 and ImageNet-100 ( Tables 2 and 3 ) . For example , PAT on ImageNet-100 gives 32.5 % accuracy against the union of these five attacks , whereas L∞ and L2 adversarial training give 0.5 % and 12.3 % accuracy , respectively ( Table 1 ) . PAT achieves more than double the accuracy against this union of five threat models despite not explicitly training against any of them . Thus , it generalizes well to unseen threat models . Does the LPIPS distance accurately reflect human perception when it is used to evaluate adversarial examples ? We performed a study on Amazon Mechanical Turk ( AMT ) to determine how perceptible 7 different types of adversarial perturbations such as L∞ , L2 , spatial , and recoloring attacks are at multiple threat-specific bounds . We find that LPIPS correlates well with human judgements across all the different adversarial perturbation types we examine . This indicates that the NPTM closely matches the true perceptual threat model and reinforces the utility of our perceptual attacks to measure adversarial robustness against an expansive threat model . Furthermore , this study allows calibration of a variety of attack bounds to a single perceptibility metric . We have released our dataset of adversarial examples along with the annotations made by participants for further study1 . 2 RELATED WORK . Adversarial robustness Adversarial robustness has been studied extensively for L2 or L∞ threat models ( Goodfellow et al. , 2015 ; Carlini and Wagner , 2017 ; Madry et al. , 2018 ) and non-Lp threat models such as spatial perturbations ( Engstrom et al. , 2017 ; Xiao et al. , 2018 ; Wong et al. , 2019 ) , recoloring of an image ( Hosseini and Poovendran , 2018 ; Laidlaw and Feizi , 2019 ; Bhattad et al. , 2019 ) , and perturbations in the frequency domain ( Kang et al. , 2019 ) . The most popular known adversarial defense for these threat models is adversarial training Kurakin et al . ( 2016b ) ; Madry et al . ( 2018 ) ; Zhang et al . ( 2019a ) where a neural network is trained to minimize the worst-case loss in a region around the input . Recent evaluation methodologies such as Unforeseen Attack Robustness ( UAR ) ( Kang et al. , 2019 ) and the Unrestricted Adversarial Examples challenge ( Brown et al. , 2018 ) have raised the problem of finding an adversarial defense which gives good robustness under more general threat models . Sharif et al . ( 2018 ) conduct a perceptual study showing that Lp threat models are a poor approximation of the perceptual threat model . Dunn et al . ( 2020 ) and Xu et al . ( 2020 ) have developed adversarial attacks that manipulate higher-level , semantic features . Jin and Rinard ( 2020 ) train with a manifold regularization term , which gives some robustness to unseen perturbation types . Stutz et al . ( 2020 ) also propose a method which gives robustness against unseen perturbation types , but requires rejecting ( abstaining on ) some inputs . Perceptual similarity Two basic similarity measures for images are the L2 distance and the Peak Signal-to-Noise Ratio ( PSNR ) . However , these similarity measures disagree with human vision on perturbations such as blurring and spatial transformations , which has motivated others including SSIM ( Wang et al. , 2004 ) , MS-SSIM ( Wang et al. , 2003 ) , CW-SSIM ( Sampat et al. , 2009 ) , HDR-VDP-2 ( Mantiuk et al. , 2011 ) and LPIPS ( Zhang et al. , 2018 ) . MAD competition ( Wang and Simoncelli , 2008 ) uses a constrained optimization technique related to our attacks to evaluate perceptual measures . Perceptual adversarial robustness Although LPIPS was previously proposed , it has mostly been used for development and evaluation of generative models ( Huang et al. , 2018 ; Karras et al. , 2019 ) . Jordan et al . ( 2019 ) first explored quantifying adversarial distortions with LPIPS distance . However , to the best of our knowledge , we are the first to apply a more accurate perceptual distance to the 1Code and data can be downloaded at https : //github.com/cassidylaidlaw/perceptual-advex . problem of improving adversarial robustness . As we show , adversarial defenses based on L2 or L∞ attacks are unable to generalize to a more diverse threat model . Our method , PAT , is the first adversarial training method we know of that can generalize to unforeseen threat models without rejecting inputs . 3 NEURAL PERCEPTUAL THREAT MODEL ( NPTM ) . Since the true perceptual distance between images can not be efficiently computed , we use approximations of it based on neural networks , i.e . neural perceptual distances . In this paper , we focus on the LPIPS distance ( Zhang et al. , 2018 ) while we note that other neural perceptual distances can also be used in our attacks and defenses . Let g : X → Y be a convolutional image classifier network defined on images x ∈ X . Let g have L layers , and let the internal activations ( outputs ) of the l-th layer of g ( x ) for an input x be denoted as gl ( x ) . Zhang et al . ( 2018 ) have found that normalizing and then comparing the internal activations of convolutional neural networks correlates well with human similarity judgements . Thus , the first step in calculating the LPIPS distance using the network g ( · ) is to normalize the internal activations across the channel dimension such that the L2 norm over channels at each pixel is one . Let ĝl ( x ) denote these channel-normalized activations at the l-th layer of the network . Next , the activations are normalized again by layer size and flattened into a single vector φ ( x ) , ( ĝ1 ( x ) √ w1h1 , . . . , ĝL ( x ) √ wLhL ) where wl and hl are the width and height of the activations in layer l , respectively . The function φ : X → A thus maps the inputs x ∈ X of the classifier g ( · ) to the resulting normalized , flattened internal activations φ ( x ) ∈ A , where A ⊆ Rm refers to the space of all possible resulting activations . The LPIPS distance d ( x1 , x2 ) between images x1 and x2 is then defined as : d ( x1 , x2 ) , ‖φ ( x1 ) − φ ( x2 ) ‖2 . ( 1 ) In the original LPIPS implementation , Zhang et al . ( 2018 ) learn weights to apply to the normalized activations based on a dataset of human perceptual judgements . However , they find that LPIPS is a good surrogate for human vision even without the additional learned weights ; this is the version we use since it avoids the need to collect such a dataset . Now let f : X → Y be a classifier which maps inputs x ∈ X to labels f ( x ) ∈ Y . f ( · ) could be the same as g ( · ) , or it could be a different network ; we experiment with both . For a given natural input x with the true label y , a neural perceptual adversarial example with a perceptibility bound is an input x̃ ∈ X such that x̃ must be perceptually similar to x but cause f to misclassify : f ( x̃ ) 6= y and d ( x , x̃ ) = ‖φ ( x ) − φ ( x̃ ) ‖2 ≤ . ( 2 )

Summary: This work proposes a new form of adversarial training, supported by two proposed adversarial attacks based off a perceptual distance. The choice of perceptual distance (LPIPS), is computed by comparing the activations of (possibly different) two neural networks with respect to a pair of inputs. The authors propose two new attacks based off this perceptual distance: PPGD and LPA, as it is distinct from the common choice of L_2 or L_inf. This work claims that performing adversarial training against adversarial examples crafted by the proposed attacks, induces robustness to a wide range of "narrow" threat models e.g. L_2, JPEG, L_inf.