Fine-grained Synthesis of Unrestricted Adversarial Examples

1 INTRODUCTION . Adversarial examples , inputs resembling real samples but maliciously crafted to mislead machine learning models , have been studied extensively in the last few years . Most of the existing papers , however , focus on normconstrained attacks and defenses , in which the adversarial input lies in an -neighborhood of a real sample using the Lp distance metric ( commonly with p = 0 , 2 , ∞ ) . For small , the adversarial input is quasi-indistinguishable from the natural sample . For an adversarial image to fool the human visual system , it is sufficient to be normconstrained ; but this condition is not necessary . Moreover , defenses tailored for norm-constrained attacks can fail on other subtle input modifications . This has led to a recent surge of interest on unrestricted adversarial attacks in which the adversary is not bounded by a norm threshold . These methods typically hand-craft transformations to capture visual similarity . Spatial transformations [ Engstrom et al . ( 2017 ) ; Xiao et al . ( 2018 ) ; Alaifari et al . ( 2018 ) ] , viewpoint or pose changes [ Alcorn et al . ( 2018 ) ] , inserting small patches [ Brown et al . ( 2017 ) ] , among other methods , have been proposed for unrestricted adversarial attacks . In this paper , we focus on fine-grained manipulation of images for unrestricted adversarial attacks . We build upon state-of-the-art generative models which disentangle factors of variation in images . We create fine and coarsegrained adversarial changes by manipulating various latent variables at different resolutions . Loss of the target network is used to guide the generation process . The pre-trained generative model constrains the search space for our adversarial examples to realistic images , thereby revealing the target model ’ s vulnerability in the natural image space . We verify that we do not deviate from the space of realistic images with a user study as well as a t-SNE plot comparing distributions of real and adversarial images ( see Fig . 7 in the appendix ) . As a result , we observe that including these examples in training the model enhances its accuracy on clean images . Our contributions can be summarized as follows : • We present the first method for fine-grained generation of high-resolution unrestricted adversarial examples in which the attacker controls which aspects of the image to manipulate , resulting in a diverse set of realistic , on-the-manifold adversarial examples . • We demonstrate that adversarial training with our examples improves performance of the model on clean images . This is in contrast to training with norm-bounded perturbations which degrades the model ’ s accuracy . Unlike recent approaches such as Xie et al . ( 2020 ) which use a separate auxiliary batch norm for adversarial examples , our method does not require any modifications to the architecture . • We propose the first method for generating unrestricted adversarial examples for semantic segmentation and object detection . Training with our examples improves segmentation results on clean images . • We demonstrate that our proposed attack can break certified defenses on norm-bounded perturbations . 2 RELATED WORK . 2.1 NORM-CONSTRAINED ADVERSARIAL EXAMPLES . Most of the existing works on adversarial attacks and defenses focus on norm-constrained adversarial examples : for a given classifier F : Rn → { 1 , . . . , K } and an image x ∈ Rn , the adversarial image x′ ∈ Rn is created such that ‖x− x′‖p < and F ( x ) 6= F ( x′ ) . Common values for p are 0 , 2 , ∞ , and is chosen small enough so that the perturbation is imperceptible . Various algorithms have been proposed for creating x′ from x. Optimization-based methods solve a surrogate optimization problem based on the classifier ’ s loss and the perturbation norm . In their pioneering paper on adversarial examples , Szegedy et al . ( 2013 ) use box-constrained L-BFGS [ Fletcher ( 2013 ) ] to minimize the surrogate loss function . Carlini & Wagner ( 2017 ) propose stronger optimization-based attacks for L0 , L2 and L∞ norms using better objective functions and the Adam optimizer . Gradient-based methods use gradient of the classifier ’ s loss with respect to the input image . Fast Gradient Sign Method ( FGSM ) [ Goodfellow et al . ( 2014 ) ] uses a first-order approximation of the function for faster generation and is optimized for the L∞ norm . Projected Gradient Descent ( PGD ) [ Madry et al . ( 2017 ) ] is an iterative variant of FGSM which provides a strong first-order attack by using multiple steps of gradient ascent and projecting perturbed images to an -ball centered at the input . Other variants of FGSM are proposed by Dong et al . ( 2018 ) and Kurakin et al . ( 2016 ) . Several methods have been proposed for defending against adversarial attacks . These approaches can be broadly categorized to empirical defenses which are empirically robust to adversarial examples , and certified defenses which are provably robust to a certain class of attacks . One of the most successful empirical defenses is adversarial training [ Goodfellow et al . ( 2014 ) ; Kurakin et al . ( 2016 ) ; Madry et al . ( 2017 ) ] which augments training data with adversarial examples generated as the training progresses . Many empirical defenses attempt to combat adversaries using a form of input pre-processing or by manipulating intermediate features or gradients [ Guo et al . ( 2017 ) ; Xie et al . ( 2017 ) ; Samangouei et al . ( 2018 ) ] . Few approaches have been able to scale up to high-resolution datasets such as ImageNet [ Liao et al . ( 2018 ) ; Xie et al . ( 2018 ) ; Kannan et al . ( 2018 ) ] . Athalye et al . ( 2018 ) show that many of these defenses fail due to obfuscated gradients , which occurs when the defense method is designed to mask information about the model ’ s gradients . Vulnerabilities of empirical defenses have led to increased interest in certified defenses , which provide a guarantee that the classifier ’ s prediction is constant within a neighborhood of the input . Several certified defenses have been proposed [ Wong & Kolter ( 2017 ) ; Raghunathan et al . ( 2018 ) ; Tsuzuku et al . ( 2018 ) ] which typically do not scale to ImageNet . Cohen et al . ( 2019 ) use randomized smoothing with Gaussian noise to obtain provably L2-robust classifiers on ImageNet . Lecuyer et al . ( 2019 ) propose an alternative certified defense at ImageNet scale leveraging a connection between robustness against adversarial examples and differential privacy theory . 2.2 UNRESTRICTED ADVERSARIAL EXAMPLES . For an image to be adversarial , it needs to be visually indistinguishable from real images . One way to achieve this is by applying subtle geometric transformations to the input image . Spatially transformed adversarial examples are introduced by Xiao et al . ( 2018 ) in which a flow field is learned to displace pixels of the image . Similarly , Alaifari et al . ( 2018 ) iteratively apply small deformations to the input in order to obtain the adversarial image . Engstrom et al . ( 2017 ) show that simple translations and rotations are enough for fooling deep neural networks . Alcorn et al . ( 2018 ) manipulate pose of an object to fool deep neural networks . They estimate parameters of a 3D renderer that cause the target model to misbehave in response to the rendered image . Another approach for evading the norm constraint is to insert new objects in the image . Adversarial Patch [ Brown et al . ( 2017 ) ] creates an adversarial image by completely replacing part of an image with a synthetic patch , which is image-agnostic and robust to transformations . Existence of on-the-manifold adversarial examples is also shown by Gilmer et al . ( 2018 ) , that consider the task of classifying between two concentric n-dimensional spheres . Stutz et al . ( 2019 ) demonstrate that both robust and accurate models are possible by using on-the-manifold adversarial examples . A challenge for creating unrestricted adversarial examples and defending against them is introduced by Brown et al . ( 2018 ) using the simple task of classifying between birds and bicycles . The recent work by Gowal et al . ( 2020 ) show that adversarial training with examples generated by StyleGAN can improve performance of the model on clean images . They consider the classification task on low-resolution datasets such as ColorMNIST and CelebA , and only use fine changes in their adversarial training . Our approach is effective on high-resolution datasets such as CelebA-HQ and LSUN , uses a range of low-level to high-level changes for adversarial training and encompasses several tasks including classification , segmentation and detection . In addition , we demonstrate that our adversarial examples can break certified defenses on norm-constrained perturbations and are realistic as verified by human evaluation . Song et al . ( 2018 ) search in the latent ( z ) space of AC-GAN [ Odena et al . ( 2017 ) ] to find generated images that can fool a target classifier but yield correct predictions on AC-GAN ’ s auxiliary classifier . They constrain the search region of z so that it is close to a randomly sampled noise vector , and show results on MNIST , SVHN and CelebA datasets . Requiring two classifiers to have inconsistent predictions degrades sample quality of the model . As we show in the appendix , training with these adversarial examples hurts the model ’ s performance on clean images . Moreover , this approach has no control over the generation process since small changes in the z space can lead to large changes in generated images and even create unrealistic samples . On the other hand , our method manipulates high-resolution real or synthesized images in a fine-grained manner owing to the interpretable disentangled latent space . It also generates samples which improve the model ’ s accuracy on clean images both in classification and segmentation tasks . To further illustrate difference of our approach with Song et al . ( 2018 ) , we plot t-SNE embeddings of real images from CelebA-HQ as well as adversarial examples from our method and Song et al. ’ s approach in the appendix and show that our adversarial images stay closer to the manifold of real images . 3 APPROACH . Most of the existing works on unrestricted adversarial attacks rely on geometric transformations and deformations which are oblivious to latent factors of variation . In this paper , we leverage disentangled latent representations of images for unrestricted adversarial attacks . We build upon state-of-the-art generative models and consider various target tasks : classification , semantic segmentation and object detection . 3.1 CLASSIFICATION . Style-GAN [ Karras et al . ( 2018 ) ] is a state-of-the-art generative model which disentangles high-level attributes and stochastic variations in an unsupervised manner . Stylistic variations are represented by style variables and stochastic details are captured by noise variables . Changing the noise only affects low-level details , leaving the overall composition and high-level aspects intact . This allows us to manipulate the noise variables such that variations are barely noticeable by the human eye . The style variables affect higher level aspects of image generation . For instance , when the model is trained on bedrooms , style variables from the top layers control viewpoint of the camera , middle layers select the particular furniture , and bottom layers deal with colors and details of materials . This allows us to manipulate images in a controlled manner , providing an avenue for fine-grained unrestricted attacks . Formally , we can represent Style-GAN with a mapping function f and a synthesis network g. The mapping function is an 8-layer MLP which takes a latent code z , and produces an intermediate latent vector w = f ( z ) . This vector is then specialized by learned affine transformations A to style variables y , which control adaptive instance normalization operations after each convolutional layer of the synthesis network g. Noise inputs are single-channel images consisting of un-correlated Gaussian noise that are fed to each layer of the synthesis network . Learned perfeature scaling factors B are used to generate noise variables η which are added to the output of convolutional

Summary: This paper proposes a mechanism to generate adversarial examples by applying latent variables level manipulation, based on the styleGAN framework. Unlike previous works mostly focused on image level perturbations and geometry transformations, this work tends to control higher level latent sampling such as style, so as to generate a style-adversarial examples. Although a similar idea has been proposed by Song et al. (2018), this work is along the same direction and achieves better performance. The loss is proposed for general classification tasks such as object classification, object detection and semantic segmentation. The experimental results show not only qualitatively confusing human vision but also quantitatively improve the performance on testing clean images.