PPD: Permutation Phase Defense Against Adversarial Examples in Deep Learning

1 INTRODUCTION . Recent advancements in deep learning have brought Deep Neural Networks ( DNNs ) to securitysensitive applications such as self-driving cars , malware detection , face recognition , etc . Although DNNs provide tremendous performance in these applications , they are vulnerable to adversarial attacks ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Athalye & Sutskever , 2017 ) . Especially in computer vision applications , small adversarial perturbation of the input can mislead the state-ofthe-art DNNs while being imperceptible to human eye ( Goodfellow et al. , 2014 ; Nguyen et al. , 2015 ; Moosavi-Dezfooli et al. , 2017 ; Akhtar & Mian , 2018 ) . Since the existence of adversarial examples was discovered ( Szegedy et al. , 2013 ) , numerous attacks and defenses have been proposed . Attacks typically try to push the image towards the decision boundaries of the classifier ( Goodfellow et al. , 2014 ; Carlini & Wagner , 2016 ; Kurakin et al. , 2016 ; Dong et al. , 2018 ) . Defenses , on the other hand , can be categorized into three groups : ( a ) adversarial training , ( b ) hiding the classifier , and ( c ) hiding the input image . Adversarial Training . As a natural defense , Szegedy et al . ( 2013 ) ; Goodfellow et al . ( 2014 ) suggested to augment the training set with adversarial examples to reshape the decision boundaries of the classifier . Adversarial training requires an attack algorithm to generate adversarial examples during training . The resulting classifier is shown to be robust against the same attack used during training ( Goodfellow et al. , 2014 ; Madry et al. , 2017 ; Tramèr et al. , 2017 ) . However , it is still vulnerable to other attacks or even the same attack with more perturbation budget ( Sharma & Chen , 2018 ; Madry et al. , 2017 ) . Hiding the Classifier . The idea of hiding the classifier has been investigated in various ways . One naive way is to not reveal the weights of the neural network . However , adversarial examples generated by a substitute model can still fool the hidden model ( Papernot et al. , 2017 ; Liu et al. , 2016 ) , a phenomenon known as transferability of adversarial examples across different structures . Another way is to conceal the gradient of loss function to guard against attacks that take advantage of gradient ( Buckman et al. , 2018 ; Song et al. , 2017 ; Samangouei et al. , 2018 ; Guo et al. , 2017 ) . However , all of these defenses are circumvented by approximately recovering the gradient ( Athalye et al. , 2018 ) . Hiding the Input Image . Another general approach to diminish adversarial examples is to hide the input image . Following this idea , Guo et al . ( 2017 ) suggested using randomization to hide the input image by two methods : ( a ) image quilting : reconstruct image by replacing small patches with clean patches from a database and ( b ) randomly drop pixels and recover them by total variance minimization . One fundamental restriction of these randomized defenses is that a large portion of pixels has to be preserved for correct classification . This leaves the door open for adversarial attacks . Indeed , both of these defenses are completely broken ( Athalye et al. , 2018 ) . Given that the defenses proposed under the last two categories are bypassed , to the best of our knowledge , adversarial training with examples generated by Projected Gradient Descent ( PGD ) attack ( Madry et al. , 2017 ) remains current state-of-the-art . However , its robustness is limited to specific attack types with certain strength . Changing the attack type or exceeding the perturbation limit degrades the accuracy significantly . In this paper , Permutation Phase Defense ( PPD ) is proposed to mitigate adversarial examples . Following the philosophy of hiding the input image , we suggest to “ encrypt ” the image by applying two transformations before feeding the image to the neural network : First , randomly shuffle the pixels of the image with a fixed random seed hidden from the adversary . Second , convert the permuted image to the phase of its two dimensional Fourier transform ( 2d-DFT ) ( see Figure 1 ) . This pipeline is used in both training and inference phases . The Fourier transformation captures the freqnency at which adjacent pixels of the input change along both axes . This frequency depends on the relational positions of the pixels rather than their absolute value . On the other hand , the relational positions are already hidden from the adversary by using the permutation block . The random seed plays the role of the key in cryptography . PPD offers some advantages as follows : • PPD outperforms current state-of-the-art defense in two ways : First , PPD is a general defense and is not restricted to specific attack types . Second , accuracy of PPD does not drop drastically by increasing perturbation budget of adversary ( Table 1 ) . • In contrast to image quilting and random pixel drop ( Guo et al . ( 2017 ) ) , PPD randomly shuffles all the pixels in the pixel domain . Therefore , important pixels of the phase domain remain hidden because of their dependence on the relational positions of the pixels in the pixel domain . • Increasing adversarial perturbation budget is expected to decrease accuracy . PPD , however , can distribute perturbation over different pixels in the phase domain such that adversarial perturbation is not more effective than random noise ( Figures 4 and 5 ) . 2 PRELIMINARIES . 2.1 NOTATION . Let ( X , d ) be a metric space where X = [ 0 , 1 ] H×W×C is the image space in the pixel domain for images with height H , width W and C channels and d : X × X → [ 0 , ∞ ) is a metric . A classifier h ( · ) is a function from X to label probabilities such that hi ( x ) is the probability that image x corresponds to label i . Let c∗ ( x ) be the true label of image x and c ( x ) = argmaxi hi ( x ) be the label predicted by the classifier . 2.2 ADVERSARIAL EXAMPLES . Intuitively speaking , adversarial examples are distorted images that visually resemble the clean images but can fool the classifier . Thus , given a classifier h ( · ) , > 0 and image x ∈ X , adversarial image x′ ∈ X satisfies two properties : d ( x , x′ ) ≤ and c ( x′ ) 6= c∗ ( x ) . Ideally , metric d should represent visual distance . However , there is no clear mathematical metric for visual distance . Therefore , to compare against a common ground , researchers have considered typical mathematical metrics induced by ` 2 and ` ∞ norms . 3 PERMUTATION PHASE DEFENSE ( PPD ) . 3.1 DEFENSE STRUCTURE . We address the problem of defending against adversarial examples . We hide the input space by two transformations preceding the main neural network ( see Figure 1 ) : • Random permutation block : pixels of the input image are permuted according to a fixed random permutation . The permutation seed is fixed and hidden from the adversary . • Pixel to phase block : permuted image is converted to the phase of its 2-dimensional discrete Fourier transform ( 2d-DFT ) . These two blocks precede the neural network in both training and testing phases . Mathematically speaking , consider channel c of image x ∈ X . Its 2d-DFT can be written as Flk = 1√ HW H−1∑ h=0 W−1∑ w=0 xhwe −j2π ( lH h+ k W w ) ( 1 ) where l = 0 , · · · , H − 1 and k = 0 , · · · , W − 1 and xhw is ( h , w ) image pixel value . Flk is a complex number that can be represented in the polar coordinates as Flk = Alk exp ( jϕlk ) where Alk ∈ [ 0 , ∞ ) is the magnitude and ϕlk ∈ [ −π , π ) is the phase of Flk . Let P = [ −π , π ) H×W×C be the phase space and p : X → P be a function that maps each channel of the image in the pixel domain to its phase . Let σ : X → X be a channel-wise permutation , i.e. , pixels in each channel of the input image are permuted according to a fixed permutation . Note that same permutation is used for all channels and all images . The PPD classifier can then be written as h ( x ) = f ( p ( σ ( x ) ) ( 2 ) where f ( · ) is the neural network . 3.2 INTUITION BEHIND THE DEFENSE . It is known that random noise is unable to fool a trained neural network ( Fawzi et al. , 2016 ) . Motivated by this fact , if we train the neural network in a domain that is hidden from adversary , adversarial perturbations will not affect more than random noise in the actual domain fed to the neural network and thus not fool the classifier . Following this idea , Fourier transformation is used to capture the frequency at which neighboring pixels change . Indeed , the values of the image in the Fourier domain do not purely depend on the values of pixels in the pixel domain . Instead , they depend on what pixels are adjacent in the pixel domain , while neighboring pixels are hidden from adversary by the random permutation block . Therefore , adversary will have little hope to attack the classifier due to almost no knowledge about the input space . 3.2.1 WHY RANDOM PERMUTATION IS NOT ENOUGH ? . We would like to emphasize that input transformations that partially hide the input space are not secure . For example , total variance minimization ( randomly drop pixels and replace them with total variance minimization ) , image quilting ( replacing patches of the input image by patches from clean images ) ( Guo et al. , 2017 ) are all defeated ( Athalye et al. , 2018 ) . One possible explanation for this failure is that although the adversary does not have full knowledge of the actual image fed to the neural network ( by using randomness in dropping pixels and patching ) , good portion of the image is still preserved . Hence , it is possible for adversary to attack . Similarly , we have observed that random permutation by itself ( without the pixel2phase block ) is not sufficient to defeat adversary . Adversarial images generated by attacking a vanilla neural network can substantially decrease the accuracy of a permuted network . This is because important pixels ( e.g . white pixels in MNIST ) in the original domain will still remain important pixels in the permuted domain . Therefore , regardless of the employed permutation , if adversary attacks important pixels of the original domain ( e.g . by converting white pixels to gray pixels in MNIST ) , it has successfully attacked important pixels in the permuted domain . 3.2.2 WHY PHASE OF 2D-DFT IS USED ? . It is well known that phase component of Fourier transform is crucial in visual perception of human ( Oppenheim & Lim , 1980 ; Morrone & Burr , 1988 ) . Indeed , an image reconstructed from phase only preserves essential information about the edges ( see Figure 2 ) . This suggests that phase carries key information about the image and thus it is possible to train neural networks to operate in the phase domain rather than the pixel domain . One might ask what if both phase and magnitude are fed to the neural network by doubling the size of the input space ? Our effort to train such a model was not fruitful but we leave it as a future research direction .

Summary: This paper proposes Permutation Phase Defense (PPD), a novel image hiding method to resist adversarial attacks. PPD relies on safekeeping of the key, specifically the seed used for permuting the image pixels. The paper demonstrated the method on MNIST and CIFAR10, and evaluates it against a number of adversarial attacks. The method appears to be robust across attacks and distortion levels.