PPD: Permutation Phase Defense Against Adversarial Examples in Deep Learning

1 INTRODUCTION . Recent advancements in deep learning have brought Deep Neural Networks ( DNNs ) to securitysensitive applications such as self-driving cars , malware detection , face recognition , etc . Although DNNs provide tremendous performance in these applications , they are vulnerable to adversarial attacks ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Athalye & Sutskever , 2017 ) . Especially in computer vision applications , small adversarial perturbation of the input can mislead the state-ofthe-art DNNs while being imperceptible to human eye ( Goodfellow et al. , 2014 ; Nguyen et al. , 2015 ; Moosavi-Dezfooli et al. , 2017 ; Akhtar & Mian , 2018 ) . Since the existence of adversarial examples was discovered ( Szegedy et al. , 2013 ) , numerous attacks and defenses have been proposed . Attacks typically try to push the image towards the decision boundaries of the classifier ( Goodfellow et al. , 2014 ; Carlini & Wagner , 2016 ; Kurakin et al. , 2016 ; Dong et al. , 2018 ) . Defenses , on the other hand , can be categorized into three groups : ( a ) adversarial training , ( b ) hiding the classifier , and ( c ) hiding the input image . Adversarial Training . As a natural defense , Szegedy et al . ( 2013 ) ; Goodfellow et al . ( 2014 ) suggested to augment the training set with adversarial examples to reshape the decision boundaries of the classifier . Adversarial training requires an attack algorithm to generate adversarial examples during training . The resulting classifier is shown to be robust against the same attack used during training ( Goodfellow et al. , 2014 ; Madry et al. , 2017 ; Tramèr et al. , 2017 ) . However , it is still vulnerable to other attacks or even the same attack with more perturbation budget ( Sharma & Chen , 2018 ; Madry et al. , 2017 ) . Hiding the Classifier . The idea of hiding the classifier has been investigated in various ways . One naive way is to not reveal the weights of the neural network . However , adversarial examples generated by a substitute model can still fool the hidden model ( Papernot et al. , 2017 ; Liu et al. , 2016 ) , a phenomenon known as transferability of adversarial examples across different structures . Another way is to conceal the gradient of loss function to guard against attacks that take advantage of gradient ( Buckman et al. , 2018 ; Song et al. , 2017 ; Samangouei et al. , 2018 ; Guo et al. , 2017 ) . However , all of these defenses are circumvented by approximately recovering the gradient ( Athalye et al. , 2018 ) . Hiding the Input Image . Another general approach to diminish adversarial examples is to hide the input image . Following this idea , Guo et al . ( 2017 ) suggested using randomization to hide the input image by two methods : ( a ) image quilting : reconstruct image by replacing small patches with clean patches from a database and ( b ) randomly drop pixels and recover them by total variance minimization . One fundamental restriction of these randomized defenses is that a large portion of pixels has to be preserved for correct classification . This leaves the door open for adversarial attacks . Indeed , both of these defenses are completely broken ( Athalye et al. , 2018 ) . Given that the defenses proposed under the last two categories are bypassed , to the best of our knowledge , adversarial training with examples generated by Projected Gradient Descent ( PGD ) attack ( Madry et al. , 2017 ) remains current state-of-the-art . However , its robustness is limited to specific attack types with certain strength . Changing the attack type or exceeding the perturbation limit degrades the accuracy significantly . In this paper , Permutation Phase Defense ( PPD ) is proposed to mitigate adversarial examples . Following the philosophy of hiding the input image , we suggest to “ encrypt ” the image by applying two transformations before feeding the image to the neural network : First , randomly shuffle the pixels of the image with a fixed random seed hidden from the adversary . Second , convert the permuted image to the phase of its two dimensional Fourier transform ( 2d-DFT ) ( see Figure 1 ) . This pipeline is used in both training and inference phases . The Fourier transformation captures the freqnency at which adjacent pixels of the input change along both axes . This frequency depends on the relational positions of the pixels rather than their absolute value . On the other hand , the relational positions are already hidden from the adversary by using the permutation block . The random seed plays the role of the key in cryptography . PPD offers some advantages as follows : • PPD outperforms current state-of-the-art defense in two ways : First , PPD is a general defense and is not restricted to specific attack types . Second , accuracy of PPD does not drop drastically by increasing perturbation budget of adversary ( Table 1 ) . • In contrast to image quilting and random pixel drop ( Guo et al . ( 2017 ) ) , PPD randomly shuffles all the pixels in the pixel domain . Therefore , important pixels of the phase domain remain hidden because of their dependence on the relational positions of the pixels in the pixel domain . • Increasing adversarial perturbation budget is expected to decrease accuracy . PPD , however , can distribute perturbation over different pixels in the phase domain such that adversarial perturbation is not more effective than random noise ( Figures 4 and 5 ) . 2 PRELIMINARIES . 2.1 NOTATION . Let ( X , d ) be a metric space where X = [ 0 , 1 ] H×W×C is the image space in the pixel domain for images with height H , width W and C channels and d : X × X → [ 0 , ∞ ) is a metric . A classifier h ( · ) is a function from X to label probabilities such that hi ( x ) is the probability that image x corresponds to label i . Let c∗ ( x ) be the true label of image x and c ( x ) = argmaxi hi ( x ) be the label predicted by the classifier . 2.2 ADVERSARIAL EXAMPLES . Intuitively speaking , adversarial examples are distorted images that visually resemble the clean images but can fool the classifier . Thus , given a classifier h ( · ) , > 0 and image x ∈ X , adversarial image x′ ∈ X satisfies two properties : d ( x , x′ ) ≤ and c ( x′ ) 6= c∗ ( x ) . Ideally , metric d should represent visual distance . However , there is no clear mathematical metric for visual distance . Therefore , to compare against a common ground , researchers have considered typical mathematical metrics induced by ` 2 and ` ∞ norms . 3 PERMUTATION PHASE DEFENSE ( PPD ) . 3.1 DEFENSE STRUCTURE . We address the problem of defending against adversarial examples . We hide the input space by two transformations preceding the main neural network ( see Figure 1 ) : • Random permutation block : pixels of the input image are permuted according to a fixed random permutation . The permutation seed is fixed and hidden from the adversary . • Pixel to phase block : permuted image is converted to the phase of its 2-dimensional discrete Fourier transform ( 2d-DFT ) . These two blocks precede the neural network in both training and testing phases . Mathematically speaking , consider channel c of image x ∈ X . Its 2d-DFT can be written as Flk = 1√ HW H−1∑ h=0 W−1∑ w=0 xhwe −j2π ( lH h+ k W w ) ( 1 ) where l = 0 , · · · , H − 1 and k = 0 , · · · , W − 1 and xhw is ( h , w ) image pixel value . Flk is a complex number that can be represented in the polar coordinates as Flk = Alk exp ( jϕlk ) where Alk ∈ [ 0 , ∞ ) is the magnitude and ϕlk ∈ [ −π , π ) is the phase of Flk . Let P = [ −π , π ) H×W×C be the phase space and p : X → P be a function that maps each channel of the image in the pixel domain to its phase . Let σ : X → X be a channel-wise permutation , i.e. , pixels in each channel of the input image are permuted according to a fixed permutation . Note that same permutation is used for all channels and all images . The PPD classifier can then be written as h ( x ) = f ( p ( σ ( x ) ) ( 2 ) where f ( · ) is the neural network . 3.2 INTUITION BEHIND THE DEFENSE . It is known that random noise is unable to fool a trained neural network ( Fawzi et al. , 2016 ) . Motivated by this fact , if we train the neural network in a domain that is hidden from adversary , adversarial perturbations will not affect more than random noise in the actual domain fed to the neural network and thus not fool the classifier . Following this idea , Fourier transformation is used to capture the frequency at which neighboring pixels change . Indeed , the values of the image in the Fourier domain do not purely depend on the values of pixels in the pixel domain . Instead , they depend on what pixels are adjacent in the pixel domain , while neighboring pixels are hidden from adversary by the random permutation block . Therefore , adversary will have little hope to attack the classifier due to almost no knowledge about the input space . 3.2.1 WHY RANDOM PERMUTATION IS NOT ENOUGH ? . We would like to emphasize that input transformations that partially hide the input space are not secure . For example , total variance minimization ( randomly drop pixels and replace them with total variance minimization ) , image quilting ( replacing patches of the input image by patches from clean images ) ( Guo et al. , 2017 ) are all defeated ( Athalye et al. , 2018 ) . One possible explanation for this failure is that although the adversary does not have full knowledge of the actual image fed to the neural network ( by using randomness in dropping pixels and patching ) , good portion of the image is still preserved . Hence , it is possible for adversary to attack . Similarly , we have observed that random permutation by itself ( without the pixel2phase block ) is not sufficient to defeat adversary . Adversarial images generated by attacking a vanilla neural network can substantially decrease the accuracy of a permuted network . This is because important pixels ( e.g . white pixels in MNIST ) in the original domain will still remain important pixels in the permuted domain . Therefore , regardless of the employed permutation , if adversary attacks important pixels of the original domain ( e.g . by converting white pixels to gray pixels in MNIST ) , it has successfully attacked important pixels in the permuted domain . 3.2.2 WHY PHASE OF 2D-DFT IS USED ? . It is well known that phase component of Fourier transform is crucial in visual perception of human ( Oppenheim & Lim , 1980 ; Morrone & Burr , 1988 ) . Indeed , an image reconstructed from phase only preserves essential information about the edges ( see Figure 2 ) . This suggests that phase carries key information about the image and thus it is possible to train neural networks to operate in the phase domain rather than the pixel domain . One might ask what if both phase and magnitude are fed to the neural network by doubling the size of the input space ? Our effort to train such a model was not fruitful but we leave it as a future research direction .

Summary: This paper explores the idea of utilizing a secret random permutation in the Fourier phase domain to defense against adversarial examples. The idea is drawn from cryptography, where the random permutation is treated as a secret key that the adversarial does not have access to. This setting has practical limitations, but is plausible in theory.