Unifying Bilateral Filtering and Adversarial Training for Robust Neural Networks

1 INTRODUCTION . Deep neural networks are known to be vulnerable to targeted perturbations added to benign inputs . The perturbed inputs , known as adversarial examples , can cause a classifier to output highly confident , but incorrect predictions . The majority of prior work has studied adversarial examples in the context of computer vision , where they pose the clearest threat . Small perturbations , imperceptible to humans , can be added to input images that cause a classifier to output false predictions . Because of the particular success of neural networks in computer vision , these models are being deployed in areas such as autonomous driving , facial recognition , and malware detection . Recent work has shown that these systems are vulnerable in the real world to adversarial examples ( Evtimov et al. , 2017 ) , which makes the problem of resisting adversarial attacks a growing concern . There have emerged two central lines of research for defending against adversarial examples . Denoising approaches attempt to remove the adversarial perturbations from the inputs as a preprocessing step . This is often done by filtering , or by projecting the input to a lower dimensional space that can not represent high frequency perturbations ( Samangouei et al. , 2018 ; Shen et al. , 2017 ) . These methods often lead to high accuracy , even on difficult datasets like ImageNet . But it has been shown that an attacker with knowledge of the defense can successfully circumvent them ( Athalye et al. , 2018 ) . On the other hand , Adversarial training methods use principles from robust optimization to train models which resist adversarial attacks . Under the adversarial training framework , adversarial examples are combined with the natural training set to increase the model ’ s robustness to attacks . These methods are expensive , requiring many more training examples , and have not been shown to scale well to natural image datasets such as ImageNet . This paper explores the utility of bilateral filtering as both a denoising defense and a useful addition to adversarial training . Bilateral filtering is a classic approach in computer vision for edge-aware smoothing . Because natural images are more likely piecewise-smooth while adversarial perturbations are less likely to be , we hypothesize that bilateral filtering would be able to filter out adversarial noises . Indeed , in experiments we found that with appropriate parameters , a plain bilateral filter can recover 99 % of the adversarial images so that a classifier can predict the original label . Furthermore , we introduce BFNet : an end-to-end model incorporating bilateral filtering as a differentiable layer . With BFNet , it is possible to examine the performance of white-box attacks trying to bypass our bilateral filtering defense . We show that BFNet is naturally robust to attacks from many such adversaries , greatly reducing the strength of both L∞ and L2 attacks on the ImageNet dataset . Finally , we combine bilateral filtering with adversarial training , and achieve state-of-the-art results on MNIST and CIFAR10 . Our method works with zero knowledge of either the network or any incoming attack , making it applicable to a variety of models and datasets . 2 RELATED WORK . 2.1 ADVERSARIAL ATTACKS . There have been many proposed attacks for creating adversarial examples . We give a brief description of the six attacks that we used to test our models . A . Projected Gradient Descent ( PGD ) In ( Lyu et al. , 2015 ; Madry et al. , 2018 ) , generating an adversarial example is the task of solving the objective maxδ≤ L ( θ , x + δ , ytrue ) . PGD is used to maximize this objective under a loss function L , yielding an image with a perturbation magnitude less than with respect to the L∞ norm , and achieves the highest possible loss on the true class . B . Fast Gradient Sign Method ( FGSM ) FGSM ( Goodfellow et al. , 2015 ) is a one step linearization of the above objective . FGSM finds adversarial examples by assuming linearity at the decision boundary . Given an image x , we find a perturbation η under the max norm : η = · sign ( ∇xL ( θ , x , y ) ) , where θ is the parameters of the network , y is the original label , and L is the loss function used to train the network . C. Momentum Iterative Method The Momentum Iterative Fast Sign Gradient Method ( MI-FGSM ) ( Dong et al. , 2018 ) is an iterative version of the FGSM attack . MI-FGSM moves pixel values linearly along the gradient toward the decision boundary . MI-FGSM improves on FGSM by introducing a momentum term into gradient calculation : gt+1 = µ ·gt+ ∇xL ( x ∗ t , y ) ‖∇xL ( x∗t , y ) ‖1 . The gradient is then used to iteratively update the image x∗t+1 = x ∗ t +α · ( gt+1 ) . The authors claim that simply using an iterative FGSM leads to greedy overfitting of the decision boundary , and thus falls into local poor maxima . Adding momentum stabilizes the update direction and creates a stronger adversarial example . D. L-BFGS-B ( Szegedy et al. , 2014 ) used box-constrained L-BFGS to generate adversarial examples with minimal distortion under the L2 norm . Given a natural image x and a target class ytrue , the adversarial objective is as follows : min [ c · ||x− ( x+ δ ) ||22 +L ( x+ δ , ytarget ) ] . Where δ is the adversarial perturbation , L is the loss function , and the parameter c controls the trade-off between the magnitude and strength of the perturbation . E. Carlini & Wagner Attack ( L2 ) ( Carlini & Wagner , 2017 ) proposed three iterative attacks which create adversarial examples under the L0 , L2 , and L∞ norms . In this work we consider the most powerful attack , the white-box L2 attack . Specifically , they minimize min || 12 ( tanh ( w ) +1 ) −x|| 2 2+ c × f ( 12 ( tanh ( w ) + 1 ) ) , where f ( x ′ ) = max ( max { Zi ( x′ ) : i 6= t } − Zt ( x′ ) , −κ ) . Here , t is the target label , Z refers to the logits of the network , κ controls the confidence of the new classification , and the 12 tanh term constrains the result to pixel space . F. DeepFool Deepfool is an iterative , first order method used to find minimal distortion under the L2 norm ( Moosavi-Dezfooli et al. , 2016 ) . Deepfool linearizes the classifier itself and performs gradient descent until the image is misclassified . The DeepFool objective is minδ ‖δ‖2 subject to arg max f ( x ) 6= arg max f ( x+δ ) . In addition to the attacks listed above , other methods have been proposed . L0 attacks such as ( Papernot et al. , 2016b ) choose to measure adversarial perturbations by the minimum change necessary to produce an incorrect prediction . 2.2 ADVERSARIAL DEFENSES . There is a growing body of work on defenses against adversarial attacks ( Papernot & McDaniel , 2016 ; Papernot et al. , 2015 ; Xu et al. , 2017 ; Liao et al. , 2018 ) . An averaging filter was studied in ( Li & Li , 2017 ) . JPEG compression was studied in ( Dziugaite et al. , 2016 ; Das et al. , 2017 ) , and was found to be effective at removing adversarial perturbations . However , JPEG encoding is not differentiable , hence its performance when the adversary has knowledge of the defense is unknown . Our bilateral filtering approach is fully differentiable hence we can test it against counter-attacks . Other recent defenses attempt to remove adversarial perturbations by projecting inputs back onto the real data manifold ( Meng & Chen , 2017 ) . ( Shen et al. , 2017 ) projects inputs using a generative adversarial network . Given a normal or adversarial image , a generator is trained to produce a image from the normal data distribution . This method also did not test against counter-attacks , and has been shown to be successfully fooled by the CW attack ( Meng & Chen , 2017 ) . Our approach can also be seen as a projection back to the data manifold , where we impose the constraint that the resulting image must be piecewise-smooth . By fixing the filter approach , we would likely not overfit significantly to the training set and remain effective under counter-attacks . On the other hand , adversarial training methods ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ; Shaham et al. , 2018 ; Tramèr et al. , 2018 ) combine adversarial examples with the natural training set to increase the robustness of the model to adversarial attacks . These approaches are promising as they attempt to provide a guarantee on both the type of adversary and the magnitude of the perturbation they are resistant to . In practice however , these methods are hard to scale as they require expensive computation in the inner training loop to generate adversarial examples . When training on a large dataset such as ImageNet , generating a sufficient amount of strong adversarial examples can be intractable . This problem has been mitigated by training against a weak adversary like FGSM ( Tramèr et al. , 2018 ) which can quickly generate adversarial examples . But training models that are robust to strong adversaries on ImageNet or CIFAR-10 is still an open problem . 3 METHOD . In this paper , we consider white-box threat models where the attacker has full access to the training data , model parameters and architecture . This is categorically more difficult than black-box threat models where the attacker has little or no knowledge about the model or training data . We will first show the utility of bilateral filter against simple attacks without knowledge of the network , then introduce BFNet with bilateral filtering as a differentiable layer , so that we can evaluate attacks with knowledge of our defense . 3.1 THE BILATERAL FILTER AND ITS CAPABILITY OF RECOVERING ADVERSARIAL IMAGES . The bilateral filter is a non-linear Gaussian filter that is commonly used to smooth image gradients while preserving sharp edges . For an image I , window Ω centered at pixel p , the bilateral filter is formulated as a domain function Gs , and a range function Gr : Ifiltered ( p ) = 1 Wp ∑ q∈Ω Gs ( ||p− q|| ) Gr ( ‖Ip − Iq‖ ) Iq where the normalization term Wp is : Wp = ∑ q∈Ω Gs ( ‖p− q‖ ) Gr ( ‖Ip − Iq‖ ) , Gs ( x ) = exp ( − x 2 2σ2s ) and Gr ( x ) = exp ( − x 2 2σ2r ) are Gaussian filters , and σs and σr are parameters which control the strength of the domain and range functions respectively . Each neighboring pixel is assigned a weight according to both spatial closeness and value difference . Hence , if the color of the pixels p and q are very different , then q will affect the filtered image at pixel p very little . At sharp image boundaries , this would effectively lead to smoothing on only one side of the boundary , since the other side would have very different color . Hence , sharp boundaries can be preserved and oversmoothing or blurring that are commonly seen in Gaussian smoothing or averaging can be prevented . In Fig.1 one can see the effect of denoising an L-BFGS-B adversarial image , where an averaging filter will leave the image significantly blurred , but bilateral filtering would preserve the edges . More images are shown in the appendix in Fig . 3 . We believe piecewise-smoothness is an inherent property of many images hence bilateral filtering offered a projection back to this manifold of piecewise-smooth images . Convolutional networks only work on images from the natural image manifold , which left the hole for adversarial examples to maneuver by creating off-manifold images . By using bilateral filtering to force images to be on the manifold , we would leave significantly less holes for adversarials to maneuver on . To test the efficacy of the bilateral filter to recover clean inputs from adversarial examples , we generated a set of adversarial examples from a range of powerful adversaries . Our first approach was to manually tune parameters for each input image , to test the effective range of parameters which could recover the original label from an adversarial example . We found that with carefully chosen parameters , the corrupted labels could indeed be recovered . Our experiments showed that the small perturbations created by iterative methods like the Carlini & Wagner attack and DeepFool were easier to remove with a bilateral filter than the larger perturbations created with one step attacks . To remove perturbations generated by iterative attacks , we used small kernels 3 - 5 pixels wide , and σs , σr values of 0.5 . Filtering with larger kernel sizes offers no benefit , as the resulting images from iterative attacks have imperceptible perturbations which are removed with small filters . One step attacks perturb every pixel in the image with the same magnitude of noise . As a result , we increased kernel width to 7 and σs to 3 , holding σr constant . These parameters reliably removed adversarial perturbations from L∞ attacks with a bounded distance of 0.3 , as well as unbounded L2 attacks . The results can be found in Table 1 .

Summary: The paper proposes the use of bilateral filtering as a defense to adversarial examples (AE). Bilateral Filtering (BF) is a smoothening technique that averages pixels that are close to each other both in position and value. As a result BF preserves sharp edges and the denoised images appear to be of higher quality than those produces via simple Gaussian smoothing.