Title: UNIFYING BILATERAL FILTERING AND ADVERSARIAL TRAINING FOR ROBUST NEURAL NETWORKS Anonymous

Abstract: Recent analysis of deep neural networks has revealed their vulnerability to carefully structured adversarial examples. Many effective algorithms exist to craft these adversarial examples, but performant defenses seem to be far away. In this work, we explore the use of edge-aware bilateral filtering as a projection back to the space of natural images. We show that bilateral filtering is an effective defense in multiple attack settings, where the strength of the adversary gradually increases. In the case of adversary who has no knowledge of the defense, bilateral filtering can remove more than 90% of adversarial examples from a variety of different attacks. To evaluate against an adversary with complete knowledge of our defense, we adapt the bilateral filter as a trainable layer in a neural network and show that adding this layer makes ImageNet images significantly more robust to attacks. When trained under a framework of adversarial training, we show that the resulting model is hard to fool with even the best attack methods.

1. INTRODUCTION:

Deep neural networks are known to be vulnerable to targeted perturbations added to benign inputs. The perturbed inputs, known as adversarial examples, can cause a classifier to output highly confident, but incorrect predictions. The majority of prior work has studied adversarial examples in the context of computer vision, where they pose the clearest threat. Small perturbations, imperceptible to humans, can be added to input images that cause a classifier to output false predictions. Because of the particular success of neural networks in computer vision, these models are being deployed in areas such as autonomous driving, facial recognition, and malware detection. Recent work has shown that these systems are vulnerable in the real world to adversarial examples (Evtimov et al., 2017) , which makes the problem of resisting adversarial attacks a growing concern.

There have emerged two central lines of research for defending against adversarial examples. Denoising approaches attempt to remove the adversarial perturbations from the inputs as a preprocessing step. This is often done by filtering, or by projecting the input to a lower dimensional space that cannot represent high frequency perturbations (Samangouei et al., 2018; Shen et al., 2017) . These methods often lead to high accuracy, even on difficult datasets like ImageNet. But it has been shown that an attacker with knowledge of the defense can successfully circumvent them (Athalye et al., 2018) . On the other hand, Adversarial training methods use principles from robust optimization to train models which resist adversarial attacks. Under the adversarial training framework, adversarial examples are combined with the natural training set to increase the model's robustness to attacks. These methods are expensive, requiring many more training examples, and have not been shown to scale well to natural image datasets such as ImageNet.

This paper explores the utility of bilateral filtering as both a denoising defense and a useful addition to adversarial training. Bilateral filtering is a classic approach in computer vision for edge-aware smoothing. Because natural images are more likely piecewise-smooth while adversarial perturbations are less likely to be, we hypothesize that bilateral filtering would be able to filter out adversarial noises. Indeed, in experiments we found that with appropriate parameters, a plain bilateral filter can recover 99% of the adversarial images so that a classifier can predict the original label. Furthermore, we introduce BFNet: an end-to-end model incorporating bilateral filtering as a differentiable layer. With BFNet, it is possible to examine the performance of white-box attacks trying to bypass our bilateral filtering defense. We show that BFNet is naturally robust to attacks from many such adversaries, greatly reducing the strength of both L ∞ and L 2 attacks on the ImageNet dataset.

Finally, we combine bilateral filtering with adversarial training, and achieve state-of-the-art results on MNIST and CIFAR10. Our method works with zero knowledge of either the network or any incoming attack, making it applicable to a variety of models and datasets.

2. RELATED WORK:



2.1. ADVERSARIAL ATTACKS:

There have been many proposed attacks for creating adversarial examples. We give a brief description of the six attacks that we used to test our models. A. Projected Gradient Descent (PGD) In (Lyu et al., 2015; Madry et al., 2018) , generating an adversarial example is the task of solving the objective max δ≤ L(θ, x + δ, y true ). PGD is used to maximize this objective under a loss function L, yielding an image with a perturbation magnitude less than with respect to the L ∞ norm, and achieves the highest possible loss on the true class. B. Fast Gradient Sign Method (FGSM) FGSM (Goodfellow et al., 2015) is a one step linearization of the above objective. FGSM finds adversarial examples by assuming linearity at the decision boundary. Given an image x, we find a perturbation η under the max norm: η =

• sign(∇ x L(θ, x, y)), where θ is the parameters of the network, y is the original label, and L is the loss function used to train the network. C. Momentum Iterative Method The Momentum Iterative Fast Sign Gradient Method (MI-FGSM) (Dong et al., 2018) is an iterative version of the FGSM attack. MI-FGSM moves pixel values linearly along the gradient toward the decision boundary. MI-FGSM improves on FGSM by introducing a momentum term into gradient calculation:

g t+1 = µ•g t + ∇xL(x * t ,y) ∇xL(x * t ,y) 1 . The gradient is then used to iteratively update the image x * t+1 = x * t + α • (g t+1 ).

The authors claim that simply using an iterative FGSM leads to greedy overfitting of the decision boundary, and thus falls into local poor maxima. Adding momentum stabilizes the update direction and creates a stronger adversarial example. D. L-BFGS-B (Szegedy et al., 2014) used box-constrained L-BFGS to generate adversarial examples with minimal distortion under the L 2 norm. Given a natural image x and a target class y true , the adversarial objective is as follows:

min c • ||x -(x + δ)|| 2 2 + L(x + δ, y target ) .

Where δ is the adversarial perturbation, L is the loss function, and the parameter c controls the trade-off between the magnitude and strength of the perturbation. E. Carlini & Wagner Attack (L 2 ) (Carlini & Wagner, 2017) proposed three iterative attacks which create adversarial examples under the L 0 , L 2 , and L ∞ norms. In this work we consider the most powerful attack, the white-box L 2 attack. Specifically, they minimize min

|| 1 2 (tanh(w)+1)-x|| 2 2 + c × f ( 1 2 (tanh(w) + 1)), where f (x ) = max(max{Z i (x ) : i = t} -Z t (x ), -κ).

Here, t is the target label, Z refers to the logits of the network, κ controls the confidence of the new classification, and the 1 2 tanh term constrains the result to pixel space. F. DeepFool Deepfool is an iterative, first order method used to find minimal distortion under the L 2 norm (Moosavi-Dezfooli et al., 2016) . Deepfool linearizes the classifier itself and performs gradient descent until the image is misclassified. The DeepFool objective is min δ δ 2 subject to arg max f (x) = arg max f (x+δ). In addition to the attacks listed above, other methods have been proposed. L 0 attacks such as (Papernot et al., 2016b) choose to measure adversarial perturbations by the minimum change necessary to produce an incorrect prediction.

2.2. ADVERSARIAL DEFENSES:

There is a growing body of work on defenses against adversarial attacks (Papernot & McDaniel, 2016; Papernot et al., 2015; Xu et al., 2017; Liao et al., 2018 ). An averaging filter was studied in (Li & Li, 2017) . JPEG compression was studied in (Dziugaite et al., 2016; Das et al., 2017) , and was found to be effective at removing adversarial perturbations. However, JPEG encoding is not differentiable, hence its performance when the adversary has knowledge of the defense is unknown. Our bilateral filtering approach is fully differentiable hence we can test it against counter-attacks.

Other recent defenses attempt to remove adversarial perturbations by projecting inputs back onto the real data manifold (Meng & Chen, 2017) . (Shen et al., 2017) projects inputs using a generative adversarial network. Given a normal or adversarial image, a generator is trained to produce a image from the normal data distribution. This method also did not test against counter-attacks, and has been shown to be successfully fooled by the CW attack (Meng & Chen, 2017) . Our approach can also be seen as a projection back to the data manifold, where we impose the constraint that the resulting image must be piecewise-smooth. By fixing the filter approach, we would likely not overfit significantly to the training set and remain effective under counter-attacks.

On the other hand, adversarial training methods (Goodfellow et al., 2015; Madry et al., 2018; Shaham et al., 2018; Tramèr et al., 2018) combine adversarial examples with the natural training set to increase the robustness of the model to adversarial attacks. These approaches are promising as they attempt to provide a guarantee on both the type of adversary and the magnitude of the perturbation they are resistant to. In practice however, these methods are hard to scale as they require expensive computation in the inner training loop to generate adversarial examples. When training on a large dataset such as ImageNet, generating a sufficient amount of strong adversarial examples can be intractable. This problem has been mitigated by training against a weak adversary like FGSM (Tramèr et al., 2018) 

3. METHOD:

In this paper, we consider white-box threat models where the attacker has full access to the training data, model parameters and architecture. This is categorically more difficult than black-box threat models where the attacker has little or no knowledge about the model or training data. We will first show the utility of bilateral filter against simple attacks without knowledge of the network, then introduce BFNet with bilateral filtering as a differentiable layer, so that we can evaluate attacks with knowledge of our defense.

3.1. THE BILATERAL FILTER AND ITS CAPABILITY OF RECOVERING ADVERSARIAL IMAGES:

The bilateral filter is a non-linear Gaussian filter that is commonly used to smooth image gradients while preserving sharp edges. For an image I, window Ω centered at pixel p, the bilateral filter is formulated as a domain function G s , and a range function G r :

I f iltered (p) = 1 W p q∈Ω G s (||p -q||)G r ( I p -I q ) I q

where the normalization term W p is:

W p = q∈Ω G s ( p -q )G r ( I p -I q ), G s (x) = exp(-x 2 2σ 2 s ) and G r (x) = exp(-x 2 2σ 2 r

) are Gaussian filters, and σ s and σ r are parameters which control the strength of the domain and range functions respectively. Each neighboring pixel is assigned a weight according to both spatial closeness and value difference. Hence, if the color of the pixels p and q are very different, then q will affect the filtered image at pixel p very little. At sharp image boundaries, this would effectively lead to smoothing on only one side of the boundary, since the other side would have very different color. Hence, sharp boundaries can be preserved and oversmoothing or blurring that are commonly seen in Gaussian smoothing or averaging can be prevented. In Fig. 1 one can see the effect of denoising an L-BFGS-B adversarial image, where an averaging filter will leave the image significantly blurred, but bilateral filtering would preserve the edges. More images are shown in the appendix in Fig. 3 .

We believe piecewise-smoothness is an inherent property of many images hence bilateral filtering offered a projection back to this manifold of piecewise-smooth images. Convolutional networks only work on images from the natural image manifold, which left the hole for adversarial examples to maneuver by creating off-manifold images. By using bilateral filtering to force images to be on the manifold, we would leave significantly less holes for adversarials to maneuver on.

To test the efficacy of the bilateral filter to recover clean inputs from adversarial examples, we generated a set of adversarial examples from a range of powerful adversaries. Our first approach was to manually tune parameters for each input image, to test the effective range of parameters which could recover the original label from an adversarial example. We found that with carefully chosen parameters, the corrupted labels could indeed be recovered. Our experiments showed that the small perturbations created by iterative methods like the Carlini & Wagner attack and DeepFool were easier to remove with a bilateral filter than the larger perturbations created with one step attacks. To remove perturbations generated by iterative attacks, we used small kernels 3 -5 pixels wide, and σ s , σ r values of 0.5. Filtering with larger kernel sizes offers no benefit, as the resulting images from iterative attacks have imperceptible perturbations which are removed with small filters. One step attacks perturb every pixel in the image with the same magnitude of noise. As a result, we increased kernel width to 7 and σ s to 3, holding σ r constant. These parameters reliably removed adversarial perturbations from L ∞ attacks with a bounded distance of 0.3, as well as unbounded L 2 attacks. The results can be found in Table 1 . One caveat to the above approach, is that the parameters for the bilateral filter must be carefully chosen to be able to recover the accuracy and confidence of the original classification. Large values for the parameters σ s and σ r can create an excessively blurred image, and a small filter size K may capture insufficient information to remove the adversarial perturbations. With this in mind, we train a small network which will predict the parameters of the bilateral filter (K, σ s , σ r ) for an input image. This network will serve as a cheap preprocessing step that will remove adversarial perturbations without affecting the underlying class label.

To build our classifier we first extract information about the distribution of pixel gradients by convolving the input with a Sobel filter in the x and y direction. Because adversarial attacks directly change values of the input, adversarial examples will often have larger color gradients in the x and y direction than natural images. We concatenate the gradient map depth-wise with the input image, and use three dilated convolutional layers with 64, 128, and 256 filters respectively, followed by 2x2 max pooling and a linear layer of 64 units. We use a dilation rate of 2 for each convolutional layer. Note this experiment is stand-alone and it is not utilized in the BFNet proposed in the next section.

3.3. BFNET: ADDING BILATERAL FILTERING AS A TRAINABLE LAYER:

The main idea of BFNet is to always preprocess the input image with bilateral filter before inputting it into the CNN. Namely, instead of computing f (x) where f is learned by a deep network, always computing f (BF (x)) instead. Hence we can then optimize for attacks that have full knowledge and gradients about our defense. This has two utilities, one is to examine the robustness of the defense, and secondly we can add the newly generated adversarial examples back to the training set of the network, in order to perform adversarial training.

A brute-force implementation of the bilateral filter has a O(n 2 ) cost associated with computing the response of individual pixels. Making it the most expensive operation in the graph. To reduce computation time, We choose as our preprocessing function the Permutohedral Lattice implementation of the bilateral filter (Adams et al., 2010) , which is also fully differentiable and can be computed in O(n) time. This can then be attached as the first layer to any other network, and the bilateral filter parameters can be trained jointly with other parts of the network.

3.4. ADVERSARIAL TRAINING:

It has been shown that under the white-box threat model, using a denoiser as the only defense is insufficient to stop the strongest adversarial attacks. Currently the most promising direction for training models robust to adversarial attacks is adversarial training. Despite continuing progress on both MNIST and CIFAR10, adversarial training is still very expensive, and performs worse than denoising approaches on the same datasets. We propose an approach combining adversarial training with BFNet, giving a robust, performant classifier on different threat models.

Following (Madry et al., 2018; Athalye et al., 2018; Carlini & Wagner, 2017) , the adversarial training framework can be expressed as the following saddle point problem with model parameters θ, and input x with true label y:

min θ f (x; θ) where f (x; θ) = E (x,y)∼D max δ L(θ, x + δ, y)

where a solution to the inner maximization problem represents the most adversarial example within some perturbation budget. Solving the outer minimization problem yields a classifier which is robust to the above adversary. (Madry et al., 2018) showed that PGD could reliably solve the inner maximization problem without linearization, and is thus a better adversary to train against than FGSM.

We propose a modification to the above saddle point formulation which incorporates the BFNet:

min θ f (BF (x)) where f (x) = E (x,y)∼D max δ L(θ, x + δ, y true )

where BF (x) is the bilateral filter in BFNet. Incorporating this, we train the entire BFNet adversarially, including the filtering layer. We report the results of our experiemnts in the following section.

4. EXPERIMENTS:



4.1. ADAPTIVE FILTERING MODEL:

In this section we show that our adaptive filtering model can correctly predict filtering parameters which will restore an adversarial input. To construct our training set we use a separate 1,000 images generated from each of the attacks in table 1 . For each image, we collect labels in the form of triples (K, σ s , σ r ), K denotes the kernel size, and σ s , σ r are the standard deviation for the spatial and range kernels respectively. Given any adversarial example, there may be many permutations of parameters for the bilateral filter that successfully denoise the input. For this reason we collect a maximum of 10 different parameter configurations for each image in our training set. Given this is a multi-class prediction problem, we train using a sigmoid function at the output of our network to predict a set of candidate parameter configurations. At test time we evaluate with the parameters predicted by the maximally activated output unit. We evaluate results under two threat models. The first (presented in table 2 ) is where we attempt to change the label given to the adversarial example by our classifier. In this setting, we are trying to defeat an attack who needs a specific label to be the output. In other settings, the attacker believes any label is sufficient so long as its not the true label. In this case we measure our ability to recover the true label in table 3 .

We used the pretrained Inception V3 (Szegedy et al., 2016) and InceptionResNet V2 (Szegedy et al., 2017) ImageNet classifiers as our source networks. To generate adversarial examples on these networks, we used the open-source Cleverhans toolbox (Papernot et al., 2016a) . the model was trained using SGD with Nesterov momentum for 25 epochs. We then test on six different validation sets, one for each adversary respectively. (Li & Li, 2017) , which used a 3x3 average filter to recover images. Our Adaptive Filtering network succeeds in removing adversarial examples generated on natural images, with a relatively simple network. This makes the Adaptive Filtering network a viable method for defending networks against an adversary who employs a wide range of attacks.

4.2. BFNET DEFENDING AGAINST COUNTER ATTACKS ON IMAGENET:

Due to the high cost of adversarial training on natural images, on ImageNet we perform only one round of counter-attack, which is: have the attack knows about BFNet and attack it by backpropagating through the entire BFNet defense. We use this as an opportunity to test the robustness of BFNet that cannot be attributed to adversarial training. To this end, we use the Inception V3 and the Inception-ResNet V2 networks, and add our bilateral filter layer to the input, keeping the pretrained ImageNet weights. We test against both L 2 and L ∞ adversaries to obtain a complete picture of the robustness of BFNet. L ∞ is a more informative metric when discussing the magnitude of adversarial attacks on very small images, because a large perturbation measured under the L ∞ norm equates to a large visual change across few pixels.

To measure resistance to attacks under the L 2 norm, we use the unbounded attacks L-BFGS and DeepFool. It is impossible to be fully resistant to unbounded attacks, because any image can be changed to a completely different image and its CNN output would certainly change. Hence, we report the average L 2 and L ∞ distance of the adversarial images to the original ones from the unbounded attacks. From Table 4 we can see that our approach yields a very robust model against adversarial perturbations under the L 2 metric. When attacking our BFNet models with DeepFool, we see that the generated adversarial image has an L ∞ distance over 30x larger, when compared to an unmodified network of the same architecture. Similarly, we can see that the L 2 distance of an adversarial generated against BFNet is far larger when compared to adversarial images generated against a network of the same architecture without the bilateral filter. With respect to the L-BFGS attack, we see a similarly large disparity between BFNet and a vanilla network. Fig. 2 shows some examples of images generated by those adversarial counterattacks. One can see that the DeepFool and LBFGS attacks had to significantly modify the image to defeat BFNet, creating clearly visible patterns.

For the L ∞ attacks such as FGSM, and MI-FGSM we measure the resistance of our model to different values of perturbation . We can see that our BFNet significantly decreases the attack strength of L ∞ adversaries, in most cases by over 50%. Of particular note is that we show more significant resistance to adversarial perturbations of ≤ 0.3. Larger perturbations are visually discernible, and thus are less adversarial than smaller fooling perturbations. For both attacks we use 1,000 random images sampled from the ILSVRC 2012 validation set, and report the percentage of successful attacks against the natural model and BFNet respectively.

Finally, we experiment on adversarial training with BFNet on the MNIST and CIFAR-10 datasets.

4.3.1. MNIST:

To show that BFNet is robust to strong first-order adversaries, we train a small CNN to 99.2% accuracy on the test set. Table 5 : Performance of BFNet against FGSM and MI-FGSM adversaries for a range of perturbation sizes (lower is better). For our MI-FGSM attack, we use a momentum decay factor of 1.0, and run the attack for 10 iterations each followed by 2 x 2 max pooling and ReLU. We use a final fully connected layer with 1024 units. We modify our network into a BFNet by adding our bilateral filter layer at the input of the first convolutional layer. We then train the entire BFNet (including the filtering layer) with adversarial training using three distinct adversaries: FGSM, PGD, and PGD with the proposed CW loss function. We report the results in table 6. Our results perform well against the state-of-the-art adversarial training results. We also show that when our network is trained on a single strong adversary, we are robust to attacks from other adversaries. (Tramèr et al., 2018) , as well as B: the performance of architecture B from (Madry et al., 2018) ; RIGHT: Performance of our two adversarially trained BFNets on CIFAR-10. BF pgd denotes our model trained against a PGD adversary, while BFNet fgsm is trained against a FGSM adversary. Network type (A) refers to the ResNet network used in (Madry et al., 2018) , while (B) refers to the smaller architecture.

During training we observe a faster convergence in training loss (see Appendix), and increased robustness to white-box FGSM, PGD, and CW attacks, when trained against only the PGD attack. However, the model trained against FGSM does worse against stronger adversaries such as PGD, as the attack itself is a weak adversary.

4.3.2. CIFAR10:

We perform similar experiments to test BFNet on CIFAR-10. We use a network with four convolutional layers, each followed by 2x2 max pooling. A linear layer of 4, 096 units is used before softmax. For BFNet, a differentiable bilateral filter layer to preprocess the images. When naturally trained with Adam for 30 epochs, this network reaches an accuracy of 79.04% on the test set. We also train the original ResNet-18 model used in (Madry et al., 2018) for 80K iterations. Trained on natural examples we reached an accuracy of 92.7% on the test set. Each model is then trained adversarially with PGD and FGSM, respectively. We use an L ∞ bound of = 8 for both adversaries. We use 20-step PGD with a learning rate of 2.0. We report our results in Table 6 and it can be seen that our BFNet trained on PGD outperforms (Madry et al., 2018) significantly on the PGD adversary.

In contrast to MNIST, CIFAR-10 remains a very challenging dataset. The higher dimensionality makes robust training significantly more difficult. Because CIFAR-10 is too small, the edges are not so obvious, which could have hurted our performance. We believe that the bilateral filtering poses more constraint in larger natural images, such as ImageNet.

CONCLUSION:

The continued existence of adversarial examples, and the lack of effective defenses limits our ability to deploy AI systems in critical areas where safety and security are necessary. In this paper we showed that a bilateral filter can be used as a core part of versatile, effective defenses to recover clean images from perturbed ones. The bilateral filter remains effective when deployed with numerous defense strategies: as a manual preprocessing step, a trained denoiser, or a robust model that is trained end-to-end. Our defense holds in multiple attack settings where the attacker has knowledge about it. In the future we hope to explore even better filter approaches which projects even better to the natural image manifold and limit adversarial examples, as well as combining it with an adversarial detection approach to construct a comprehensive defense.

5. APPENDIX:



5.1. MORE EXAMPLES OF BILATERAL FILTERING RESULTS ON ADVERSARIAL IMAGES:

Fig. 3 shows more examples of adversarial images from several different attacks after bilateral filtering. 

5.5. BLACK-BOX ATTACKS ON BFNET:

We performed two seperate black box attacks on BFNet to tests its robustness to a new kind of adversary. We tested on the non-adversarially trained imagenet models InceptionResNet-V2 and Inception-V3 (corresponding to Table 4 ). We report the results in the updated appendix of the paper.

For both attacks we used the implementation given in the Foolbox toolbox.

First, we used the Boundary attack from (Brendel et al., 2017) : a decision-based attack that begins with a large adversarial perturbation, then reduces the magnitude of the perturbation while retaining an adversarial label. This attack has been shown to be effective and scale to high dimensional inputs. Second, we used an approximate L-BFGS attack where the gradients are computed numerically. Using a numerical gradient avoids the instabilities in iterative attacks which occur in gradient-masking defenses. We report the average L2 distance achieved of these attacks against BFNet of 1000 images. We show the results in table 7 From the results given, we can see the BFNet does well against both the decision-based attack and the approximate L-BFGS, showing that it is indeed resistant to such kind of attacks as well.

:



Figures and Tables:

Figure 1: (a) The original LBFGS-B adversarial image, (b) The image after 3x3 bilateral filtering and (c) The image after 3x3 averaging filtering. The bilateral filter is superior since it removes small perturbations while preserving sharp edges in the image, keeping it from becoming blurry

Figure 2: Adversarial images created with BFNet. See the appendix for adversarial images from the same original images without BFNet DeepFool L-BFGS Network L ∞ L 2 L ∞ L 2 Inception V3 Natural 0.015 0.43 0.02 0.67 Inception V3 BFNet 0.621 148.29 0.39 90.52 IncResNet V2 Natural 0.025 0.44 0.06 0.77 IncResNet V2 BFNet 0.793 187.45 0.65 90.65 Table 4: Performance of BFNet against DeepFool and L-BFGS attacks. We report the average L 2 and L ∞ distance of 1, 000 adversarial images on Inception V3 and Inception-ResNet V2.

Figure 3: The effect of bilateral filtering on adversarial inputs. From left to right, we show the adversarial perturbation, the clean image, the adversarial images generated by the respective attack algorithms, and the recovered image after bilateral filtering. Note that bilateral filtering does not destroy image quality, and images can be correctly classified.

(a) Adversarial images generated with DeepFool on a vanilla Inception V3 classifier. The adversarial images are visually identical to the real images, and have an average L2 norm of 0.09 (b) Adversarial examples generated by an L-BFGS adversary on a vanilla Inception V3 classifier. The adversarial examples have an average L2 distance of 0.025 from their natural counterparts, and have visually imperceptible perturbations.

Figure 4: Adversarial images generated by DeepFool and L-BFGS without BFNet, to be compared with Fig.2

which can quickly generate adversarial examples. But training models that are robust to strong adversaries on ImageNet or CIFAR-10 is still an open problem.

Table content: <table/>



Table content: <table><tr><td/><td/><td>5</td><td>98.8</td><td>99.2</td><td>97.8</td></tr><tr><td colspan="2">InceptionResNet V2 94.2</td><td>98.4</td><td>96.3</td><td>98.8</td><td>95.1</td></tr><tr><td>ResNet V2</td><td>96.5</td><td>98.0</td><td>96.1</td><td>98.1</td><td>98.0</td></tr></table>

Performance of our adaptive bilateral filter (AF) network across different attacks. We show (A) the top-5 accuracy of recovering the original predicted classification label from the adversarial example (note this is not necessarily the ground truth label), as well as (B) how often AF is able to defeat the adversarial attack -changing the prediction from the adversarial label to a new one

Table content: <table><tr><td>Source Network</td><td colspan="6">Clean FGSM PGD MIM CW DeepFool L-BFGS</td></tr><tr><td>AF+Inception V3 A</td><td>95.0</td><td>89.0</td><td colspan="2">90.7 79.1</td><td>89.1 90.3</td><td>81.3</td></tr><tr><td>AF+Inception V3 B</td><td>95.0</td><td>95.9</td><td colspan="2">98.0 96.4</td><td>94.1 95.3</td><td>96.2</td></tr><tr><td colspan="2">AF+InceptionResNet V2 A 91.1</td><td>87.2</td><td colspan="2">87.1 75.3</td><td>87.8 85.0</td><td>80.8</td></tr><tr><td colspan="2">AF+InceptionResNet V2 B 91.1</td><td>93.1</td><td colspan="2">98.0 94.3</td><td>97.8 92.0</td><td>95.3</td></tr><tr><td>Network</td><td colspan="6">Clean FGSM PGD MIM CW DeepFool L-BFGS</td></tr><tr><td>Inc V3 top1</td><td>78.8</td><td>30.1</td><td>0.2</td><td>0.1</td><td>0.1 0.7</td><td>0.0</td></tr><tr><td>Inc V3 top5</td><td>94.4</td><td>65.2</td><td>4.8</td><td>5.5</td><td>7.3 0.5</td><td>12.1</td></tr><tr><td>AF + Inc V3 top1</td><td>71.7</td><td>71.0</td><td colspan="3">71.6 63.1 71.1 70.1</td><td>64.2</td></tr><tr><td>AF + Inc V3 top5</td><td>89.6</td><td>84.0</td><td colspan="3">86.3 74.6 84.1 85.2</td><td>76.7</td></tr><tr><td>IncResNet V2 top1</td><td>80.4</td><td>55.3</td><td>0.8</td><td>0.5</td><td>0.3 2.5</td><td>0.0</td></tr><tr><td>IncResNet V2 top5</td><td>95.3</td><td>72.1</td><td colspan="3">15.8 10.2 10.3 8.5</td><td>19.2</td></tr><tr><td colspan="2">AF + IncResNet V2 top1 73.1</td><td>70.1</td><td colspan="3">70.8 60.5 70.3 70.5</td><td>65.0</td></tr><tr><td colspan="2">AF + IncResNet V2 top5 86.7</td><td>83.1</td><td colspan="3">82.8 71.7 83.6 85.6</td><td>77.0</td></tr></table>

Top-1 and top-5 accuracy of InceptionV3 and Inception-ResNetV2 on adversarial examples. We can see that adaptive filtering significantly increases the robustness of the classifier against many diverse attacks It can be seen that we recover adversarial examples generated by FGSM, PGD, CW and DeepFool near perfectly, while missing nearly 15% of the examples of MIM and L-BFGS. These results are significantly better than the results in

Table content: <table/>

Our model consists of 2 convolutional layers with 32 and 64 filters respectively,

Table content: <table><tr><td/><td/><td colspan="2">FGSM</td><td colspan="2">MI FGSM</td></tr><tr><td>Network</td><td colspan="5">Epsilon Natural BFNet Natural BFNet</td></tr><tr><td>Inception V3</td><td>0.1</td><td>26.8</td><td>30.2</td><td>58.8</td><td>21.0</td></tr><tr><td>Inception V3</td><td>0.15</td><td>21.4</td><td>36.6</td><td>65.6</td><td>30.1</td></tr><tr><td>Inception V3</td><td>0.3</td><td>6.8</td><td>46.6</td><td>88.2</td><td>42.2</td></tr><tr><td>Inception V3</td><td>0.5</td><td>1.0</td><td>63.2</td><td>98.0</td><td>52.4</td></tr><tr><td>Inception V3</td><td>0.75</td><td>0.0</td><td>90.8</td><td>99.6</td><td>53.4</td></tr><tr><td>Inception V3</td><td>1.0</td><td>0.0</td><td>99.8</td><td>99.6</td><td>70.8</td></tr><tr><td>IncResNet V2</td><td>0.1</td><td>41.2</td><td>42.6</td><td>89.1</td><td>59.0</td></tr><tr><td>IncResNet V2</td><td>0.15</td><td>34.4</td><td>55.4</td><td>90.6</td><td>38.6</td></tr><tr><td>IncResNet V2</td><td>0.3</td><td>11.8</td><td>75.4</td><td>92.2</td><td>59.4</td></tr><tr><td>IncResNet V2</td><td>0.5</td><td>2.0</td><td>91.0</td><td>100.0</td><td>74.6</td></tr><tr><td>IncResNet V2</td><td>0.75</td><td>0.4</td><td>99.6</td><td>100.0</td><td>80.2</td></tr><tr><td>IncResNet V2</td><td>1.0</td><td>0.4</td><td>99.6</td><td>100.0</td><td>91.8</td></tr></table>

LEFT: Comparison of our method with state of the art adversarial training results on MNIST. BFNet pgd denotes our model trained against a PGD adversary, while BFNet fgsm is trained against a FGSM adversary. For Tramer We report A: the strongest white box attack given against a non-ensembled model from

Table content: <table><tr><td>Network</td><td colspan="5">Clean FGSM PGD CW CW</td><td/><td/><td/><td/></tr><tr><td/><td/><td/><td/><td/><td>(κ=50)</td><td>Method</td><td colspan="4">Type Clean FGSM PGD</td></tr><tr><td>BFNet pgd</td><td>99.0</td><td>95.5</td><td colspan="3">98.0 93.2 -</td><td>BFNet pgd</td><td>A</td><td>87.1</td><td>55.2</td><td>50.4</td></tr><tr><td colspan="2">BFNet fgsm 99.0</td><td>98.1</td><td colspan="3">36.4 88.2 96.0</td><td>BFNet pgd</td><td>B</td><td>73.1</td><td>64.5</td><td>38.1</td></tr><tr><td>Madry</td><td>98.8</td><td>95.6</td><td colspan="3">93.2 94.0 93.9</td><td>BFNet fgsm</td><td>B</td><td>76.5</td><td>70.6</td><td>12.2</td></tr><tr><td>Tramer A</td><td>98.8</td><td>95.4</td><td colspan="2">96.4 -</td><td>95.7</td><td>Madry</td><td>A</td><td>87.3</td><td>56.1</td><td>45.6</td></tr><tr><td>Tramer B</td><td>98.8</td><td>97.8</td><td>-</td><td>-</td><td>-</td><td/><td/><td/><td/></tr></table>

Peer review (iclr2019):
 The paper proposes the use of bilateral filtering as a defense to adversarial examples (AE). Bilateral Filtering (BF) is a smoothening technique that averages pixels that are close to each other both in position and value. As a result BF preserves sharp edges and the denoised images appear to be of higher quality than those produces via simple Gaussian smoothing. First, the method is evaluated on black-box attacks by constructing AE for the undefended network and then applying the defense. The method is then tested on white-box attacks which attack the defended system end-to-end with first order methods. Finally, the authors propose combining their defense with adversarial training. Overall, I find the main idea of the paper interesting. BF seems like a natural approach to remove low-magnitude perturbations from the image while preserving the salient characteristics of the image. Thus, if it did not hurt classification accuracy significantly it might be considered as part of a cheap preprocessing pipeline that protects the model from simple attacks. However, as shown in Table 3 and 6, BF reduces the accuracy of the original model by at least 5% on ImageNet and CIFAR10. My main objection about the paper is that BF is proposed as a defense that is extremely robust against even the best attacks. However, the evaluation is clearly incomplete for such a claim and fails to pass simple sanity checks. *Epsilon values used to claim robustness are way too high*. Table 4 reports an L_infinity bound of 0.793 (almost 40% the image range) and L_2 of 187 (again roughly a quarter of the L2 radius of the image domain). One can change the class of the image completely (even for humans) with a smaller epsilon bound. This is a very clear indication that the attacks used are not sufficient to properly evaluate the robustness of the defense. Even in the images produced by the authors (Figure 2) it is clear that these perturbations are sufficient in terms of magnitude to completely distort the image. *Only first-order attacks are considered*. From the point about epsilon values above, it is fairly clear that the defense is not easy to attack with first-order methods. This is understandable since BF, even if "fully differentiable", will lead to vanishing gradients for certain pixels. I think it is necessary that the authors to evaluate their defense on other attacks such as a) a straight-through approximation of BF (see BPDA from https://arxiv.org/abs/1802.00420), b) a finite-differences-based attack (see SPSA from https://arxiv.org/abs/1802.05666). As a sanity check, I would even recommend evaluating using the nearest neighbor of each image from a wrong class. It is clear that this attack will always succeed and the L_p distance will be smaller than those reported here. I believe that these are fundamental issues about the paper that need to be addressed before even considering the paper for acceptance and I thus recommend rejection at this time. Other comments and concerns: --Table 6: what is the performance of BFNet without adversarial training? This is a necessary baseline to understand the results of the table. -- I am confused by the adaptation of adversarial training to BFNet (last equation of section 3.4). The equations themselves are confusing since x is used twice (as a parameter to f and sampled from D). My first guess would be that the method is performing end-to-end adversarial training from the loss to the input x (before BF). It appears however that adversarial training is performed with respect to the image *after* applying the BF filter. If this is the case, it goes against the fundamental principles of adversarial training. -- I do not see the point of adaptive filtering (AF) as a defense. BF is attractive because it is simple. Adding a classifier to figure out the right parameters seems dubious at best. The idea seems fundamentally flawed given that the classifier is only trained on known attacks and might thus not capture new, different attacks. Lastly, since the classifier is a ML model, it will clearly be susceptible to adversarial perturbations. So the adversary can choose an attack that fools the original model and forces the classifier to choose bad BF parameters. I would suggest that the authors completely remove AF from future versions of their paper. -- The performance on ImageNet is significantly better than the performance on MNIST and CIFAR10. This is counter-intuitive since a much harder dataset both from a standard and adversarial point of view. Minor comments to authors: -- A JPEG compression-based defense can be bypassed (https://machine-learning-and-security.github.io/papers/mlsec17_paper_54.pdf). -- Last paragraph of page 3 is too informal ("less holes for adversarials to maneuver on"). -- Top of page 4: you seem to confuse "iterative methods" with "small magnitude" attacks. One can use iterative attacks to create large adversarial perturbations (e.g. PGD). -- I would suggest normalizing to the more common [0, 1] range as the current numbers can be confusing. -- I am confused by Tables 2, 3. First row of Table 2 shows that on 95% of the clean inputs the label is not changed after BF. But table 3 shows that the accuracy drop from 78.8% to 71.7%. What is going on here? -- Last part of first paragraph of 4.2 is hard to understand. -- End of page 8, "the higher dimensionality of CIFAR10 makes it harder than MNIST". Yet the paper claims excellent performance on ImageNet.