Training on Foveated Images Improves Robustness to Adversarial Attacks
Authors: Muhammad A Shah
Aqsa Kashaf
Bhiksha Raj
Venue: NeurIPS 2023 poster (2023)
Keywords: adversarial robustness
computer vision
biologically-inspired
retina
blurring

Abstract: Deep neural networks (DNNs) have been shown to be vulnerable to adversarial attacks
-- subtle,  perceptually indistinguishable perturbations of inputs that change the response of the model. In the context of vision, we hypothesize that an important contributor to the robustness of human visual perception is constant exposure to low-fidelity visual stimuli in our peripheral vision. To investigate this hypothesis, we develop RBlur, an image transform that simulates the loss in fidelity of peripheral vision by blurring the image and reducing its color saturation based on the distance from a given fixation point. We show that compared to DNNs trained on the original images, DNNs trained on images transformed by RBlur are substantially more robust to adversarial attacks, as well as other, non-adversarial, corruptions, achieving up to 25% higher accuracy on perturbed data.

Training on Foveated Images Improves Robustness to Adversarial Attacks
Muhammad A. Shah
Aqsa Kashaf ByteDance San Jose, CA 95110 akashaf@cmu edu
Bhiksha Raj
Language Technologies Institute Carnegie Mellon University Pittsburgh, PA 15213 mshah1 hemhhh1@mmmm.mmm edu
Language Technologies Institute Carnegie Mellon University Pittshurgh, PA 15213 bhiksha@cs. cmu edu
Abstract
Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks subtle, perceptually indistinguishable perturbations of inputs that change the response of the model. We hypothesize that an important contributor to the robust- ness of human visual perception is constant exposure low-fidelity visual stimuli in our peripheral vision. To investigate this hypothesis, we develop R-Blur, an image transform that simulates the loss fidelity of peripheral vision by blurring the image and reducing its color saturation based on the distance from a given fixation point. We show that compared DNNs trained on the original images, DNNs trained on images transformed by R-Blur are substantially more robust to adversarial attacks, as well as other, adverrrrrsaiial, corruptions, achieving up 25% higher accuracy perturbed data?
Introduction
Deep Neural Networks (DNNs) are exceptionally adept at many computer vision tasks and have emerged as of the best models of the biological neurons involved in visual object recognition [1.2]. However, their lack of robustness to subtle image perturbations that humans are largely invariant 345 to has raised questions about their reliability in real-world scenarios. Of these perturbations, perhaps the most alarming are adversarial attacks, which are specially crafted distortions that can change the response of DNNs when added to their inputs 3 6 but either imperceptible humans or perceptually irrelevant enough to be ignored by them.
While several defenses have been proposed over the years to defend DNNs against adversarial attacks, only of them have sought inspiration from biological perception, which, perhaps axiomatically, is one of the most robust perceptual systems in existence. Instead, most methods seek to teach DNNs to be robust to adversarial attacks by exposing them to adversarially perturbed images 7 8 9 or random noise [10 [12 during training. While this approach is highly effective in making DNNs robust to the types of perturbations used during training, the robustness often does not generalize to In contrast,
other types of perturbations [13
defenses seek to make DNNs robust by integrating into them biological mechanisms that would bring their behavior more in line with human/animal vision [6
14
13
22 As these defenses do not require DNNs to be trained on any particular type of perturbation, they yield models that, like humans, are robust to variety of perturbations [18] in addition adversarial attacks. For this reason, and in light of the evidence indicating a positive correlation between biological alignment and adversarial robustness believe biologically inspired defenses are more promising in the long run.
18
[9 20 21
18
23
work done while Carnegie Mellon University
'The code for R-Blur is available http:://tttabbccom////cchhaa1112
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

10,9
633
10.4
104
88.5
9.9
10.0
3.9
9.3
(4)
(bl
(c)
(e)
Figure 1: R-Blur adds Gaussian noise to image (a) with the fixation point (red dot) to obtain (b). then creates colored and grayscaled copy of the image and applies adaptive Gaussian blurring to them to obtain the low-fidelity images (c) and (d), where the numbers indicate the standard deviation of the Gaussian kernel applied in the region bounded by the boxes. The blurred color and gray images are combined in a pixel-wise weighted combination to obtain the final image (e), where the weights of the colored and gray pixels a function of their respective estimated acuity values (see 2.2
Following this line of inquiry, we investigate the contribution of low-fidelity visual sensing that occurs in peripheral vision to the robustness of human/animal vision. Unlike DNNs, which sense visual stimuli at maximum fidelity at every point in their visual field, humans sense most of their visual field in low fidelity, i.e without fine-grained contrast (24) and color information [25]. In adults with fully developed vision, only a small region (less than 1% by area) of the visual field around the point of fixation (26) can be sensed with high fidelity. In the remainder of the visual field (the periphery). the fidelity of the sensed stimuli decreases exponentially with distance from the fixation point (27]. This phenomenon is called "foveation". Despite this limitation, humans accurately categorize objects that appear in the visual periphery into high-level classes [28] Meanwhile, the presence of a small amount of noise or blurring can decimate the accuracy of otherwise accurate DNN. Therefore, we hypothesize that the experience of viewing the world at multiple levels of fidelity, perhaps even the same instant, causes human vision to be invariant low-level features, such textures, and high-frequency patterns, that can be exploited by adversarial attacks.
In this paper, we propose R-Blur (short for Retina Blur), which simulates foveation by blurring the image and reducing its color saturation adaptively based on the distance from given fixation point. This causes regions further away from the fixation point to appear more blurry and less vividly colored than those closer to it. Although adaptive blurring methods have been proposed as computational approximations of foveation 29 30 31), their impact robustness has not been evaluated to the best of our knoweedge. Furthermore, color sensitivity is known to decrease in the periphery of the yet most of the existing techniques do not account for this phenomenon.
visual field 25 32 33]
Similar to how the retina preprocesses the visual stimuli before reaches the visual cortex, we use R-Blur to preprocess the input before it reaches the DNN. To measure the impact of R-Blur, we evaluate the object recognition capability of ResNets 34 trained with and without R-Blur on three image datasets: CIFAR-10 [B5 . Ecoset (36] and Imagenet (37]. under different levels of adversarial attacks and common image corruptions [3). We find that R-Blur models retain most of the high classification accuracy of the base ResNet while being more robust. Compared to the base ResNet, R-Blur models achieve 12-25 percentage points (pp) higher accuracy on perturbed images. Furthermore, the robustness achieved by R-Blur is certifiable using the approach from (10). We also compare R-Blur with two biologically inspired preprocessing defenses, namely VOneBlock [18. fixed parameter module that simulates the primate V1, and non-uniform sampling -based foveation technique (22), which we refer to as R- R-Warp. We observe that R-Blur induces a higher level of robustness, achieving accuracy up 33 pp higher than R-Warp and up to 15 pp higher than VOneBlock against adversarial attacks. Compared to adversarial training (AT) 7 g the state-of-the-ant art non- -ioooooogia defense, R-Blur achieves up to 7 pp higher accuracy on average against non-adversarial corruptions of various types and strengths thus indicating that the robustness of R- Blur generalizes better to non-adversarial perturbations than AT. Finally, an ablation study showed that both adaptive blurring and desaturation contribute to the improved robustness of R-Blur.
2 Retinal Blur: An Approximation for Peripheral Vision
To simulate the loss in contrast and color sensitivity of human perception with increasing eccentricity, we propose R-Blur, adaptive Gaussian blurring, and color desaturation technique. The operations

---

performed by R- R-Blur, given an image and fixation point, are shown in Figure 1 First, R-Blur adds Gaussian noise to the image to simulate stochastic firing rates of biological photoreceptors 39 then creates color and grayscale copies of the image and estimates the acuity of color and grayscale vision at each pixel location, using distributions that approximate the relationship between distance from the fixation point (eccentricity) and visual acuity levels in humans. R-Blur then applies adaptive Gaussian blurring to both image copies such that the standard deviation of the Gaussian kernel at each pixel in the color and the grayscale image is a function of the estimated color and grayscale acuity at that pixel. Finally, R-Blur combines the two blurred images in a pixel-wise weighted combination in which the weights of the colored and gray pixels are function of their respective estimated acuity values. Below we describe some of the more involved operations in detail.
2.1 Eccentricity Computation
The distance of a pixel location from the fixation point, i.e. its eccentricity, determines the standard deviation of the Gaussian kemnel applied to it and the combination weight of the color and gray images at this location. While eccentricity is typically measured radially, in this paper we use different distance metric that produces un-rotated square level sets. This property allows us to efficiently extract regions having the same eccentricity by simply slicing the image tensor. Concretely, we compute the eccentricity of the pixel location (Kp: as
max(1xp
xyl. Up Vfl) Wy
(1) where (xy, ys) and Wr represent the fixation point and the width of the visual field, i.e. the rectangular region over which R-Blur operates and defines the maximum image size that is expected by R-Blur. We normalize by Wy to make the Cr,.D, invariant to the size of the visual field.
Cxp.Vr
2.2 Visual Acuity Estimation
We compute the visual acuity at each pixel location based on its eccentricity. The biological retina contains two types of photoreceptors. The first type, called cones, are color sensitive and give rise to high-fidelity visual perception at the fovea, while the second type, called rods, are sensitive to only illumination but not color and give rise low-fidelity vision in the periphery. We devise the following two sampling distributions, DR(ez.)) and (e to model the acuity of color and grayscale vision, arising from the cones and rods at each pixel location, (xr
D(e; o, a) = max [M(e; (,);;,,), o), ((e;0,
(2)
Dele;ac,a) D(e; ac,
DR(e o, amaan) = Pmna (1 D(e;(R)))))),
where X(:; μt, o) and 7/: μ. σ) are the PDFs of the Laplace and Cauchy distribution with location and scale parameters μ and σ,, and a is parameter used to control the width of the distribution. We set ac 0.12, on 0.09, a 2.5 and Pinax 0.. We choose the above equations and their parameters to approximate the curves of photopic and scotopic visual acuity from [27]. The resulting acuity estimates are shown in Figure 2b. Unfortunately. the measured photopic and scotopic acuity
curves from 27 cannot be reproduced here due to copyright reasons, however, they can be viewed at 14. html (see Figure 14.3).
2.3 Quantizing the Visual Acuity Estimate
In the form stated above, we would need to create and apply as many Gaussian kernels as the distance between the fixation point and the farthest vertex of the visual field. This number can be quite large as the size of the image increases and will drastically increase the per-image computation time. To mitigate this issue we quantize the estimated acuity values. As result, the locations to which the same kernel is applied no longer constitute a single pixel perimeter but become a much wider region (see Figure [ (c) and (d)), which allows to apply the Gaussian kernel in these regions very efficiently using optimized implementations of the convolution operator.
To create a quantized eccentricity-acne accitty mapping, we do the following. We first list all the color and gray acuity values possible in the visual field by assuming a fixation point at (0, 0), computing eccentricity values e0,y for y E [o, Wy] and the corresponding values of DR (DR(eo,w) y e
3

---

Adiley
Acuiyy
vd = 0 vd =1 vd =
vd =3
vd = vd 5
(a) unquantized
(b) quantized
Figure 3: Illustration of increasing the viewing distance (left to right). As the viewing distance is increased, more of the image is brought into focus. We used vd during inference.
Figure 2: Estimated visual acuity of sharp and colorful, photopic, and gray and blurry, scotopic, vision us- ing equations and 4
[0, Wy]] and De (Do(eo.))ly E [0, Wy]]. We then compute and store the histograms, HR and He. from DR and De, respectively. To further reduce the number of kernels we need to apply and increase the size of the region each of them is applied to, we merge the bins containing less than elements in each histogram with the adjacent bin to their left. After that, given an image to process, we will compute the color and gray visual acuity for each pixel, determine which bin falls HR and He, and assign it the average value of that bin.
2.4 Changing the Viewing Distance
Increasing the viewing distance can be beneficial allows the viewer gather a more global view of the visual scene and facilitates object recognition. To increase the viewing distance we drop the k lowest acuity bins and shift the pixels assigned to them & bins ahead such that the pixels that were in bins through k are now assigned to bin 1. Figure shows the change in the viewing distance as the value of l increases from 5. Formally, given D [d1,.., d.,] represent the value assigned to each bin and P: be the pixel locations assigned to the tt bin, with P1 and Pa corresponding to points with the lowest and highest eccentricity, respectively, To increase the viewing distance, we merge bins 1 through such that D' [d1, ..., dn- k) and the corresponding pixels are P/ PP..... Pa. and Pi>1 = Pk+1-
quantized De(ez.g) and DR(ea.y) let
2.5 Blurring and Color Desaturation
We map the estimated visual acuity at each pixel location, (Ip, Up). the standard deviation of the Gaussian kernel that will be applied at that location as (ap v1 D(ez.g)), where / is constant to control the standard deviation and is set to = 0.05 this paper, and pixels in the colored image and D = DR for pixels in the grayscaled image. We then apply Gaussian kernels of the corresponding standard deviation to each pixel in the colored and grayscale image to obtain an adaptively blurred copy of each, which we combine in pixel-wise weighted combination to obtain the final image. The weight of each colored and gray pixel given by the normalized color and gray acuity, respectively, at that pixel. Formally, the pixel at (Ips Up) in the final image has value
De for
D. ((r.-t)))) Delex, FC, oC, a)) + (lpp. Du Du(ezyiocca0
(5)
(zp)))))
Delen oe, ox) + Dn(eagi 00, a.)
e ((ap.1p) and v/ ((p))))))
are the pixel value (Irps Up) in the blurred color and gray images respectively.
3 Evaluation
In this section, we determine the accuracy and robustness of R- R-Blur by evaluating it on clean data and data that has been perturbed by either adversarial attacks or common aon-saderslrial corruptions. We compare the performance of R Blur with an unmodified ResNet, two existing biologically-inspin defenses, R- Warp 22 and VOneBlock [18), and a biollogiaa adversarial defenses: Adversarial Training (AT) 7 We show that R-Blur is significantly more robust to adversarial attacks and
common corruptions than the unmodified ResNet and prior biologically inspired methods. Moreover,
use Randomize Smoothing [10 show that R-Blur is provably robust. While AT is more robust

---

6
4
%
(a) R-Blur
(b) R- Warp
Figure 4: Illustration of fixation selection. The initial fixation point is set to top-left (0,0) and the image at ty is processed with R-Blur /R- R-Warp to get the image at 41 DeepGaze- is used to generate a fixation heatmap from this image. The next fixation point is sampled from the heat map, and R-Blur /R- Warp is applied to get the image at tz. The region in the heatmap around the chosen fixation point is masked with an inverted Gaussian kernel to prevent spatial clustering of fixation points. This process is repeated to get sequence of fixation points.
than R-Blur against adversarial attacks, R-Blur is more robust than AT against common corruptions, thus indicating that the robustness of R-Blur generalizes better to different types of perturbation than AT. We also analyze the contribution of the various components of R-Blur in improving robustness.
3.1 Experimental Setup
35 Imagenet ILSVRC 2012 (37, Ecoset [36] and a 10-class subset of Ecoset (Ecoset- 10). Ecoset contains around 1.4M images, mostly obtained from ImageNet database [40] (not the ILSVRC dataset), that are organized into 565 basic object classes. The classes in Ecoset correspond to commonly used nouns that refer to concrete objects. To create Ecoset-10, we select 10 classes from Ecoset that have the highest number of images. The training/validatioonnest splits of Ecoset- 10 and Ecoset are 48K/859/1 K, and 1 1.4M/28K/28K respectively. For most experiments with Ecoset and Imagenet, we use 1 1130, and 2000 test images, with equal number of images per class. During training, we use random horizontal flipping and padding + random cropping, as well as AutoAugment 41 for CIFAR-10 and RandAugmen for Ecoset and Imagenet. All Ecoset and Imagenet images were resized and cropped to 224 224. We applied these augmentations to all the models we trained - those with biological and -onobiological defenses, well as the baseline models.
Datasets: We use natural image datasets, namely CIFAR-10
model with 22 convolutional layers For CIFAR- 10 we use a Wide-Resnet and widening factor of 4, and for Ecoset and Imagenet we use XResNet-18 from fastai 4 with a widening factor of 2. Moving forward, we will refer to both these models as ResNet and indicate only the training/evaluation datasets from which the exact architecture may be inferred. Results for additional architectures are presented Appendix C
Model Architectures:
We compare the performance of R-Blur to two baselines: (1) an unmodified ResNet trained on clean data (ResNet), and (2) a ResNet which applies five affine transformations to the input image and averages the logits (RandAffine). We also compare R-Blur with two biologically inspired defenses: VOneBlock pre- pre-processing proposed in [8] which simulates the receptive fields and activations of the primate V1 V and R- R-Warp preprocessing proposed in [22.. which simulates foveation by resampling input images such that the sampling density of pixels is maximal at the point of fixation and decays progressively in regions further away from it. Finally, we compare R- B-Blur with two non-biological adversarial defenses: fast adversarial training (8] with || = 0.008 (AT), and Randomized Smoothing (RS) [10].
Baselines and Existing Methods:
Fixation Selection for B-Blur and R- Warp: While training models with R-Blur and R-Warp, we split each batch into sub-batches of 32 images, and for each bub-batch, we randomly sample single fixation point that we use to apply R- R-Blur or Warp all the images in that sub-batch While training the R- Blur model, we also set the viewing distance uniformly at random using the procedure
We apply rotation, translation. and shearing, with their parameters sampled from [-8.6% 8.6%), [-49, 49) and (88.6", 8.6"] respectively. The ranges are chosen to match the ranges used in RandAugment. The random seed is fixed during evaluation to prevent interference with adversarial attack generation.
As in [18). remove the first cony, batch norm, ReLU, and MaxPool from the ResNet with VOneBlock.
5

---

103
100
72
0.0
0
03
3.0
2.5
Pertaraation Dietencw
Fetturbetion Distance |+
Pwturbetion Distanow
Pwturbelien Destence
8
Accurvo
Accursce
6 5
50
35
12
0.50
2/03a
0.502
0.024
0,010
0.60
D 1055
0.028
0.002
0.0c6
Distance I+]-
Distance [+]-
Perturtation Distance [6]-
Distance [4]-
(a) CIFAR-10
(b) Ecoset-10
(c) Ecoset
(d) Imagenet
Figure 5: Comparison of accuracy on various datasets (a-d) under adversarial attacks of several & (top) and &o (bottom) norms between R-Blur (green) and two baseline methods: RandAffine (orange) and ResNet (blue). The dashed lines indicate accuracy on clean images. R- R-Blur models consistently achieve higher accuracy than baseline methods on all datasets, and adversarial perturbation sizes.
described in 2.4 During inference, we determine a sequence of five fixation points (a scanpath) using DeepGaze- III DeepGaze-IlI passes the input image through pretrained CNN backbone (DenseNet-201 in (44) and extracts the activations from several intermediate layers of the CNN. It then applies a sequence of pointwise convolution and normalization layers the activations obtain a heatmap indicating where a human is likely to fixate. We found that it was more efficient to not use the scanpath prediction module in DeepGaze-III, and instead obtain scanpaths by keeping track of the past fixation points, and masking the predicted heatmap at these locations prior to sampling the next fixation point from it. This process illustrated in Figure
We trained two instances of DeepGaze- IIII using the ResNets we trained with R-Blur and R- R-Warp as the CNN backbone. We use the corresponding DeepGaze- models to predict the scanpaths for R-Blur and R- Warp models. To train deepgaze-ii we used the code from the official github repository [41]. The only significant modification we made was replace the pretrained DenseNet-201 with the pretrained R- R-Warp/R-BBi performance, while keeping the total number of parameters low. Following [41] we train DeepGaze on the SALICON dataset [45). This corresponds to phase of training mentioned in Table I of [41]. We did not notice any benefits in our use case of phases 2-4, so we skipped them.
augmented XResNet-18 18 we trained on ImageNet. This improves
3.2 Results
R-Blur improves robustness to white-box attacks. We evaluate robustness by measuring the accuracy of models under Auto-PGD (APGD) attack, which a state-of-the-art white-box adversarial attack. We run APGD for 25 steps each image. We find that increasing the number of steps beyond 25 only minimally reduces accuracy (Appendix(A) AA). We take a number of measures to avoid the pitfalls of gradient obfuscation 47 48 that results reflect the true robustness of R-Blur. These steps and detailed settings used for adversarial attacks are mentioned in Appendix A
To determine if R- B-Biur improves robustness, we compare R-Blur with the unmodified ResNet and RandAffine under the APGD attack. We observe that R- B-Blur is significantly more robust than the unmodified ResNet and RandAffine models, consistently achieving higher accuracy than the two on all datasets and against all perturbation types and sizes, while largely retaining accuracy on clean data (Figure 5). While RandAffine does induce some level of robustness, it significantly underperforms R-Blur On smaller datasets, R-Blur suffers relatively little loss in accuracy at small to moderate levels (1 0.004, ||ô1|a < 1) of adversarial perturbations, while the accuracy of bascline methods quickly deteriorates to chance or worse. On larger datasets Ecoset and Imagenet, even the smallest amount of adversarial perturbation (|,² = 0.002, ||ö1|2 0.5) is enough to drive the accuracy of the baselines to 10%, while R-Blur still is able to achieve 35-44% accuracy. As the perturbation is increased to ||ō| oc 0.004 and |öl|2 1.0, the accuracy of the baselines goes to 0%, while R-Blur achieves 18-22%. We do observe that the accuracy of R-Blur on clean data from Ecoset and Imagenet is noticeably lower than that of the baseline methods.
We also compare R-Blur to two existing biologically motivated adversarial defenses: VOneBlock and R- Rarrp and find that B-ulur achieves higher accuracy than both of them at all perturbation sizes and

---

1.22
122
127
78
+26
-15
16
-12
+6
.17
-11
+2
+18
2.0
20
0.003
2.0
5.004
0,000
Perturbetion Dicrance
Perturbaltin Didtnnee
Petturbwtion Dicrance I+j?
Perturistion Distance
Ecoset
(b) Imagenet
Figure 6: The difference in accuracy under adversarial attacks of several ES and foe norms between R-Blur and two biologically inspired defenses: R-Warp (blue) and VOneBlock (orange). R-Blur con- sistently achieves higher accuracy on all adversarial perturbation sizes than R- R-Warp and VOneBlock.
Imagenst
0.5
Zowet
0.5
0.6
0.4
0.4
0.4
0.3
0.3
0.2
0.2
C-Mels
Aike 5P1 io. EL
0.0
Goite (A 6.125)
0.1
0.4
022
0.3
0.5
0.0
0.4
1.0
0.2
0.6
radius
(a) Matched Setting (ot = de = 0.125)
(b) Unmatched Setting (d+ < d. = 0.25)
The certified accuracy at various fo-m radii of R- R-Blur and G-Noise models. R-Blur-CF uses fixation at the center of the image, and R-Blur-5FI, averages logits from 5 fixation (corners + center). or denotes the scale noise added during training and is 0. 1.125 unless specified, whereas de is the scale of the noise used to compute the certified accuracy. G-Noise outperforms R-Blur in the matched scenario, while R- R-Blur is superior in the unmatched scenario indicating that the robustness of R-Blur is more generalizable.
Figure 7:
types. From Figure we see that R-Blur achieves up to 33pp higher accuracy than R- R-Warp, and up to 15 pp higher accuracy than VOneBlock on adversarially perturbed data.
R-Blur is certifiably robust. To verify that the gains in robustness observed above are indeed reliable. we use the certification method (CERTIFY) from [IO] provide formal robustness guarantees for R-Blur. This entails obtaining predictions for an input under a very large number (10") of noise samples drawn from (O0 oz), and using hypothesis test to determine the certified radius around the input in which the model's prediction is stable with high probability (2 99.9%). Given dataset, we can compute the certified accuracy at a radius r as the proportion of data points for which the certified radius is 2 r and the model's 's prediction is correct. It was shown in [I0] that model trained on data perturbed with Gaussian noise achieves high certified accuracy. We call this model G-Noise. We compare the certified accuracy of G-Noise and R-Blur on 200 images from Imagenet and Ecoset.
We expose both R-Blur and G-Noise to Gaussian noise of scale o, = 0.125 during training and compute their certified accuracy at radii r E {0.5, 1.0]. According to [00], if the scale of the noise used in CERTIFY is dee then the maximum radius for which certified accuracy can be computed (with 10s noise samples) is = 40.. Therefore, when computing certified accuracy at r S 0.5 CERTIFY adds noise of the same scale as was used during training (a.- = 0.125 = ot), thus we call this the matched setting. However, to compute certified accuracy at r < 1.0 CERTIFY adds noise of larger scale than was used during training (a,- = 0.25 > ot), and thus in order to achieve high certified accuracy at < 1.0 the model must be able to generalize to change in noise distribution. We call this the unmatched setting.
Figure 7a and 7b show the certified accuracy of R-Blur and the G-Noise on Ecoset and Imagenet at several & norm radii under matched and unmatched settings. In both settings, we see that R-Blur achieves a high certified accuracy on both Ecoset and Imagenet, with the certified accuracy at r R 0.5 and r s 1.0 being close to the ones observed in Figure [ indicating that our earlier results are faithful representation of R-Blur's robustness. Furthermore, we see that even if R- R-Blur was trained without any noise, it can still achieve more than 50% of the certified accuracy achieved by R- R-Blur trained with noise. This indicates that adaptive blurring and desaturation do in fact endow the model with a significant level of robustness. Finally, we note that while G-Noise has (slightly) higher certified accuracy than R-Blur in the matched setting, R-Blur achieves significantly higher certified accuracy in the unmatched setting, outstripping G-Noise by more than 10 pp at r R 1.0 on Imagenet. This shows that the robustness of R- R-Blur generalizes beyond the training conditions, while G-Noise
7

---

60
Nusted 5-Handrttie
M-lars Winellac
50
40
80
20
10
Corruption Severity
Corruption Severity
(a) Imagenet
(b) Ecoset
Figure 8: The accuracy of the models on Imagenet and Ecoset under the common corruptions from
38
at various severity levels. We see that R-Blur generally achieves the highest accuracy.
Method
Mean
CC
WB
Clean
Mean
CC
Wb
Clean
Ecoset
Imagenet
ResNet
37.1
39.4
0.8
71.2
34.7
33.6
0.1
70.3
RandAffine
35.7
35.8
3.6
67.6
33.7
30.8
2.0
68.3
AT
49.0
38.5
47.5
61.1
46.3
34.2
43.5
61.3
R- R-Warp
38.5
40.0
4.5
71.1
34.1
32.5
2.2
67.7
VOneBlock
42.9
40.7
16.1
72.0
38.8
35.8
11.9
68.7
R-Blur
44.2
45.6
23.8
63.3
38.9
39.0
17.2
60.5
best, second best Table 1: Accuracy of the evaluated models on clean, and perturbed data from Imagenet. "WB"
refers to the accuracy under APGD attacks, while "CC" refers the accuracy under common adversarial corruption [38]. R-Blur significantly improves the robustness of ResNet, and outperforms prior biologically motivated defenses, while approaching the performance of AT.
overfits them. This makes R-Blur particularly suited for settings in which the exact adversarial attack budget is not known, and the model must be able to generalize.
R-Blur Improves accuracy on common anvrsarvesariial) corruptions. Adversarial perturbations constitute only a small subset of perturbations that human vision is invariant to, therefore we evaluate R-Blur set of common image corruptions [38] that humans are largely invariant to but DNNs
not. We sample 2 images/class from Imagenet and 5 images/class from Ecoset. Then we apply 17 common corruptions proposed in (38) at different severity levels generate 85 corrupted versions of each image. This yields corrupted versions of Imagenet and Ecoset containing 170K and 240K images, respectively.
Figure 8 shows the accuracy the models on corrupted Ecoset and Imagenet. Here we also compare against an adversarially trained model (AT) trained with || = 0.008 using the method of [8] We see that at severity greater than R-Blur consistently achieves the highest accuracy. Furthermore, we also note that R-Blur, and VOneBlock consistently achieve higher accuracy than AT, which supports our hypothesis that the robustness of biologically motivated methods, and particularly R-Blur, is more general than non-biological defenses, like AT. In fact, the accuracy of AT common corruptions is generally lesser than or at par with the accuracy of the unmodified ResNet, indicating that the robustness of AT does not generalize well.
Summary Results: Table summarizes the results of our paper and reiterates two key observa- tions from earlier sections. Firstly, R-Blur makes models more significantly robust to adversarial perturbations than the unmodified ResNet, and other biologically inspired defenses. R-Blur, however, achieves lower accuracy against white-box attacks than AT. This is to be expected because AT is trained on adversarially perturbed data. Secondly, R-Blur augmented models are significantly more robust to common corruptions than all other models, including AT. I. In contrast, the accuracy of AT on common corruptions is almost the same as that of the unmodified ResNet, indicating that the robustness of AT does not generalize.
'ee exclude Gaussian blur and Gaussian noise since they are similar to the transformations done R-Blur.
8

---

Figure 9: Accuracy on clean and APGD (| = 0.008) perturbed Ecoset-10 images when individual components of R-Blur are removed (left), and when non-adaptive trans- forms are used (right). Removing the biologically-motivaeed compo- nents of R-Blur harms the robustness
Everything
Everything
244
39
x +Dymamic Fixation
Non
Blur Noise
Desaturation
Multiple
68
Noise
Fixations
Non-Adaptive
Adaptiee
Noise
Desaturation
40
00
40
60
Accuracy
Accuracy
G-Blunjo 10.5)
R-kur
NNo Neise)
100
Noiseto 0.125)
100
80
Accuracy
80
Accuracy
80
Accuracy
60
60
40
3435
40
3035
20
20
7
10
0
0
20
1
17
0
0
0.002
0,004
0.008
0.5
1.0
2.0
0.004
0.008
Perturbation Distance |||||
Perturbation Distance |e|]2
Perturbation Distance |c|-
(a)
(b)
Figure 10: Comparison of R-Blur with models trained with non-adaptive Gaussian blur or Gaussian noise augmentations. (a) compares the accuracy under adversarial attack on Ecoset-10 10 of R-Blur and models augmented with non-adaptive Gaussian blur of various standard deviations (o). While non-adaptive Gaussian blur does increase robustness, the adaptive blurring in R- R-Blur outperforms by margin. (b) compares the accuracy under adversarial attack on Imagenet of R-Blur and models augmented with either non-adaptive Gaussian blur or Gaussian noise. We see that R-Blur achieves better robustness than either of these methods.
3.3 Ablation Study
We examine, by way of ablation, how much each component of R-Blur contributes towards its overall robustness as shown in Figure 9 The most significant contributor to robustness is the addition of noise, This echoes the findings from [18). which showed that neural stochasticity contributes significantly to the robustness of the visual system. Nevertheless, even without noise R-Blur achieves an 11 point improvement over the vanilla ResNet which archives 0% accuracy under the attack, which indicates that other components of R-Blur also contribute towards robustness. Furthermore, experimental results reveal that robustness induced by noise diminishes as the complexity of the dataset increases and the size of the perturbations increases. As observed Figure [0b] Gaussian noise augmentation achieves 45-58% (8-10 10 points) lower accuracy than R-Blur, and in Figure 7b. which shows that at larger perturbation sizes R-Blur achieves higher certified accuracy.
The second most significant contributor to robustness is the blurring performed by R-Blur. Importantly, we note that Gaussian blurring in and of itself does not greatly improve robustness. Figure 9 shows that non-adaptive blurring with a single Gaussian kernel having a = 10.5 (o 10.9 is the maximum used in R- R-Blur) improves robustness by only points. Furthermore, Figure 10a shows that increasing the strength of ada-avae blurring trades off clean accuracy for robustness. However, after = 8 the gains in robustness hit (a rather low) ceiling, and increasing a further reduces both clean accuracy and robustness. On the other hand, R-Blur, without any additive noise, achieves similar clean accuracy as non-adaptive blurring (o = 10.5) but achieves significantly better adversarial robustness, thereby demonstrating that Gaussian blurring alone accounts for only a fraction of R R-Blur's robustness. Furthermore, Figure 10b shows that the contribution of non-adaptive blurring declines on the more complex Imagenet, where it achieves only 1 1% accuracy moderate-sized perturbations.
The next most significant factor, after noise and adaptive blurring, is evaluating multiple fixation points which improved robustness significantly compared to single fixation point in the center of the image, which suggests that, multiple fixations and saccades are important when the image is hard to recognize and presents a promising direction for future work. Furthermore, not adaptively desaturating the colors reduces the robustness slightly. Finally, we note that dynamic fixation does not improve performance compared to 5 predefined fixation points. To summarize, most of the biologically-motive components of R-Blur contribute towards improving the adversarial robustness of object recognition DNNs from close to 0% to 45% (l.c 0.008 for Ecoset-10).

---

4 Related Work
Non-biological defenses: Perhaps the most successful class of adversarial defenses are adversarial
training algorithms 9 49 17 [], which train models adversarially perturbed data generated by backpropagating gradients from the loss to the input during each training step. Another popular class of defenses is certified defenses [10] [I [1] which are accompanied by provable guarantees of the form: with probability 1 - 6, the model's 's output will not change a given image is perturbed at most Perhaps, most closely related to our work are preprocessing defenses [52 533 that apply large number of transforms to the input during inference. Usually, these defenses rely on non-differentiable transformations, and high degree of randomization in the number, sequence, and parameters of the transforms they apply to each image. Therefore, these defenses tend to obfuscate gradients [47], and have been shown to be compromised by attacks with higher step budget. We would like to point out that R-Blur does not have these aforementioned pitfalls - the transforms that R-Blur applies (Gaussian blur and desaturation) are fully differentiable and totally deterministic. In general, it is our opinion that by not being cognizant of the biological basis of robust vision, current approaches are excluding a large set of potentially effective approaches for defending against adversarial attacks.
Biologically inspired defenses: Several biological defenses have been proposed over the years. These defenses involve integrating computational analogues of biological processes that absent from common DNNs, such as predictive/sparse coding [16] 17), biologically constrained visual filters, nonlinearities, and stochasticity (18), foveation [99 20. 21 222, into DNNs. The resulting models are made more robust to adversarially perturbed data, and have been shown to better approximate the
responses of biological neurons [8].
Most relevant to our work are defenses that have integrated foveation with DNNs. One of the earliest works (20) implements foveation by cropping the salient region of the image at inference time. This work has several shortcomings. Firstly, the biological plausibility of this method is questionable because it does not simulate the degradation of visual acuity in the periphery of the visual field, rather it discards the periphery entirely. Secondly, it crops the image after applying the adversarial attack, which means that the attack does not take into account the cropping, which is akin to obfuscating the gradients, and hence any reported improvements in robustness are suspect. A later work [22] (R- Warp) avoids the aforementioned pitfalls and simulates foveation via non-uniform sampling (regions further away from the fixation points are sampled less densely). Since this method is fully differentiable and highly biologically plausible, we compare against it in this paper. Some recent works 19 21 apply foveation in the latent feature space (the intermediate feature maps generated by a CNN). These works implement foveation by changing the receptive field sizes of the convolutional kernels based on the distance to the fixation. Since they operate on the latent feature space, rather than image pixels, their methods not directly comparable to ours.
5 Limitations
Adding R-Blur reduces accuracy on clean data, however, it is possible to significantly improve the accuracy of R-Blur by developing better methods for selecting the fixation point. Further experimental results presented in Appendix [ show that if the optimal fixation point was chosen by an oracle the clean accuracy of R-Blur can be improved to within 2% of the accuracy of the unmodified ResNet.
Conclusion
Since the existence of adversarial attacks presents a divergence between DNNs and humans, we ask if some aspect of human vision fundamental to its robustness that not modeled by DNNs. To this end, we propose R-Blur, a foveation technique that blurs the input image and reduces its color saturation adaptively based on the distance from a given fixation point. We evaluate R-Blur and other baseline models against APGD attacks on two datasets containing real-world images. R- Bluu outperforms other biologically inspired defenses. Furthermore, R-Blur also significantly improves robustness to common, non-adversarial corruptions and achieves accuracy greater than that of adversarial training. The robustness achieved by R-Blur is certifialle using the approach from [10] and the certified accuracy achieved by R-Blur is at par or better than that achieved by randomized smoothing [00. Our work provides fürther evidence that biologically inspired techniques can improve the accuracy and robustness of Al models.
10

---

References
[1] Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. Performance-opimizee hierarchical models predict neural responses higher visual cortex. Proceedings of the national academy of sciences, 11 111(2))8669-886) 2014.
[2] Charles Cadieu, Ha Hong. Daniel LK Yamins, Nicolas Pinto, Diego Ardila, Ethan Solomon, Najib J Majaj, and James DiCarlo. Deep neural networks rival the representation of primate it cortex for core visual object recognition. PLoS computational biology, 10(12)ee000363,, 2014.
[3] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
[4] Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wichmann. Generalisation in humans and deep neural networks. Advances in neural information processing systems, 31, 2018.
[5] Samuel Dodge and Lina Karam. A study and comparison of human and deep learning recogni- tion performance under visual distortions. In 2017 26th international conference on computer communication and networks (ICCCN), pages 1-7. IEEE, 2017.
[6] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32, 2019.
[7] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.
[8] Eric Wong, Leslie Rice, and J Zico Kolter. Fast better than free: Revisiting adversarial training. In International Conference Learning Representations, 2019.
[9] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pages 7472-7482. PMLR, 2019.
[10] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing, In International Conference on Machine Learning, pages 1310-1320. PMLR, 2019.
[11] Mare Fischer, Maximilian Baader, and Martin Vechev. Certified defense image transfor- mations via randomized smoothing. Advances in Neural Information Processing Systems, 33:8404 1484404-417,, 2020.
[12] Nicholas Carlini, Florian Tramer, J Zico Kolter, et al. (certified!!) adversarial robustness for free! arXiv preprint urXiv:2206. 10550, 2022.
[13] Sander Joos, Tim Van hamme, Davy Preuveneers, and Wouter Joosen. Adversarial robustness is not enough: Practical limitations for securing facial authentication. In Proceedings of the 2022 ACM International Workshop Security and Privacy Analytics, pages 2- 12, 2022.
[14] Yash Sharma and Pin-Yu Chen. Attacking the madry defense model with 1_ -based adversarial examples. arXiv preprint arXiv: 10733, 2017.
[15] Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversar- ially robust neural network model on mnist. arXiv preprint 17X5v:1805.091 2018
[16] Dylan M Paiton, Charles G Frye, Sheng Y Lundquist, Joel D Bowen, Ryan Zarcone, and Bruno Olshausen. Selectivity and robustness of sparse coding networks. Journal of vision. 30(12):10-10 2020.
[17] Tao Bai, Jinqi Luo, Jun Zhao Bihan Wen, and Qian Wang. Recent advances in adversarial training for adversarial robustness. arXiv preprint arXiv:2102.01356 2021.
11

---

[18] Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David Cox, and James J DiCarlo. Simulating primary visual cortex at the front of enns improves robustness image perturbations. Advances in Neural Information Processing Systems, 33:13073-13087, 2020.
[19] Aditya Jonnalagadda, William Yang Wang, B.S. Manjunath, and Miguel Eckstein. Foveater:
Foveated transformer for image classification, 2022.
[20] Yan Luo, Xavier Boix, Gemma Roig, Tomaso Poggio, and Qi Zhao. Foveation-basee mecha- nisms alleviate adversarial examples. arXiv preprint arXiv:1511.0629: 2015.
[21] Jonathan M Gant, Andrzej Banburski, and Arturo Deza. Evaluating the adversarial robustness of a foveated texture transform module a cnn. In SVRHM 2021 Workshop@ @ NeurIPS, 2021.
[22] Manish Reddy Vuyyuru, Andrzej Banburski, Nishka Pant, and Tomaso Poggio. Biologically inspired mechanisms for adversarial robustness. Advances in Neural Information Processing Systems, 33-2145,, 2020.
[23] Anne Harrington and Arturo Deza. Finding biological plausibility for adversarially robust features via metameric tasks. In International Conference on Learning Representations, 2021.
[24] Emma E. M. Stewart, Matteo Valsecchi, and Alexander C. Schütz. A review of interactions between peripheral and foveal vision. Journal of Vision, 20(12):2-2) 11 2020.
[25] Thorsten Hansen, Lars Pracejus, and Karl R Gegenfurtner. Color perception in the intermediate periphery of the visual field. Journal of vision, 944):66- 26, 2009.
[26] Helga Kolb. Facts and figures conceriing the human retina- nebi bookshelf, May 2005.
[27] Valentin Dragoi and Chieyeko Tsuchitani. Visual processing: Eye and retina (section chapter 14) neuroscience online: An electronic textbook for the neurosciences: Department of neurobiology and anatomy the university of texas medical school at houston, 2020.
[28] Farzad Ramezani, Saeed Reza Kheradpisheh, Simon J Thorpe, and Masoud Ghodrati. Object categorization in visual periphery is modulated by delayed foveal noise. Journal of Vision, 19(9):111, 2019.
[29] Arturo Deza and Talia Konkle. Emergent properties of foveated perceptual systems, 2021.
[30] RT Pramod, Harish Katti, and SP Arun. Human peripheral blur is optimal for object recognition. arXiv preprint
2018.
[31] Panqu Wang and Garrison W Cottrell. Central and peripheral vision for scene recognition: A neurocomputational modeling exploration. Journal of vision, 17(4):9 9, 2017.
[32] MARY A Johnson. Color vision in the peripheral retina. American journal of optometry and physiological optics, 63(2):97-103, 1986.
[33] Michael A Cohen, Caroline Ostrand, Nicole Frontero, and Phuong-Nghi Pham. Characterizing snapshot of perceptual experience. Journal of Experimental Psychology: General, 150(9):1695, 2021.
[34] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[35] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced research).
[36] Johannes Mehrer, Courtney J Spoerer, Emer C Jones, Nikolaus Kriegeskorte, and Tim C Kietzmann. An ecologically motivated image dataset for deep learning yields better models of human vision. Proceedings of the National Academy of Sciences, 118(8)e2011417) 1417118, 2021.
[37] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 1 11:21112252, 2015.
12

---

[38] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint M.K.:::90031226... 2019.
[39] Lisa J Croner, Keith Purpura, and Ehud Kaplan. Response variability in retinal ganglion cells of primates. Proceedings of the National Academy of Sciences, 90((7):8128-811 28380 1993.
[40] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248-255 leee, 2009.
[41] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.00 1e...88050.111 2018.
[42] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. British Machine Vision Conference, 2016.
[43] Jeremy Howard and Sylvain Gugger. Fastai: A layered api for deep learning. Information, 1 11(2))108, 2020.
[44] Matthias Kümmerer, Matthias Bethge, and Thomas SA Wallis. Deepgaze ii:: Modeling free- viewing human scanpaths with deep learning, Journal of Vision, 22(5):7-7, 2022.
[45] Ming Jiang, Shengsheng Huang, Juanyong Duan, and Qi Zhao. Salicon: Saliency in context.
In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1072-1080, 2015.
[46] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference machine learning. pages 2206-2216. PMLR, 2020.
[47] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning, pages 274-283, PMLR, 2018
[48] Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv: 190,066500 2019.
[49] Sylvestre-Alvise Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy A Mann. Data augmentation can improve robustness. Advances in Neural Information Processing Systems, 14429935-229944 2021.
[50] Aounon Kumar and Tom Goldstein. Center smoothing: Certified robustness for networks with structured outputs. Advances in Neural Information Processing Systems, 34:5560-5575, 2021.
[51] Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. Certified adversarial robustness with additive noise. Advances in neural information processing systems, 32, 2019.
[52] Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. Countering adversarial images using input transformations. arXiv preprint arXiv:1711.00117 2017.
[53] Edward Raff, Jared Sylvester, Steven Forsyth, and Mark McLean. Barrage of random transforms for adversarially robust defense. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 5528-6537 2019.
[54] Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adver- sarial examples. In International conference on machine learning, pages 284-293. PMLR, 2018.
[55] Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp- mixer: An all-mip architecture for vision. Advances Neural Information Processing Systems, 34:24261-24272, 2021.
13

---

[56] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani. Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
14

---

24
52.05
50
23
Accuracy
Accuracy
40
22
30
21
18.80
20
20
10
19
1510
25
50 # Steps
100
STE
EOT-10
Method
(b)
Accuracy of a R-Blur model trained on Imagenet under APGD attack with different settings. (a) shows the accuracy when APGD attack is applied with different numbers of update steps. (b) shows the accuracy when 10 step of perertattoon-vee-trannfrommmaa
Figure 11:
(EOT-10) (54) is used and R-Blur is converted into trriightthrough--ssiittto (STE) in the backward pass. The dashed line in (b) shows the accuracy of 25-step APGD attack without EOT and normal gradient computation for R-Blur. Together these results strongly indicate that R-Blur does not obfuscate gradients and legitimately improves the adversarial robustness of the model.
Appendix
A Preventing Gradient Obfuscation
We take number of measures to ensure that our results correspond to the true robustness of our method, and we avoid the pitfalls gradient obfuscation [47 48).
Firstly, we remove inference time stochasticity from all the models we test. We do this by sampling the Gaussian noise used in R-Blur and VOneBlock once and applying the same noise all test images. Similarly, we sample the affin transform parameters for RandAffine once and use them for all test images. We also compute the fixation point sequences for R-Blur and Warp on unattacked images and do not update them during or after rumning APGD.
APGD for to 100 iterations and observed that as the number of iterations increases the success rate the attack increases (Figure [Ta). The success rate plateaus at 50 iterations. Since the attack success rate with 25 steps is only 0.1% lower than the success rate with 50 steps, APGD with 25 steps in most of our experiments.
Secondly,
Thirdly, evaluate R-Blur against AutoAttack [46), an ensemble of 4 state-of-the-art white and black box adversarial attacks. Figure compares the accuracy of R-Blur on Imagenet under APGD and Attock We that the accuracy under AutoAttack only slightly lower than the accuracy under APGD, with the maximum difference being 3%, which would not change any of the trends observed in the paper. Since computing AutoAttack requires lot of time and compute, and given that it does not decrease accuracy significantly compared to 25-step APGD, we chose to use the latter for most of the results presented in the paper.
Finally, we applied expectation over transformation [4] by computing gradient samples at each APGD iteration and averaging them obtain the final update. We found this did not change the attack success rate so we take only gradient sample in most our experiments (Figure TTb). Finally, we also used a srraight-though-esiee them and found that doing so reduces the attack success rate, thus indicating that gradients that pass through R-Blur retain valuable information that be used by the adversarial attack (Figure llbb.
to pass gradients through R-Blur in case it may be obfuscating
B Fixation Point Selection
In this study, we did not attempt to develop optimal fixation point selection algorithm, and instead, we operate under the assumption that points at which humans tend to fixate are sufficiently informative to perform accurate object classification. Therefore, we used DeepGaze-Il IIlI [44), which is neural
15

---

100
100
Figure 12: Accuracy of R-Blur on Imagenet under APGD (blue) and Auto.Attack (orange). The accuracy under APGD and Auto/tttac is very similar, which shows that evaluating with only APGD provides a reliable
Accuracy
APGD
Accuracy
AutoAttack.
40
35
32
40
36
33
20
18 16
20
18
17
3
0
0.002
0.004
0,008
0.5
1.0
2.0
Perturbation Distance |
Perturbation Distance ||||
measure R-Blur's robustness
pertu Bedd
deen
penuteed
diaa
penurtte
100
94
100
89
100
8B
Accuracy
80
77
70
Accuracy
70
60
Accuracy
61
60
60
60
.22
50
40
20
40
26
40
0
0
0
Fcoset-1D
Fraset
Imagenet
Froset-10
Fcoset
Imagenet
Fraset-1D
Froset Dataset
Imagenet
Dataset
Dataset
(a)
Optimal Fixation
DeepGaze-II Five Fixations
(c) Adversarial Training
Figure 13: The accuracy obtained on clean and adversarial data when (a) the optimal fixation point was selected, (b) when the five fixation approach from Section 3 was used, and (c) an adversarially trained model was used.
network model trained to model the human gaze. DeepGaze-Il uses a deep CNN backbone to extract features from the image, and based these features another DNN predicts a heatmap that indicates, for each spatial coordinate, the probability that a human will fixate on it. However, it is possible could be developed. Though
that this algorithm is sub-optimal, and with further study, better developing such an algorithm is out of the scope of this paper, we conduct a preliminary study to determine if it is possible to select better fixation points than the ones predicted by DeepGaze-IIL.
To this end, we run the following experiment to pick an optimal fixation point for each image during inference. For each testing image, we select 49 fixation points, spaced uniformly in grid. Using the models we trained in earlier (see section 3) we obtain predictions for each image and each of the 49 fixation points. If there was at least one fixation point at which the model was able to correctly classify the image, we consider to be correctly classified for the purpose of computing accuracy. We repeat this experiment for Ecoset-10, 10, Ecoset, and Imagenet, using clean and adversarially perturbed data. We obtain the adversarially perturbed images for each of the 49 fixation points by fixing the fixation point at one location running the APGD attack with lo -nom bounded to 0.004. Figure illustrates this experiment with some example images.
The results are presented in Figure 13 on both clean and adversarially perturbed data improves, with the improvement in clean accuracy being the most marked. The clean accuracy on Ecoset-10, 10, Ecoset, and Imagenet improved by 5%, 1 11%, and 10% respectively. which makes the clean accuracy of the R-Blur model on par or better than the clean accuracy achieved by the unmodified ResNet. Furthermore, when the optimal fixation point, is chosen R-Blur obtains higher clean accuracy than AT on all the datasets.
We see that when the optimal fixation point is chosen accuracy
These results are meant to lay the groundwork for future work toward developing methods for determining the optimal fixation point based on the input image. However, they also illustrate that models trained with R-Blur learn features that are not only more adversarially robust features than ResNet but also allow the model to make highly accurate predictions on clean data.
C Evaluations With Different Architectures
To demonstrate that the benefits of R-Blur are not limited to CNNs, we trained MLP-Mixer 55 and ViT (56) models with R-Blur preprocessing and evaluated their robustness. We use the configuration of MiX-Mixer referred to as $16 in [55). Our VIT has similar configuration, with 8 layers each having a hidden size of 512, an intermediate size of 2048, and 8 self-attention heads. We train both models with batch size of 128 for 60 epochs Ecoset- 10 using the Adam optimizer. The learning
16

---

0
200
200
200
200
0
300
0
200
0
300
300
BIRS
200
200
200
200
0
300
0
0
0
200
200
200
200
200
0
309
0
300
300
300
200
200
200
200
200
0
200
0
200
0
200
Figure 14: This figure indicates the locations of the optimal fixation points for some sample images. Each square in the grid corresponds to one of 49 fixation locations and represents the highest resolution region of the image if the model fixates the center of the square. Squares that shaded green indicate that the model's prediction at the corresponding fixation point was correct, while squares shaded red indicate that the model's prediction at the corresponding fixation point was incorrect. We see that there are certain images in which there are only a few optimal fixation points and they may not be in the center in the corners of the image.
rate of the optimizer is linearly increased 0.001 over 12 epochs and is decayed linearly to almost zero over the remaining epochs. The results are shown Figure 15
We observe that R Blur significantly improves the robustness of MLP-Mixer models, and achieves greater accuracy than Warp higher levels of perturbations. These results show that the robustness endowed to ResNets by R- R-Blur was not dependent on the model architecture, and they further strengthen our claim that loss in fidelity due foveation contributes to the robustness of human and computer vision.
D Breakdown of Accuracy Against Common Corruption by Corruption Type
In Figure 16 we break down the performance of the models on common corruptions by higher-level corruption categories. The individual members of each category are listed Table We see that in most of the categories, R-Blur achieves the highest median accuracy against the most severe corruptions. We also note that R-Blur exhibits remarkable degree of robustness to noise, which substantially greater than all the other models we evaluated. It is pertinent note here that Gaussian noise was just of the types of noise included in the noise category, and thus the performance of Blur can not be attributed to overfitting Gaussian noise during training. Furthermore, robustness to
17

---

Milerp
60
60
20
2,623
3.304
0,00
2.006
Pertutation Distance [4]-
Perturbation Distance |C|-
(a) ResNet-18
(b) MLP-Mixer
(c) ViT
Figure 15: The accuracy obtained on Ecoset- 10 against adversarial perturbations of various Eo norms when R-Blur is used with ResNet, MLP-Mixer and VIT backbones.
Noise
Blur
Weather snow
Digital
gaussian noise shot noise
defocus blur
contrast
glass blur
frost
clastic transform
impulse noise
motion blur
fog
pixelate
speckle noise
zoom blur
brightness spatter
jpeg compression
gaussian blur
saturate
Table Categories of corruptions used to evaluate robustness to common corruptions. This catego- rization follows the one from 38
one type random noise does not typically generalize to other types of random noise 4 Therefore, the fact that R-Blur exhibits improved robustness to multiple types of noise indicates that it is not just training on Gaussian noise, rather the synergy of all the components of R-Blur that likely source of its superior robustness.
E Sensitivity Analysis of Hyperparameters in R-Blur
To measure the influence of the various Hyperparameters of R-Blur conduct sensitivity analysis. First, we vary the scale the Gaussian noise added to the image, the viewing distance during inference, and the value of 3 from Section 2.5 which is the scaling factor that maps eccentricity (see cquation[[to standard deviation, and measure the impact on accuracy on clean as well as adversarially perturbed data. The results of this analysis
presented in Figure 17 We see that, as expected, increasing the scale the noise improves accuracy on adversarially perturbed data, however, this improvement does significantly degrade clean accuracy. It appears that the adaptive blurring is mitigating the deleterious impact of Gaussian noise on clean accuracy. On the other hand, increasing beyond 0.01 surprisingly does not have significant impact accuracy and robustness. We also measured the accuracy on clean and perturbed data after varying the viewing distance (see 2.4) and the number of fixation points over which the logits are aggregated. These results are plotted in Figure 18 and they show that accuracy on clean and perturbed data is maximized when the width of the in-focus region is 48 (this corresponds vd and aggregating over more fixation points improves accuracy on clean and perturbed data.
F Training Configuration
Table 3 presents the configurations used to train the models used in our evaluation, For all the models the SGD optimizer was used with Nesterov momentum=0.9.
G Implementation Details
We used Pytorch v1.11 and Python 3.9. 12 to our implementation. We used the implementation of Auto-PGD from the Torchattacks library
For Warp we used the code from the official repo http:///ttthb..oommmmmuyyy////rt Likewise, for VOneBlock we used the code from https//gghhbb.ccm///tttoodovvnnn and
18

---

Compption Severiy
Comppton Seretiy
(a) Imagenet
2.00
230
310
Standirire
70
Compution Seuutty
Compptio Sovariy
Comppttin
(b) Ecoset
Bos
09
08
0.5
0.
Compraan Soverty
Compation severty
Camust oe soverit
Comussce Saverity
(c) blur
(d) noise
(e) weather
(f) digital
(g) CIFAR-10
Figure 16: The accuracy achieved by R-Blur and baselines on various classes of common corruptions, proposed in (38) The boxplot shows the distribution of accuracy values 4-5 different corruptions in each class applied at different severity levels (X-axis) with referring to least severe and being the most severe corruption. R-Blur generally achieves the highest median accuracy on the highest severity levels.
0.004
0016
00-0.3231 025)
(o-5.250
01
100
0.003
Accuracy
Accuracy
60
60
40
40
0.0
0.002
0.004
0.008
0.016
0.00 0.01
0.02
0.03
0.04
0.05
Perturbation Distarce [|]]
B
Figure 17: The impact of the hyperparameters of R-Blur on the accuracy and robustness of models trained on Ecoset-10. (left) the standard deviation of Gaussian noise, and (right) from Section 2.5
for DeepGaze-III models we used the code from http://gtttub.commmmttttt--eDppp The training code for DeepGaze-III with R-Blur and R- Warp backbones is based on wwwpa///wwmaanaaseeeeeeeeeeeeaaa
derrrranndddeege....i..y. and can be found in http://///ff.iatoopeeeeiiee
AavarsariaalM//boooggaaaallll Our clones of these repositories are included in the supplementary material. multi-gpu training, we used Pytorch Lightning v1.7.6. We used 16-bit mixed pre- cision training to train most of our models. The code for R-Blur can be found in which
serrainddeeggzeepre -p.. For
httprrrraaaaailoooolllllaa.aeeea http:///aaaaaa....eeeeeeeeeeeeee part of the supplemental material.
19

---

60
Accuracy
is perturrddddd
Accuracy
is perturbed
40
True 30
True
10
15
31
48
87
Width of In Focus Region
Fixation Points
Figure 18:
The impact of the size of the in-focus region by varying the viewing distance (left) and the number of fixation points over which the logits are aggregated (right) on accuracy. The plots are computed from a R-Blur model trained on Imagenet, and perturbed data is obtained by conducting 25-step APGD attack with ||ō maximized when the width of the in- focus region is 48 (this corresponds to ed 3) and aggregating over more fixation points improves accuracy on clean and perturbed data.
0.004. We see that accuracy clean and perturbed data is
Dataset CIFAR-10
Method
Batch Size
nEpochs
LR
LR-Schedule
Weight Decay
nGPUs
ResNet
128
0.4
60
Warmup-Decay(0.2)
5e-5
AT
128
0.4
60
L WarmuupDecay(O))
5e-5
Warp
128
0.4
60
L- Warmup-Decay(O)
5e-5
R-Blur
128
0.4
60
L- Warmup-Decay(O..
5e-5
G-Noise
128
0.4
60
L- Warmup-Decay((..
5e-5
Ecoset- 10
ResNet
128
0.4
60
L Warmupp-ecaay((
5e-4
AT
128
0.4
60
L- Warmup-Decay(0.2)
5e-4
R- R-Wa
128
0.4
60
L Warmup-Decay(0)
5e-4
R-Blur
128
0.1
60
L- Warmup-Deeca((.1)
5e-4
VOneBlock
128
0.1
60
L Warmup-DDecay(0.1)
5e-4
G-Noise
128
60
Warmup--Decay(0..2)
5e-4
Ecoset
ResNet
256
0.2
25
L Warmup--eccay(0.)
5e-4
2
AT
256
0.2
25
L- Warmup--eccay0..))
5c-4
R- R-Warp
256
0.1
25
L- Warmup-Deccay(..))
5e-4
4
R-Blur
256
0.1
25
C- C-Warmup-2xDecajj
5e-4
VOneBlock
256
0.1
25
--aamup-24Decay(..11
5
4
G-Noise
256
0.1
25
C- <-aamup-2DDecayy00)
5e-4
Imagenet
ResNet
256
0.2
25
L-Warmup-Decay(0.22
5e-4
2
AT
256
0.2
25
L-Warmup-Deccy((222
5e-4
R- R-Wa
256
0.1
25
L-Warmup-Decay(0.2)
5e-4
4
R-Blur
256
0.1
25
C- C-Warmup-21Decay((11
5e-4
VOneBlock
256
0.1
25
C-Warmup-2xDccay(/
5e-4
4
G-Noise
256
0.1
25
C-Warmu-2xDecay(0.1
5e-4
4
Table 3: The configurations used train the models used in our evaluation. L-Warmup-DecayL represents schedule that linearly warms up and decays the learning rate and f represents the fraction of iterations devoted warmup. C- decay follow cosine function, and there are two decay phases. Both the schedulers are implemented using forchhoptiin 1r_ lr_scheduler OneCycleLR from Pytorch.
is similar except that the warmup and
20

---

Method
Dynamic Fixation
# Fixations
Train Speed (img/s)
Test Speed (img/s)
ResNet
410
370
AT
X
232 (1.8x)
VOneBlock
277 (1.5x)
289 (1.3x)
R- Warp
377 (1.1x)
314 (1.2x)
R-Blur
369 (1.1x)
334 (1.1x)
R-Blur
111 (3.3x)
R- Warp
5
26 (14.2x)
R-Blur
115 (3.2x)
R-Blur
29 (12.9x)
Table Training and inference speed on Nvidia 2080Ti, measure in images per second, for each model. The value in the parentheses indicates the slowdown relative to the unmodified ResNet computed as the (train/test) speed the ResNet divided by the speed of the other method. We see that R-Blur, in and of itself, causes very minimal slowdown (only 1.1x) during training and testing. Increasing the number fixations slows R-Blur only sub-lincarly (5 predefined fixations lead to 3x slowdown). Introducing dynamic fixation prediction has greater impact on speed because each image assigned different fixation points and so R-Blur/R- Warp can not be applied to them as single batch. This shortcoming is likely common to most fixation transforms, and is not unique to R-Blur.
H Hardware Details and Computation Cost
We trained our models compute clusters with Nvidia GeForee 2080 Ti and V100 GPUs. Most of the Imagenet and Ecoset models were trained and evaluated on the V V100s, while the CIFAR-10 and Ecoset- 10 models were trained and evaluated on the 2080 Ti's.
H.1 Analysis of Computation Cost
Table presents this comparison and shows that R-Blur causes minimal slowdown (1.1x 1x compared to the vanilla ResNet) during both training and testing. Also, increasing the number of fixations slows R-Blur only sub-linearly (5 predefined fixations => 3x slowdown). Introducing dynamic fixation prediction has greater impact speed because each image is assigned different fixation points and so R-B R-Blur/R- Warp can not be applied to them as a single batch. This shortcoming is likely common to most fixation transforms, and is not unique to R-Blur, In fact, under dynamic fixation prediction, R- R-Blur is faster than R-W R-ra...
21


Reviews:
[{"review_id": "XcQfldGBFP", "reviewer": "Reviewer_6sqj", "summary": "This paper proposes a data augmentation technique named R-Blur to improve the robustness of vision classifiers against adversarial perturbations and other non-adversarial image corruptions. The method is inspired by human visual systems where the perceived scene consists of varying levels of fidelity. As such, the training images are modified in a way that adaptive Gaussian filtering is applied centered around the fixation point in the image. Results on CIFAR-10, Ecoset, and Imagenet demonstrate that R-Blur improves robustness to adversarial perturbations and common corruptions compared to standard trained models. ", "questions": "Question/clarification:\nIs $W_{V}$ in (1) the width of the image? \nAre (2), (3), and (4) from previous work ([26]?), or are they part of the novel contributions from this paper? Why are Laplace and Cauchy distributions used? Perhaps some additional discussion following the definitions can be helpful to further improve the clarity.\n\nIn the evaluation of adversarial robustness, why is only APGD in Autoattack used, rather than the complete version of AutoAttack?\n\nIn other adversarial training methods, we can explicitly control the trade-off between standard accuracy and robust accuracy. For instance, $\\beta$ in TRADES, $\\epsilon$ of the perturbation in standard adversarial training. Does such a concept exist for R-Blur? I think understanding such a mechanism can further improve the adversarial robustness of models trained with R-Blur.\n\nSuggestion:\nIn Figure 4, it seems that the sequence of fixation points does not converge at all. Also, from Ln182, it seems that the results are based on randomly selected fixation points. One suggestion is to identify the fixation point as the pixel location with the highest saliency. Saliency-based data augmentation (i.e., Ma et al and Uddin et al) can be a good starting point.\n\nBe consistent with the use of gray or grey (grayscale or greyscale).\n\n\nMa, Avery, et al. \"SAGE: Saliency-Guided Mixup with Optimal Rearrangements.\" arXiv preprint arXiv:2211.00113 (2022).\nUddin, A. F. M., et al. \"Saliencymix: A saliency guided data augmentation strategy for better regularization.\" arXiv preprint arXiv:2006.01791 (2020).", "limitations": "Limitations including the trade-off between the robust accuracy and the standard accuracy are discussed.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: There are only a few biologically inspired adversarial defense techniques. The proposed method mimics the peripheral vision in human visual systems and modifies the training images with an adaptive Gaussian filter. The approach is very unique and interesting.\n\nQuality: The paper is well-written. \n\nClarity: The motivation behind the proposed method as well as the overall structure of the paper is clear. The technical details are explained clearly.\n\nSignificance: The presence of adversarial examples presents a security concern for deep neural networks utilized in various applications. This paper introduces a novel approach to bolster network robustness.", "weaknesses": "This paper has two main weaknesses.\n1. The choice of baseline methods in evaluation. The R-Blur method, in its essence, is a Gaussian data augmentation technique, while the only non-adversarial training technique in the baseline is RandAffine. To concretely verify that the adaptive filtering from R-Blur is indeed improving the robustness of the model beyond simple Gaussian data augmentation, other baseline methods such as Gaussian augmentation (with difference variances) and l2 regularization are necessary. \n\n2. The paper position itself as an approach to improve the adversarial robustness of deep neural networks. However, results in Sec. 3 show that the improvement in adversarial robustness towards the APGD attack is significantly lower than adversarial training. Also, the choice of $\\epsilon$ in the adversarial robustness is much smaller than the standard values used in other works. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We hope that in our responses below we will be able to fully address the reviewer's outstanding concerns and that the reviewer will consider increasing their score of our paper.\n\n**verify that the adaptive filtering from R-Blur is indeed improving the robustness of the model beyond simple Gaussian data augmentation**\n\nTo verify if adaptive blurring in R-Blur does indeed contribute to robustness we compare R-Blur with models that either add Gaussian noise (with the same variance as the noise in R-Blur) or perform non-adaptive Gaussian blurring (with the variance equal to the maximum variance used by R-Blur). The results are presented in Figure 8 and we observe that both of these models are less robust than R-Blur. We have repeated this analysis for Imagenet models ( Figure 1 of the global response) and observed that the R-Blur model consistently achieves higher accuracy than baseline methods on all adversarial perturbation sizes. \n\nWe also trained models augmented with non-adaptive Gaussian blur of different variances, as well as a model that combines non-adaptive Gaussian blur with Gaussian noise (Figure 4 of the global response), and found R-Blur to have superior robustness then all of them\n\n** improvement in adversarial robustness towards the APGD attack is significantly lower than adversarial training.**\n\nWe would like to clarify that the objective of this paper was to study the impact of foveation on the robustness of DNNs and we have not claimed that R-Blur is superior to adversarial training as a defense. Our claims are that (1) R-Blur improves the robustness of CNNs to image perturbation – both adversarial and non-adversarial, compared to the vanilla CNN and other biologically-inspired methods, and (2) unlike adversarial training, which seems to have negligible effect on robustness to non-adversarial image perturbations, R-Blur increases robustness to a variety of image perturbations, not just L_p bounded adversarial attacks. We believe that these claims are justified by the results. Nevertheless, we will rewrite certain parts of the paper to ensure that no confusion remains regarding our objectives and claims.\n\n**the choice of eps in the adversarial robustness**\n\nWe sought to validate the claim that R-Blur improves robustness to perturbations – both adversarial and non-adversarial, compared to the vanilla CNN and other biologically-inspired methods. Therefore we chose $\\epsilon$ values on which the compared models had accuracy greater than 0%. As shown in Figure 5 in the paper and Figure 1 in the global response, the accuracy of all the models, except AT, on ecoset/imagenet becomes close to 0% on the largest perturbation we used so evaluating at larger perturbation would not have served any purpose. Furthermore, the perturbation sizes ($\\epsilon$) used in our evaluation are on par or higher than the sizes used in papers on biologically-plausible adversarial defenses [18,21,22].\n\n**Is W_V in (1) the width of the image?**\n\nYes, W_V is the width of the image – 224.\n\n**Are (2), (3), and (4) from previous work ([26]?), [...]?**\n\nThe equations 2, 3, and 4 are not from previous work [26]. We devised these equations to match the curves of visual acuity presented in Figure 14.21 (B) in [26].\n\n**Why are Laplace and Cauchy distributions used?**\n\nThe Laplace and Cauchy distributions are chosen to approximate the shape of the curves in Figure 14.21 (B). We will update the paper to make this clearer.\n\n**why is only APGD in Autoattack used**\n\nWe used only APGD because (1) the certified accuracy of R-Blur (Figure 7) was similar to empirically measured accuracy under adversarial perturbations (Figure 5 top row), and (2) our analysis in Section A of the Appendix revealed no gradient obfuscation effects. Under these conditions, we were confident that evaluating with only APGD would give a reliable measure of robustness.\n\nAt the reviewers’ request, we ran these experiments for the R-Blur augmented model trained on Imagenet. Figure 3 in the global response shows that the accuracy under AutoAttack is only slightly lower than the accuracy under APGD, with the maximum difference being 3%, which would not change any of the trends observed in the paper.\n\n**Can the accuracy-robustness tradeoff be controlled for R-Blur?**\n\nThe accuracy and robustness can be traded with each other by controlling the noise added in R-Blur. Figure 6 in the Appendix illustrates this trade-off. We see that as the variance of the noise is increased from 0.125 to 0.5 the clean accuracy drops from 90% to 85%, while the accuracy under attack increases from around 30% to 50%.\n\n**In Figure 4, [...] the sequence of fixation points does not converge**\n\nThe fixation points in Figure 4 do not converge to a single spatial location by design, as stated in the caption. This is in line with human/animal behavior, where it has been observed that they fixate on different salient parts of the scene to progressively accumulate more information.\n\n**from Ln182, it seems that the results are based on randomly selected fixation points.**\n\nWe would like to point out that the Ln180-182 talk about the training set up. However, as mentioned in Ln185-185, during inference a model of human gaze (DeepGaze-III) is used to select the fixation points. \n\n**One suggestion is to identify the fixation point as the pixel location with the highest saliency.**\n\nWhile the reviewer’s suggestion is well taken and we thank the reviewer for sharing relevant papers, we would like to clarify that our approach does in fact subsume this technique. We use DeepGaze-III to predict a saliency map for a given image (see Figure 4 top-row). From this saliency map we pick the most salient coordinate as the fixation point. We repeat this process, as shown in Figure 4, to get a sequence of multiple fixation points. We will rewrite the relevant parts of the paper to ensure that it is clear that we are already using saliency maps to select fixation points. \n"}, {"review_id": "8PkGq5i0Yf", "reviewer": "Reviewer_74Y4", "summary": "The paper presents a biologically inspired approach that improves the robustness of DNNs against samples with adversarial perturbations or common corruptions. In the proposed approach the models are trained using the images transformed using the proposed R-Blur (Retina Blur) transformation. The proposed R-Blur simulates foveation by adaptively blurring the image pixels and reducing the color saturation based on the distance from the given fixation point. The effectiveness of the proposed approach is demonstrated by considering models trained on CIFAR-10, Ecoset, and ImageNet datasets, respectively.", "questions": "1. Is the proposed R-blur frame differentiable? \n2. For the white box adversarial attack, was the R-blur and the fixation model considered for generating the adversarial samples?\n3. Is the perturbation size ($\\epsilon$) defined for 0-1 or 0-255 pixel range (L178 - L179)?\n4. To further validate the adversarial robustness of the proposed approach present sanity check described in [a,b].\n5. How is the pixel wise fusion of color (HxWx3) and the grayscale (HxW) image performed?\n6. Numbering for equation below L142 missing.\n7. It is not clear why the subset of test images are used for Ecoset and ImageNet. Are the models trained on the entire training set or subset for these cases?\n8. The data-augmentation techniques described in L164 to L167 are used only for the proposed approach or for all compared methods?\nSensitivity of the proposed approach to fixation method. Training details of DeepGaze-iii are missing (i) dataset and (ii) depth of the ResNet [L192]. \n9. Provide results for AutoAttack, apart from APGD attack. \n10. In L202, L204, and L254: does \"..ResNet..\" mean WideResNet-22-4\n11. Is the model robust to geometric attacks (e.g. translation, rotation, and affine)?\n12. Provide comparison of the training and inference time for the methods considered.\n\n[a] Carlini et al. \"On Evaluating Adversarial Robustness\" arxiv 2019\n\n[b] Athalye et al. \"Obfuscated Gradients Give a False Sense of Security\" ICML 2018", "limitations": "The main limitation of the proposed approach is dependency on the external fixation model which can act as a bottleneck in terms of computation and performance. Further, it is not straightforward to adapt the proposed approach to other computer vision tasks such as segmentation, and multi-object classification. The authors are suggested to include a discussion on the same.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The proposed biologically inspired approach is a non-adversarial training approach that yields robust models. These models are robust to unseen adversarial and common corruptions. Experimental results validate the same.", "weaknesses": "1. The proposed approach depends on an external model for fixation point generation. The fixation model can act as a computational and performance bottleneck. It is not clear whether the white-box evaluation considers the susceptibility of the fixation model.\n2. From the ablation study, it can be seen that noise plays a critical role. The paper fails to highlight the role of noise in the proposed R-blur that simulates foveation. \n3. Missing experimental details: Important experimental setup details are missing (such as architecture details, training setup, and attack setting). Furthermore, certain experiments are missing to demonstrate the effectiveness of the proposed approach (computation time, auto attack). Refer to the question section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback and hope that our responses address their concerns\n**[...]The paper fails to highlight the role of noise [...].**\nWe will highlight the role of adding noise in writing. Figure 1 in the global response shows that under moderate adversarial perturbation, the model with only Gaussian noise is almost half as accurate as a model with R-Blur on Imagenet. This result and Figure 8 in the paper show that the noise alone does not explain the robustness gains of R-Blur.\n**Is [...] R-blur [...] differentiable?**\nYes, R-Blur is linear and fully differentiable.\n\n**[...] Were R-blur and the fixation model considered for generating the adversarial samples?**\nThe white-box attack takes R-Blur into account: gradients are propagated through it to the input image. We remove the fixation model after using it to sample 5 fixation points from the clean input image, which remain constant in subsequent iterations while the attack is computed. Details in Appendix Section A.\n\n**[...] present sanity check described in [a,b].**\nWe performed these checks and found no issues – more attack iterations reduce accuracy, computing the attack over the average logits from several forward passes (expectation over transformation) has no effect on accuracy, converting R-Blur to a straight-through estimator in the backward pass reduces attack effectiveness. (See Appendix Section A)\n\n**How is the pixel wise fusion [...] performed?**  \n    The grayscale image is HxWx3 with the values replicated on all color channels. The pixelwise fusion is done by taking a weighted sum according to the equation on Ln142 and the following code:\n    `final=(W1*gry_blr+W2*clr_blr)/(W1+W2)`\n    Where gry_blr and clr_blr are the blurred grayscale and colored images, and W1 and W2 are HxWx1 matrices containing the gray and colored acuity estimates at each location.\n\n**[...] why the subset of test images [...]. Are the models trained on the entire training set [...]?**\nThe models are trained on the entire training set. They are evaluated on the subset of the test data to speed up experimentation. We randomly shuffle the test set before extracting the subset to eliminate any biases due to dataset organization.\n\n**The data-augmentation techniques described in L164 to L167 are used only for the proposed approach or for all compared methods?**\nThe same data augmentation is used to train all the models.\n\n**Sensitivity of the proposed approach to fixation method.**\nThe accuracy of R-Blur on clean images _is_ sensitive to the choice of fixation point. As mentioned in Section 5, and in Appendix B, if the optimal fixation point is chosen (by exhaustive search) for each image, the accuracy of R-Blur on clean Imagenet increases from 60% to 70%, which is almost on par with the standard ResNet. Further, the robustness of R-Blur is also sensitive to the _number_ of fixation points (See  Figs 7 and 8 in the Appendix). \n\n**[...]details of DeepGaze-iii are missing [...].**\nWe will update the paper with the details mentioned below: \nWe used the training code from the deepgaze-iii github repo, and replaced the DenseNet-201 with the R-Warp/R-Blur augmented XResNet-18-2 trained on ImageNet. The ResNet in DeepGaze and the ResNet used for classification share the same parameters. This improves performance and reduces the additional parameters in the final model. Following [41] we train DeepGaze on SALICON (Jiang et. al, 2015). This corresponds to Phase 1 of training mentioned in Table 1 of [41]. We did not notice any benefits in our use case of the additional fine-tuning so we decided to skip Phases 2-4.\n\nM. Jiang, S. Huang, J. Duan, Q. Zhao, “SALICON: Saliency in Context”, CVPR’15\n\n**Provide results for AutoAttack [...].**\nWe ran these experiments for the R-Blur augmented model trained on Imagenet and show results in Figure 1 in the global response. AutoAttack reduces the accuracy by < 3% compared to APGD, and thus would not change any of the trends observed in the paper. \n\n**Is the model robust to geometric attacks (e.g. translation, rotation, and affine)?**\nWe use RandAugment and AutoAugment to introduce a lot of geometric transformations into the training data therefore _all_ the models exhibit a high degree of invariance to the geometric transformations mentioned above.\n\n**[...] comparison of the training and inference time [...].**\n\nTable 1 in the global response presents this comparison and shows that R-Blur causes minimal slowdown (1.1x compared to the vanilla ResNet) during both training and testing. Also, increasing the number of fixations slows R-Blur only sub-linearly (5 predefined fixations => 3x slowdown). Introducing dynamic fixation prediction has a greater impact on speed because each image is assigned different fixation points and so R-Blur/R-Warp can not be applied to them as a single batch. This shortcoming is likely common to most fixation transforms, and is not unique to R-Blur. In fact, under dynamic fixation prediction, R-Blur is faster than R-Warp.\n\n**[...] fixation model can act as a bottleneck [...].**\nThe fixation model is not a strict dependency. As shown in the ablation study (Figure 8), removing it (and using 5 predefined fixation points) has a minor impact on the accuracy and robustness of the model. For latency-sensitive scenarios, the fixation prediction model may be removed.\n\n**[...] not straightforward to adapt [...] to other computer vision tasks [...]**\nWe would appreciate it if the reviewer could elaborate on the potential hurdles they see. In most of the CV tasks a CNN is used to compute an embedding for the image. If only 1 fixation is used then the standard CNN can be swapped with a R-Blur augmented CNN. For multiple fixations, one can get the image embeddings for each fixation point independently and aggregate them by summation, concatenation, etc. We will discuss this in the paper.\n\nSome questions couldn’t be answered in the character limit but we can address them in the follow-up."}, {"review_id": "V09bdiI52n", "reviewer": "Reviewer_ysro", "summary": "In their paper the authors introduce R-Blur as a foveation technique for biologically inspired defense against adversarial attacks on DNNs.\nWith their approach they try to simulate the human visual field by blurring and desaturating the image depending on the distance to a given fixation point. They continue to evaluate their approach on Ecoset and ImageNet and try to show that R-Blur can achieve results like adversarial training. Furthermore, they provide an ablation study to prove the significance of the individual components.", "questions": "- The main motivation for the parameter setting for the visual acuity estimation is based on the photopic and visual acuity in human vision which should approximate the real curves (Figure 1(a)). It is not entirely clear how this estimated acuity relates to the original, because there it is measured on the x-axis through the degree in the visual field. In Figure 1 it seems to be the distance-based eccentricity although it should be zero were the original had zero degrees. How exactly does your approximation match the original and why does the transfer work?\n\n- In the related work section several other foveation-based robustness models are mentioned and only a very short comparison is given. A more detailed comparison with other foveation techniques in general would be insightful as well. It is mentioned that these techniques were not used in an adversarial context before, but how does R-Blur stand out?\n\n- For the eccentricity computation only on maximum of the Manhattan-Norm was considered. Were other quantified metrics (e.g., L2-Norm as in the actual human perceptive field) used in the experiments and how did they perform?", "limitations": "The authors address their limitations adequately in their Limitations section. These include a significant loss of clean accuracy and the current fixpoint selection method.\n", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- I find the idea of taking a biologically inspired approach to foveation and the attempt to recreate the human perceptive field through color and grey acuity appealing. \n- The results for the common non-adversarial corruption show an improvement compared to other methods indicating some significance.\n", "weaknesses": "- In Figure 5 we only see a comparison with two baselines that feature little to no adversarial defense. Here it is clear that R-Blur achieves higher accuracies under white-box attacks. A comparison with adversarial training would be more insightful, because later only the mean value of these experiments is presented which indicates that AT outperforms R-Blur by a margin. This weakens the statement that R-Blur generalizes robustness.\n\n- The ablation study provides some insights into the importance of some components but is not explained adequately, e.g., the dynamic selection of the fixation point seems to worsen the performance.\n\n- The formatting to indicate the best and the second-best entries of Table 1 is somewhat misleading: it would be better to underline the second-best result or use colors.\n\n- Writing: Some of the figure captions are not comprehensible or not descriptive enough (e.g., Figure 2 and Figure 3).\n\nOverall: Although R-Blur is well-motivated and implemented in a reasonable way, only a small part of the experiments indicates a significant improvement (mainly the non-adversarial corruptions, Table 1.) while in general adversarial training seems to be superior. Currently I decide for a borderline reject, but if the before mentioned weaknesses are addressed and the following questions are answered, I am willing to increase the rating.\n\nMinor Detail: Some figures (e.g., Figure 7 and especially Figure 2) are too small and are sometimes missing entries (e.g., R-Blur-5FL in Figure 7(b).1 or 0 entries in Figure 5)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for providing valuable feedback on our paper. We find it encouraging that the reviewer will earnestly consider improving their rating if their questions are answered. \n\n**In Figure 5 [...] A comparison with adversarial training would be more insightful**\n\nThe goal of Figure 5 is to show that simulating foveation with R-Blur improves robustness. This can be broken into two sub-goals: (1) showing that augmenting a model with R-Blur improves its adversarial robustness (ResNet vs R-Blur), and (2) showing that the improvement in robustness is due to simulating foveation and not due to simply applying any arbitrary transforms at inference time (RandAffine vs. R-Blur).  Furthermore, we do not claim that R-Blur is a SOTA adversarial defense, and, therefore, we do not position it as a competitor of AT as far as whitebox adversarial attacks are concerned. Therefore, including AT in Figure 5 would only convolute the message it was meant to convey.\n  \nHowever, at the reviewer’s request, we have created Figure 1 in the global response that compares the accuracy of R-Blur and AT at different adversarial perturbation sizes. Likewise, figure 5 in the appendix shows the breakdown of the accuracy of all the models against common corruptions of various types and strengths.\n\n**[...]AT outperforms R-Blur by a margin. This weakens the statement that R-Blur generalizes robustness.**\n\nWe would like to point out that AT outperforms R-Blur only on whitebox attacks but not on non-adversarial corruptions. It is expected that AT models will be more robust to whitebox attacks. This is because they are trained on adversarial attacks that are very similar to the attacks used during testing, and therefore  the gap between the training and test distribution of AT models is much smaller than models like R-Blur which were not trained on adversarially perturbed data.\n\nFor this reason, we state in Ln255-256 that we are comparing AT and R-Blur primarily in terms of their robustness to non-adversarial image perturbations. From Table 1 we see that AT has almost no impact on robustness to non-adversarial image perturbations. On the other hand, R-Blur improves the robustness of the model, not only to adversarial attacks but also to non-adversarial perturbations. These results show that the robustness of AT is limited to a small class of perturbations generated by norm-bounded adversarial attacks, while the robustness of R-Blur generalizes better across different types of perturbations.\n\nWe acknowledge that a lack of clarity in our presentation might have given rise to this misunderstanding. Perhaps the mean scores in columns 2-4 are not necessary and are obscuring the point we intend to convey. We will consider removing them.\n\n** importance of dynamic fixation selection not explained adequately in ablation**\n\nWe had decided to not discuss the dynamic fixation selection in the text for two reasons: (1) the fixation selection model (DeepGaze-III) is not part of R-Blur and we only use it as a tool, and (2) Figure 8 indicates that switching between dynamic and static fixations trades off <= 1% accuracy for robustness, however verifying this trade-off would require more experimentation which we felt was out of the scope of this paper. Nevertheless, we will briefly discuss these points in the text. \n\n**[...]In Figure 1 it seems to be the distance-based eccentricity[...]**\nWe request the reviewer to please correct us if we’re wrong but it seems that the reviewer is referring to Figure 2, not Figure 1. The x-axis in Figure 2 is indeed mislabeled and is showing the index of the pixel on a horizontal line through the fixation point. We have updated this figure to reflect the eccentricity as computed by equation 1. We will include this figure in the paper and have also included it in the global response (Figure 2).\n\n**[...]A more detailed comparison with other foveation techniques[...].**\n\nWe will include the following comparison in the paper.\n\n[19,20,21,22] claim to simulate some aspect of foveation. [19] implemented foveation by cropping the salient region of the image at inference time. Firstly, the biological plausibility of [19] is questionable because instead of simulating the degradation of acuity in the periphery of the visual field, it simply discards it. Secondly, they crop the image _after_ applying the adversarial attack, which likely obfuscates the gradients, and hence any robustness they report is suspect. On the other hand, [20] and [21] apply foveation in the latent feature space (the intermediate feature maps generated by a CNN) rather than directly to the image pixels as we do so their methods are not directly comparable to ours. To the best of our knowledge [22] (R-Warp) is the only foveation-based adversarial defense that is biologically plausible, works directly on the pixels, and avoids gradient obfuscation, which is why we compare against it in this paper.\n\nFurthermore, in Ln50-52 we do acknowledge that [28,29] also simulated foveation via adaptive blurring, and, point out that R-Blur builds upon these methods by (1) also simulating the loss in color sensitivity and the stochasticity of neural responses, and (2) rigorously evaluating the impact on robustness.\n\n**Were other quantified metrics [for eccentricity] used [...]?**\nWe did not try other distance metrics. We do not expect a change of distance metric to significantly influence the accuracy and robustness because it will impact only a few pixels. On the other hand, their impact on the speed and memory requirements might be more significant since extracting a circular region of the image (as necessitated by the L2-norm) would be more computationally intensive than simply slicing the tensor to extract a square region of the image.\n\nWe hope that our responses fully address the reviewer's concerns and we kindly request the reviewer to consider increasing their score of our paper. "}, {"review_id": "EOGZKFjpFt", "reviewer": "Reviewer_atHR", "summary": "This paper studies the effect of foveation via adaptive gaussian blurring and color modulation in training on an image. Their goal is not computer vision driven, nor ML based, but rather to shed light on the physiological nature of the retina and spatially adaptive computation in humans -- that may prove useful for machines towards achieving robustness without adversarial training.", "questions": "* Missing References (to add in line 32 and through-out the paper regarding links of biological vision (peripheral computation) + adversarial robustness): \n**“Finding Biological Plausibility for Adversarially Robust Features via Metameric Tasks”. Harrington & Deza. ICLR 2022.**\n\n* Missing Reference in Line 51: I believe Pramod et al. did in fact perform robustness tests on their foveated adaptive blur image.\n\n* Missing Reference in Line 52: Characterizing a snapshot of perceptual experience. By Cohen et al. from the Journal of Experimental Pscyhology 2021.\n\nOverall I think this paper is a good contribution to the field, it should provide further discussion at NeurIPS about foveation, which many vision scientists have been longing to see these type of augmentations be used in computer vision. It would be quite nice if authors release a R-Blur augmentatino module in pytorch.", "limitations": "See Weaknesses above.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* This paper was very easy to read, and understand. All the figures and tables are coherent, so I applaud the efforts of presentation of the authors.\n* Authors co-modulate adaptive gaussian blurring with color loss and noise (however the contribution of each factor in the \"foveation\" is not obvious).\n* Authors present a solid list of experiments in o.o.d. experiments and also adversarial attacks. Their benchmarks regarding adversarial training are on spot (though which type of AT I believe is not precised).\n* Their claims are grounded, and not too strong, so that is good given the evidence they have presented and also works that other authors have done.", "weaknesses": "I believe the strongest weakness is that it's not clear to me what the contribution is of each effect : 1) noise ; 2) adaptive gaussian blurring; 3) fixations.\n\nI wonder to what point do the extra fixations add as a proxy for data augmentation when training the model. I.e.: can a false conclusion be arrived that \"foveation aids in robustness\" by virtue of adding extra images. I think authors need to add 2 more controls for me to be more enthusiastic about this paper: \n1) Train all non-foveated networks with additional sets of images via data augmentation procedures (like rotation, mirroring, random crop and blurring)\n2) Train all foveated networks with less images to match the original dataset size of the networks that recieve non-foveated inputs.\n\nIn addition it's not clear to me why authors add noise as part of the foveation. Why not just go straight towards the adaptive gaussian blurring? Is there a psychological reason to add the noise in the foveation process?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for providing valuable feedback on our paper, and asking thoughtful questions. We are glad and encouraged to find that the reviewer liked our work and found it to be unique and interesting. We hope that in our responses below we will be able to fully address the reviewer's outstanding concerns and further improve the reviewer's opinion of our work.\n\n**I believe the strongest weakness is that it's not clear to me what the contribution is of each effect : 1) noise ; 2) adaptive gaussian blurring; 3) fixations.**\n\nWe agree with the reviewer that it is essential to gauge the impact of the different components of R-Blur, and therefore we have conducted an ablation study and presented it in Section 3.3 and Figure 8. To ascertain the impact of each component of R-Blur we removed or disabled each component one by one, while leaving all the other components in place. We find that removing the noise led to the greatest reduction in accuracy under adversarial attack (-33%), followed by the removal of adaptive blurring (-27%) and using 1 fixation instead of 5 (-10%). We hope that this addresses the reviewer’s concerns. We also encourage the reviewer to refer to section 3.3 for more details. If our analysis is lacking in any way we would be eager to further discuss this with the reviewer and do everything possible to improve it.\n\n**[...] can a false conclusion be arrived that \"foveation aids in robustness\" by virtue of adding extra images. I think authors need to add 2 more controls for me to be more enthusiastic about this paper: (1)Train all non-foveated networks with additional sets of images via data augmentation procedures (like rotation, mirroring, random crop and blurring. (2)  Train all foveated networks with less images to match the original dataset size of the networks that recieve non-foveated inputs.**\n\nThese tests are, in fact, reported, although, perhaps we did not communicate this clearly in our writing. Specifically, we train all our models (those with R-Blur and those without) with RandAugment or AutoAugment, which includes the data augmentations mentioned by the reviewer in addition to a wide variety of other non-geometric transformations. While we have indicated this in section 3.1 (Ln165-166), we will try to make it more explicit that the data augmentations are the same for all the models.  \n  \nUnder these conditions we believe that the current evaluation set is fair. Apart from the baselines, all the models are trained with the same number of data augmentations, that is RandAugment + (PGD attack/R-Warp/VOneBlock/R-Blur). Nevertheless, we would be happy to further discuss with the reviewer if there is something that we missed and would try to make the evaluation as fair as possible.\n\n**it's not clear to me why authors add noise as part of the foveation.**\n\nWe add noise to simulate the stochasticity in the responses of the biological neurons in the retina (Croner, et.al 1993). We seem to have omitted this citation from the paper, but we will update the paper to include it.\n\nCroner LJ, Purpura K, Kaplan E. Response variability in retinal ganglion cells of primates. Proc Natl Acad Sci U S A. 1993 Sep 1;90(17):8128-30. doi: 10.1073/pnas.90.17.8128. PMID: 8367474; PMCID: PMC47301.\n\n**Missing References (to add in line 32 and through-out the paper regarding links of biological vision (peripheral computation) + adversarial robustness): “Finding Biological Plausibility for Adversarially Robust Features via Metameric Tasks”. Harrington & Deza. ICLR 2022.** \n\nWe thank the reviewer for pointing us to this paper, we will cite this in the paper.\n    \n**Missing Reference in Line 51: I believe Pramod et al. did in fact perform robustness tests on their foveated adaptive blur image.**  \n\nWe were unable to find robustness experiments in [29]. We referred to the version on Arxiv which is also the one we cited, but we also looked at the version on PubMed. If we overlooked or misunderstood something in [29] we would be happy to be corrected.\n    \n\n**Missing Reference in Line 52: Characterizing a snapshot of perceptual experience. By Cohen et al. from the Journal of Experimental Pscyhology 2021.**  \n\nWe thank the reviewer for pointing us to this paper, we will cite this in the paper.\n\n**It would be quite nice if authors release a R-Blur augmentatino module in pytorch.**\nPyTorch code is included in the supplementary material and will be made public on GitHub after publication. \n\nWe hope that our responses fully address the reviewer's concerns and we kindly request the reviewer to consider increasing their score of our paper."}]

Author rebuttal:
We are very thankful to the reviewers for taking the time to read our paper carefully and providing valuable feedback that will undoubtedly help strengthen the paper and increase its impact. We also thank the reviewers for asking thoughtful questions and raising important concerns. 

We have responded to each reviewer’s questions and concerns in separate responses to their respective reviews. Due to space constraints we were unable to respond to comments about editorial issues, like formatting, typos and missing citations. Nevertheless, we would like to assure the reviewers that we acknowledge those comments and we will update the paper accordingly for the camera ready. To reduce ambiguity we answer each question/concern separately, quoting the reviewers' words (in bold font), either verbatim or summarized, before providing our response. If there are instances where our responses do not fully address the reviewers’ concerns or if it appears that we may have misunderstood the reviewers’ intent, we encourage the reviewers to ask follow-up questions and allow us the opportunity better understand their point of view and to provide additional details and clarification. If our responses do adequately address the reviewers’ concerns we request the reviewers to consider increasing their scores.  

We have also performed some additional experiments and analyses based on requests by the reviewers and have presented the results in the PDF attached to this response. The tables and figures in the PDF are referenced in the responses to the specific questions/comments that requested them. The list of tables and figures in the PDF is also listed below:

-   Table 1: Computation time of all the methods compared in this paper
    
-   Figure 1: Accuracy of R-Blur, baseline methods, adversarial training, non-adaptive Gaussian blur, and Gaussian noise at different levels of adversarial perturbation.
    
-   Figure 2: A corrected version of Figure 2 from the paper.
    
-   Figure 3: Accuracy of R-Blur under APGD and AutoAttack at different levels of adversarial perturbation.
    
-   Figure 4: Comparison of accuracy under clear images and adversarial attacks between R-Blur and models with Gaussian blurring of difference variances.
    
Notes:
-  We refer to the Appendix in our responses below. This appendix is included in the supplementary material submitted with the paper.


Meta-review: The paper proposes a biologically inspired approach to improve model robustness towards adversarial perturbations and common corruptions. It leverages fixation points as basis for data augmentation during training. Specifically, it mimics retina blur (thus the name R-Blur) and reduces the color saturation depending on the distance from the fixation point to simulate a foveated view on the respective images. Evaluations are provided on CIFAR-10, Ecoset, and ImageNet and show the benefit of the proposed approach.
The paper is well written and provides a well-motivated link between model robustness and human perception. While the reviewers had initial concerns on the validity of baselines (e.g. including Gaussian blurring and the number of augmented samples), these concerns could be clarified during the rebuttal and discussion phase. After the discussion, the paper has a clear accept vote from reviewers and the area chair. The authors are encouraged to include the additional results from the rebuttal phase as well as the discussion on the role of noise into their final submission.
Decision: Accept (poster)