SlowGuess/NeurIPS_2023_OCR huggingface dataset:
These results are meant to lay the groundwork for future work toward developing methods for determining the optimal fixation point based on the input image. However, they also illustrate that models trained with  $R$ -Blur learn features that are not only more adversarially robust features than ResNet but also allow the model to make highly accurate predictions on clean data.
C Evaluations With Different Architectures
To demonstrate that the benefits of  $R$ -Blur are not limited to CNNs, we trained MLP-Mixer [55] and ViT [56] models with  $R$ -Blur preprocessing and evaluated their robustness. We use the configuration of MLP-Mixer referred to as S16 in [55]. Our ViT has a similar configuration, with 8 layers each having a hidden size of 512, an intermediate size of 2048, and 8 self-attention heads. We train both models with a batch size of 128 for 60 epochs on Ecoset-10 using the Adam optimizer. The learning
Figure 14: This figure indicates the locations of the optimal fixation points for some sample images. Each square in the grid corresponds to one of 49 fixation locations and represents the highest resolution region of the image if the model fixates at the center of the square. Squares that are shaded green indicate that the model's prediction at the corresponding fixation point was correct, while squares shaded red indicate that the model's prediction at the corresponding fixation point was incorrect. We see that there are certain images in which there are only a few optimal fixation points and they may not be in the center or in the corners of the image.
rate of the optimizer is linearly increased to 0.001 over 12 epochs and is decayed linearly to almost zero over the remaining epochs. The results are shown in Figure 15.
We observe that  $R$ -Blur significantly improves the robustness of MLP-Mixer models, and achieves greater accuracy than  $R$ -Warp at higher levels of perturbations. These results show that the robustness endowed to ResNets by  $R$ -Blur was not dependent on the model architecture, and they further strengthen our claim that loss in fidelity due to foveation contributes to the robustness of human and computer vision.
D Breakdown of Accuracy Against Common Corruption by Corruption Type
In Figure 16 we break down the performance of the models on common corruptions by higher-level corruption categories. The individual members of each category are listed in Table 2. We see that in most of the categories,  $R$ -Blur achieves the highest median accuracy against the most severe corruptions. We also note that  $R$ -Blur exhibits a remarkable degree of robustness to noise, which is substantially greater than all the other models we evaluated. It is pertinent to note here that Gaussian noise was just 1 of the 4 types of noise included in the noise category, and thus the performance of  $R$ -Blur can not be attributed to overfitting on Gaussian noise during training. Furthermore, robustness to
(b) MLP-Mixer
Figure 15: The accuracy obtained on Ecoset-10 against adversarial perturbations of various  $\ell_{\infty}$  norms when  $R$ -Blur is used with ResNet, MLP-Mixer and ViT backbones.
(c) ViT
Noise	Blur	Weather	Digital
gaussian noise	defocus blur	snow	contrast
shot noise	glass blur	frost	elastic transform
impulse noise	motion blur	fog	pixelate
speckle noise	zoom blur	brightness	jpeg compression
gaussian blur	spatter	saturate
Table 2: Categories of corruptions used to evaluate robustness to common corruptions. This categorization follows the one from [38]
one type of random noise does not typically generalize to other types of random noise [4]. Therefore, the fact that  $R$ -Blur exhibits improved robustness to multiple types of noise indicates that it is not just training on Gaussian noise, but rather the synergy of all the components of  $R$ -Blur that is likely the source of its superior robustness.
E Sensitivity Analysis of Hyperparameters in  $R$ -Blur
To measure the influence of the various Hyperparameters of  $R$ -Blur we conduct a sensitivity analysis. First, we vary the scale of the Gaussian noise added to the image, the viewing distance during inference, and the value of  $\beta$  from Section 2.5, which is the scaling factor that maps eccentricity (see equation 1 to standard deviation, and measure the impact on accuracy on clean as well as adversarially perturbed data. The results of this analysis are presented in Figure 17. We see that, as expected, increasing the scale of the noise improves accuracy on adversarially perturbed data, however, this improvement does not significantly degrade clean accuracy. It appears that the adaptive blurring is mitigating the deleterious impact of Gaussian noise on clean accuracy. On the other hand, increasing  $\beta$  beyond 0.01 surprisingly does not have a significant impact on accuracy and robustness. We also measured the accuracy on clean and perturbed data after varying the viewing distance (see 2.4) and the number of fixation points over which the logits are aggregated. These results are plotted in Figure 18, and they show that accuracy on clean and perturbed data is maximized when the width of the in-focus region is 48 (this corresponds to  $vd = 3$ ) and aggregating over more fixation points improves accuracy on clean and perturbed data.
F Training Configuration
Table 3 presents the configurations used to train the models used in our evaluation. For all the models the SGD optimizer was used with Nesterov momentum=0.9.
We used Pytorch v1.11 and Python 3.9.12 to for our implementation. We used the implementation of Auto-PGD from the Torchattacks library (https://github.com/Harry24k/adversarial-attacks-pytorch). For  $R$ -Warp we used the code from the official repo https://github.com/mvuyyuru/adversary.git. Likewise, for VOneBlock we used the code from https://github.com/dicarlolab/vonenet, and
(b) Ecoset
(c) blur
(d) noise
(g) CIFAR-10
(e) weather
(f) digital
Figure 17: The impact of the hyperparameters of  $R$ -Blur on the accuracy and robustness of models trained on Ecoset-10. (left) the standard deviation of Gaussian noise, and (right)  $\beta$  from Section 2.5.
Figure 16: The accuracy achieved by  $R$ -Blur and baselines on various classes of common corruptions, proposed in [38]. The boxplot shows the distribution of accuracy values on 4-5 different corruptions in each class applied at different severity levels (x-axis) with 1 referring to least severe and 5 being the most severe corruption.  $R$ -Blur generally achieves the highest median accuracy on the highest severity levels.
for DeepGaze-III models we used the code from https://github.com/matthias-k/DeepGaze. The training code for DeepGaze-III with  $R$ -Blur and  $R$ -Warp backbones is based on https://github.com/matthias-k/DeepGaze/blob/main/train_deepgaze3.ipynb, and can be found in adversarialML/biologically-inspired_models/src/fixation_prediction/train_deepgaze.py. Our clones of these repositories are included in the supplementary material. For multi-gpu training, we used Pytorch Lightning v1.7.6. We used 16-bit mixed precision training to train most of our models. The code for  $R$ -Blur can be found in adversarialML/biologically-inspired_models/src/retina_preproc.py which is part of the supplemental material.
Figure 18: The impact of the size of the in-focus region by varying the viewing distance (left) and the number of fixation points over which the logits are aggregated (right) on accuracy. The plots are computed from a  $R$ -Blur model trained on Imagenet, and the perturbed data is obtained by conducting a 25-step APGD attack with  $\| \delta \|_{\infty} = 0.004$ . We see that accuracy on clean and perturbed data is maximized when the width of the in-focus region is 48 (this corresponds to  $vd = 3$ ) and aggregating over more fixation points improves accuracy on clean and perturbed data.
Dataset	Method	Batch Size	nEpochs	LR	LR-Schedule	Weight Decay	nGPUs
CIFAR-10	ResNet	128	0.4	60	L-Warmup-Decay(0.2)	5e-5	1
AT	128	0.4	60	L-Warmup-Decay(0.2)	5e-5	1
R-Warp	128	0.4	60	L-Warmup-Decay(0.2)	5e-5	1
R-Blur	128	0.4	60	L-Warmup-Decay(0.2)	5e-5	1
G-Noise	128	0.4	60	L-Warmup-Decay(0.2)	5e-5	1
Ecoset-10	ResNet	128	0.4	60	L-Warmup-Decay(0.2)	5e-4	1
AT	128	0.4	60	L-Warmup-Decay(0.2)	5e-4	1
R-Warp	128	0.4	60	L-Warmup-Decay(0.2)	5e-4	1
R-Blur	128	0.1	60	L-Warmup-Decay(0.1)	5e-4	1
VOneBlock	128	0.1	60	L-Warmup-Decay(0.1)	5e-4	1
G-Noise	128	0.4	60	L-Warmup-Decay(0.2)	5e-4	1
Ecoset	ResNet	256	0.2	25	L-Warmup-Decay(0.2)	5e-4	2
AT	256	0.2	25	L-Warmup-Decay(0.2)	5e-4	4
R-Warp	256	0.1	25	L-Warmup-Decay(0.2)	5e-4	4
R-Blur	256	0.1	25	C-Warmup-2xDecay(0.1)	5e-4	4
VOneBlock	256	0.1	25	C-Warmup-2xDecay(0.1)	5e-4	4
G-Noise	256	0.1	25	C-Warmup-2xDecay(0.1)	5e-4	4
Imagenet	ResNet	256	0.2	25	L-Warmup-Decay(0.2)	5e-4	2
AT	256	0.2	25	L-Warmup-Decay(0.2)	5e-4	4
R-Warp	256	0.1	25	L-Warmup-Decay(0.2)
R-Blur	256	0.1	25	C-Warmup-2xDecay(0.1)	5e-4	4
VOneBlock	256	0.1	25	C-Warmup-2xDecay(0.1)	5e-4	4
G-Noise	256	0.1	25	D-Warmup-2xDecay(0.1)	5e-4	4
Table 3: The configurations used to train the models used in our evaluation. L-Warmup-Decay  $(f)$  represents a schedule that linearly warms up and decays the learning rate and  $f$  represents the fraction of iterations devoted to warmup. C-Warmup-2xDecay(0.1) is similar except that the warmup and decay follow a cosine function, and there are two decay phases. Both the schedulers are implemented using torch optim.lr_scheduler.OneCycleLR from Pytorch.
Method	Dynamic Fixation	# Fixations	Train Speed (img/s)	Test Speed (img/s)
ResNet	X	1	410	370
AT	X	1	232 (1.8×)	-
VOneBlock	X	1	277 (1.5×)	289 (1.3×)
R-Warp	X	1	377 (1.1×)	314 (1.2×)
R-Blur	X	1	369 (1.1×)	334 (1.1×)
R-Blur	X	5	-	111 (3.3×)
R-Warp	✓	5	-	26 (14.2×)
R-Blur	✓	1	-	115 (3.2×)
R-Blur	✓	5	-	29 (12.9×)
Table 4: Training and inference speed on 1 Nvidia 2080Ti, measure in images per second, for each model. The value in the parentheses indicates the slowdown relative to the unmodified ResNet computed as the (train/test) speed of the ResNet divided by the speed of the other method. We see that  $R$ -Blur, in and of itself, causes a very minimal slowdown (only  $1.1 \times$ ) during training and testing. Increasing the number of fixations slows R-Blur only sub-linearly (5 predefined fixations lead to 3x slowdown). Introducing dynamic fixation prediction has a greater impact on speed because each image is assigned different fixation points and so R-Blur/R-Warp can not be applied to them as a single batch. This shortcoming is likely common to most fixation transforms, and is not unique to R-Blur.
H Hardware Details and Computation Cost
We trained our models on compute clusters with Nvidia GeForce 2080 Ti and V100 GPUs. Most of the Imagenet and Ecoset models were trained and evaluated on the V100s, while the CIFAR-10 and Ecoset-10 models were trained and evaluated on the 2080 Ti's.
H.1 Analysis of Computation Cost
Table 4 presents this comparison and shows that R-Blur causes minimal slowdown (1.1x compared to the vanilla ResNet) during both training and testing. Also, increasing the number of fixations slows R-Blur only sub-linearly (5 predefined fixations  $\Rightarrow$  3x slowdown). Introducing dynamic fixation prediction has a greater impact on speed because each image is assigned different fixation points and so R-Blur/R-Warp can not be applied to them as a single batch. This shortcoming is likely common to most fixation transforms, and is not unique to R-Blur. In fact, under dynamic fixation prediction, R-Blur is faster than R-Warp.