# Robustness of Deep Learning Models for Vision Tasks
## 4 Brain-Inspired Neural Architectures against Adversarial Attacks
### Biological Mechanism Model  
To explore biological processes that could contribute to adversarial robustness, two mechanisms present in biological vision systems were proposed and tested against PGD attacks [159]. The first mechanism involves nonuniform spatial sampling of visual stimuli by the retina's photoreceptors, mimicking the exponential decrease in cones' density in the eccentric direction, as taken from [156]. The second mechanism is the variation in RFs with eccentricity to avoid aliasing [157; 158; 162; 163] in \(V1\) neurons. To realize the area of scale according to each eccentricity of \(V1\) neurons, the authors constructed fovea receptacles with five frequency bands and estimated a set of scale-space bands in \(V1\). Four models in Figure 29 were applied to the ResNet-20 and ResNet-18 architectures using the CIFAR-10 and ImageNet datasets, respectively. The first model was the standard CNN, and the second mechanism was described as \"coarse fixations\", which involves roughly focusing on different regions of the image, simulating the fixation effect in a standard CNN network, to achieve a cropping effect without flipping the image. The third model was \"retina fixation\" to extract nonuniform sampling. The final mechanism was \"cortical fixations\", where the ResNet architecture is separated into multiple independent branches to process the scales at each eccentricity.  
\begin{table}
\begin{tabular}{c c c c} \hline
**Activations + Entropy** & **Orig.** & **Adv\_L\_BFGS** & **Adv\_FGSM** \\ \hline Sigmoid (trained by BP) & 0.96 & 0.57 & 0.28 \\ \hline Sigmoid & 0.94 & 0.77 & 0.45 \\ \hline Threshold & 0.93 & 0.73 & 0.52 \\ \hline \(y=|x|\) & 0.94 & 0.78 & 0.53 \\ \hline
**Activations + 0-1 loss** & **Orig.** & **Adv\_L\_BFGS** & **Adv\_FGSM** \\ \hline Sigmoid & 0.84 & 0.76 & 0.58 \\ \hline Threshold & 0.83 & 0.72 & 0.57 \\ \hline \end{tabular}
\end{table}
Table 2: The results of adversarial tests for two hidden layers artificial neural network trained by the GLR with different activation and loss functions in [161].  
Figure 29: Standard CNN and its fixation models. The (**A**) was the standard CNN, and the (**B**) was coarse fixations, which involves roughly focusing on different regions of the image. The (**C**) was retina fixation to extract nonuniform sampling, and the (**D**) was cortical fixations, where the ResNet architecture is separated into multiple independent branches to process the scales at each eccentricity.  
In almost all experiments for all datasets, the results showed that the bio-inspired mechanisms improved the robustness against adversarial examples with minor modifications. While the specific model with the most improvement varied, both the retinal and cortical fixation models had a similar effect. When evaluated on ImageNet 10, the proposed model had a greater improvement in accuracy when facing PGD attacks compared to FGSM attacks. This implies that the mechanisms are effective in PGD attacks for improving robustness, with the retinal and cortical fixation models. However, as the perturbations grew, the bio-inspired mechanisms showed no notable improvement in robustness.