We introduce foveated perceptual systems, inspired by human biological systems, and examine the impact that this foveation stage has on the nature and robustness of subsequently learned visual representation. Specifically, these%DIFDELCMD < \textit{%%% two-stage perceptual systems first foveate an image , inducing a texture-like encoding of peripheral information, which is then inputted to a convolutional neural network (CNN)and trained to perform scene categorization   . We find that : 1-- Systems trained on foveated inputs (Foveation-Nets) have similar generalization as networks trained on matched-resource networks without foveated input(Standard-Nets), yet show greater cross-generalization.2-- Foveation-Nets show higher robustness than Standard-Nets to scotoma (fovea removed) occlusions, driven by the first foveation stage. 3-- Subsequent representations learned in the CNN of Foveation-Nets weigh center information more strongly than Standard-Nets. 4-- Foveation-Nets show less sensitivity to low-spatial frequency information than Standard-Nets. Furthermore, when we added biological and artificial augmentation mechanisms to each system through simulated eye-movements or random cropping and mirroring respectively, we found that these effects were amplified. Taken together, we find evidence that foveated perceptual systemslearn a visual representation that is distinct from non-foveated perceptual systems, with implications in generalization, robustness, and perceptual sensitivity. These results provide computational support for the idea that the foveated nature of the human visual system might confer a functional advantage for scene representation .
The goal of this work is to characterize the representational impact that foveation operations have for machine vision systems, inspired by %DIFDELCMD < \textit{%%% the foveated human visual system, which has higher acuity at the center of gaze and a texture-like encoding mechanism in the periphery. To do so, we introduce models consisting of a first-stagefixed image transform followed by a second-stagelearnable convolutional neural network, and we varied the first stage component. The primary model has a foveated-textural input stage, which we compare to a model with foveated-blurred input and a model with spatially-uniform blurred input (both matched for perceptual compression), and a final reference model with minimal input-based compression . We find that the model with foveated texture-based input -- most akin to the human visual system -- shows similar scene classification accuracy as the reference model despite its compressed input, with greater i.i.d. generalization than the other models. Second, the foveated-texture model maintains sensitivity to high-spatial frequency information and greater robustness to occlusion properties, similar to the reference, and unlike the comparison models. Finally, both the foveated systems, either texture or blur-based, show a stronger center image-bias relative to the spatially-uniform systems. These results demonstrate that foveation with peripheral texture-based computations yields an efficient, distinct, and robust representational format of scene information, and provides computational insight into the representational consequences that texture-based peripheral encoding may have for processing in the human visual system .
R
We introduce foveated perceptual
The goal of this work is to characterize the representational impact that foveation operations have for machine vision
D
human biological systems, and examine the impact that this foveation stage has on the nature and robustness of subsequently learned visual representation. Specifically, these
D
two-stage
R
perceptual systems first foveate an image , inducing a
the foveated human visual system, which has higher acuity at the center of gaze and a
R
encoding of peripheral information, which is then inputted to a convolutional neural network (CNN)and trained to perform scene categorization
encoding mechanism in the periphery. To do so, we introduce models consisting of a first-stage
A
fixed
A
image transform followed by a second-stage
A
learnable
A
convolutional neural network, and we varied the first stage component. The primary model has a foveated-textural input stage, which we compare to a model with foveated-blurred input and a model with spatially-uniform blurred input (both matched for perceptual compression), and a final reference model with minimal input-based compression
R
: 1-- Systems trained on foveated inputs (Foveation-Nets) have similar generalization as networks trained on matched-resource networks without foveated input(Standard-Nets), yet show greater cross-generalization.2-- Foveation-Nets show higher robustness than Standard-Nets to scotoma (fovea removed) occlusions, driven by the first foveation stage. 3-- Subsequent representations learned in the CNN of Foveation-Nets weigh center information more strongly than Standard-Nets. 4-- Foveation-Nets show less sensitivity to low-spatial frequency information than Standard-Nets. Furthermore, when we added biological and artificial augmentation mechanisms to each system through simulated eye-movements or random cropping and mirroring respectively, we found that these effects were amplified. Taken together, we find evidence that foveated perceptual systemslearn a visual representation that is distinct from non-foveated perceptual systems, with implications in generalization, robustness, and perceptual sensitivity. These results provide computational support for the idea that
the model with foveated texture-based input -- most akin to the human visual system -- shows similar scene classification accuracy as the reference model despite its compressed input, with greater i.i.d. generalization than
R
foveated nature of the
other models. Second, the foveated-texture model maintains sensitivity to high-spatial frequency information and greater robustness to occlusion properties, similar to the reference, and unlike the comparison models. Finally, both the foveated systems, either texture or blur-based, show a stronger center image-bias relative to the spatially-uniform systems. These results demonstrate that foveation with peripheral texture-based computations yields an efficient, distinct, and robust representational format of scene information, and provides computational insight into the representational consequences that texture-based peripheral encoding may have for processing in the
D
might confer a functional advantage for scene representation