Title: Stretching Beyond the Obvious: A Gradient-Free Framework to Unveil the Hidden Landscape of Visual Invariance
Primary area: applications to neuroscience & cognitive science
Keywords: vision, visual invariance, feature visualization, human-AI alignment, deep convolutional neural networks, NeuroAI, psychophysics, computational neuroscience, robustness, adversarial attacks, evolutionary algorithm, gradient-free optimization, machine learning

Abstract: Uncovering which feature combinations are encoded by high-level visual units is critical to understanding how images are transformed into representations that support recognition. While existing feature visualization approaches typically infer a unit's most exciting images, this is insufficient to reveal the manifold of transformations under which responses remain invariant, which is critical to generalization in vision.
Here we introduce Stretch-and-Squeeze (SnS), an unbiased, model-agnostic, and gradient-free framework to systematically characterize a unit’s maximally invariant stimuli, and its vulnerability to adversarial perturbations, in both biological and artificial visual systems. SnS frames these transformations as bi-objective optimization problems. To probe invariance, SnS seeks image perturbations that maximally alter (stretch) the representation of a reference stimulus in a given processing stage while preserving unit activation downstream (squeeze). To probe adversarial sensitivity, stretching and squeezing are reversed to maximally perturb unit activation while minimizing changes to an upstream representation.
Applied to CNNs, SnS revealed invariant transformations that were farther from a reference image in pixel-space than those produced by affine transformations, while more strongly preserving the target unit's response. The discovered invariant images differed dramatically depending on the stage of the image representation used for optimization: pixel-level changes primarily affected luminance and contrast, while stretching mid- and late-layer CNN representations altered texture and pose respectively. 
By measuring the extent to which the hierarchical invariant images obtained for robust (i.e., adversarially trained) networks were correctly classified by humans and other observer networks, we discovered a substantial drop in the interpretability of the invariant images when the representation was stretched at increasingly deep layers. Interestingly, the opposite trend was found for standard (i.e., not robustified) models, whose invariant images became increasingly more interpretable when stretching was applied to deeper layers. This indicates that adversarial training fails to increase the interpretability of high-level invariances, despite strong evidence of perceptual alignment between humans and robustified models at the pixel level. This demonstrates how SnS can be used as a powerful new tool to measure the alignment between artificial and biological vision.

TL;DR: Gradient-free optimization uncovers novel invariances in deep convolutional neural nets