F INDING B IOLOGICAL P LAUSIBILITY FOR A DVERSARIALLY R OBUST F EATURES VIA M ETAMERIC T ASKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Recent work suggests that feature constraints in the training datasets of deep neural networks (DNNs) drive robustness to adversarial noise
Ilyas et al.,
The representations learned by such adversarially robust networks have also been shown to be more human perceptually-aligned than non-robust networks via image manipulations (Santurkar et al., 2019; Engstrom et al., 2019).
Despite appearing closer to human visual perception, it is unclear if the constraints in robust
DNN representations match biological constraints found in human vision.
Human vision seems to rely on texture-based/summary statistic representations in the periphery, which have been shown to explain phenomena such as crowding
(Balas et al., 2009) and performance on visual search tasks
Rosenholtz et al., 2012)
To understand how adversarially robust optimizations/representations compare to human vision, we performed a psychophysics experiment using a metamer task similar to Freeman & Simoncelli (2011); Wallis et al.
; Deza et al. (2019b) where we evaluated how well human observers could distinguish between images synthesized to match adversarially robust representations compared to nonrobust representations and a texture synthesis model of peripheral vision (Texforms (Long et al., 2018)).
We found that the discriminability of robust representation and texture model images decreased to near chance performance as stimuli were presented farther in the periphery.
Moreover, performance on robust and texture-model images showed similar trends within participants, while performance on non-robust representations changed minimally across the visual ﬁeld.
These results together suggest that (1) adversarially robust representations capture peripheral computation better than non-robust representations and (2) robust representations capture peripheral computation similar to current state-of-the-art texture peripheral vision models.
More broadly, our ﬁndings support the idea that localized texture summary statistic representations may drive human invariance to adversarial perturbations and that the incorporation of such representations in
DNNs could give rise to useful properties like adversarial robustness.
I NTRODUCTION
Texture-based summary statistic models of the human periphery have been shown to explain key phenomena such as crowding (Balas et al., 2009; Freeman & Simoncelli, 2011) and performance on visual search tasks (Rosenholtz et al., 2012) when used to synthesize feature-matching images.
These analysis-by-synthesis models have also been used to explain mid-level visual computation ( e.g. V2) via perceptual discrimination tasks on images for humans and primates (Freeman & Simoncelli, 2011; Ziemba et al., 2016; Long et al., 2018).
However, while summary statistic models can succeed at explaining peripheral computation in humans, they fail to explain foveal computation and core object recognition that involve other representational strategies (Logothetis et al., 1995; Riesenhuber & Poggio, 1999; DiCarlo & Cox, 2007;
Hinton, 2021).
Modelling foveal vision with deep learning indeed has been the focus of nearly all object recognition systems in computer vision (as machines do not have a periphery)
(LeCun et al., 2015; Schmidhuber, 2015) – yet despite their overarching success in a plethora of tasks, they are vulnerable to adversarial perturbations.
This phenomena indicates: 1) a critical failure of current artiﬁcial systems (Goodfellow et al., 2015; Szegedy et al., 2014); and 2) a perceptual mis-alignment of such systems with humans (Golan et al., 2020;
Feather et al., 2019; Firestone, 2020; Geirhos
[Figure]
et al., 2021; Funke et al., 2021) – with some exceptions (Elsayed et al., 2018).
Indeed, there are many strategies to alleviate these sensitivities to perturbations, such as data-augmentation (Rebufﬁ et al., 2021; Gowal et al., 2021), biologically-plausible inductive biases (Dapello et al., 2020;
Reddy et al., 2020; Jonnalagadda et al., 2021), and adversarial training (Tsipras et al., 2019; Madry et al., 2017).
This last strategy in particular (adversarial training) is popular but has been criticized as being non-biologically plausible – despite yielding some perceptually aligned images when inverting their representations (Engstrom et al., 2019; Santurkar et al., 2019).
Motivated by prior work on summary statistic models of peripheral computation and their potential resemblance to inverted representations of adversarially trained networks we wondered if these two apparently disconnected phenomena from different ﬁelds share any similarities (See Figure 1).
Could it be that adversarially trained networks are robust because they encode object representations similar to human peripheral computation?
We know machines do not have peripheral computation (Azulay & Weiss, 2019; Deza & Konkle, 2020; Alsallakh et al., 2021), yet are susceptible to a type of adversarial attacks that humans are not.
We hypothesize that object representation arising in human peripheral computation holds a critical role for high level robust vision in perceptual systems, but testing this has not been done.
Thus, the challenge we now face is how to compare an adversarially trained neural network model to current models of peripheral/mid-level visual processing – and ultimately to human observers as the objective ground-truth.
However, determining such perceptual parameterizations is computationally intractable.
Inspired by recent works that test have tested summary statistic models via metameric discrimination tasks (Deza et al., 2019b; Wallis et al., 2016; 2017; 2019), we can evaluate how well the adversarially robust CNN model approximates the types of computations present in human peripheral vision with a set of rigorous psychophysical experiments wrt synthesized stimuli.
Our solution consists of performing a set of experiments where we will evaluate the rates of human perceptual discriminability as a function of retinal eccentricity across the synthesized stimuli from an adversarially trained network vs synthesized stimuli from models of mid-level/peripheral computation.
If the decay rates at which the perceptual discriminability across different stimuli are similar, then this would suggest that the transformations learned in an adversarially trained network are related to the transformations done by models of peripheral computation – and thus, to the human visual system.
It is worth noting that although adversarially robust representations have been shown to be more human-perceptually aligned (Ilyas et al., 2019; Engstrom et al., 2019; Santurkar et al., 2019), they still look quite different when placed in the foveal region from the original reference image (Feather et al., 2021).
However, our eccentricity-varying psychophysical experiments are motivated by empirical work that suggests that the human visual periphery represents input in a texture-like scrambled way, that can appear quite different than how information is processed in the fovea (Nandy & Tjan, 2012; Rosenholtz, 2016; Stewart et al., 2020; Herrera-Esposito et al.
[Figure]
localized texture- distortions similar physiologically for several phenomena of human peripheral computation (Freeman & Simoncelli, 2011; et al., 2012; Long et al., 2018).
Standard stimuli, which are images synthesized from a network with Regular (Non-Adversarial) training have no resemblance to the original sample.
In this paper we evaluate how similar these models are, via their derived stimuli, with a set of controlled human psychophysics experiments where we vary the retinal eccentricity of the stimuli.
S YNTHESIZING S TIMULI AS A WINDOW TO M ODEL R EPRESENTATION
Suppose we have the functions g Adv ( ◦ ) and g Standard ( ◦ ) that represent the adversarially trained and standard (non-adversarially) trained neural networks; how can we compare them to human peripheral computation if the function g Human ( ◦ ) is computationally intractable?
One solution is to take an analysis-by-synthesis approach and to synthesize a collection of stimuli that match the feature response of the model we’d like to analyze – this is also known as feature inversion (Mahendran & Vedaldi, 2015; Feather et al., 2019).
If the inverted features (stimuli) of two models are perceptually similar, then it is likely that the learned representations are also aligned.
For example, if we’d like to know what is the stimuli x (cid:48) that produces the same response to the stimuli x for a network g (cid:48) ( ◦ ) , we can perform the following minimization:
[Equation]
In doing so, we ﬁnd x (cid:48) which should be different from x for a non-trivial solution.
This is known as a metameric constraint for the stimuli pair { x, x 0 } wrt to the model g (cid:48) ( ◦ ) : g (cid:48)
( x ) =
g (cid:48)
( x (cid:48) )
s.t.
x
(cid:54) =
x (cid:48) for a starting pre-image x 0 that is usually white noise in the iterative minimization of Eq.1.
Indeed, for the adversarially trained network of Ilyas et al.
Engstrom et al.
Santurkar et al.
(2019), we can synthesize robust stimuli wrt to the original image x via:
[Equation]
which implies – if the minimization goes to zero – that:
[Equation]
Recalling the goal of this paper, we’d like to investigate if the following statement is true: “a transformation resembling peripheral computation in the human visual system can closely be approximated by an adversarially trained network” , which is formally translated as: g Adv ∼ g r ∗ Human for some retinal eccentricity ( r ∗ ) , then from Eq.
3 we can also derive:
[Equation]
However, g Human ( ◦ ) is computationally intractable, so how can we compute Eq.4?
A ﬁrst step is to perform a psychophysical experiment such that we ﬁnd a retinal eccentricity r ∗ at which human observers can not distinguish between the original and synthesized stimuli – thus behaviourally proving that the condition above holds, without the need to directly compute g Human .
More generally, we’d like to compare the psychometric functions between stimuli generated from a standard trained network (standard stimuli), an adversarially trained network (robust stimuli), and a model that captures peripheral and mid-level visual computation (texform stimuli (Freeman &
Simoncelli, 2011; Long et al., 2018)).
Then we will assess how the psychometric functions vary as a function of retinal eccentricity.
If there is signiﬁcant overlap between psychometric functions between one model wrt the model of peripheral computation; then this would suggest that the transformations developed by such model are similar to those of human peripheral computation.
We predict that this will be the case for the adversarially trained network ( g Adv ( ◦ )) .
Formally, for any model g , and its synthesized stimuli x g – as shown in Figure 2, we will deﬁne the psychometric function δ Human , which depends on the eccentricity r as:
[Equation]
) where we hope to ﬁnd:
[Equation]
2. S TANDARD AND R OBUST M ODEL S TIMULI
To evaluate robust vs non-robust feature representations, we used the ResNet-50 models of Santurkar et al.
; Ilyas et al.
Engstrom et al.
We used their models so that our results could be interpreted in the context of their ﬁndings that features may drive robustness.
Both models were trained on a subset of ImageNet (Russakovsky et al., 2015), termed Restricted ImageNet (Table 1).
The beneﬁt of Restricted ImageNet, stated by Ilyas et al.
; Engstrom et al., is models can achieve better standard accuracy than on all of ImageNet.
One drawback is that it is imbalanced across classes.
Although the class imbalance was not problematic for comparing the adversarially robust model to standard-trained one, we did ensure that there was a nearly equal number of images per class when selecting images for our stimulus set to avoid class effects in our experiment (i.e. people are better at discriminating dog examples than ﬁshes independent of the model training).
Using their readily available models, we synthesized robust and standard model stimuli using an image inversion procedure (Mahendran & Vedaldi, 2015; Gatys et al., 2015; Santurkar et al., 2019;
Engstrom et al., 2019; Ilyas et al., 2019).
We used gradient descent to minimize the difference between the representation of the second-to-last network layer of a target image and an initial noise seed as shown in Figure 9.
Target images were randomly chosen from the test set of Restricted
ImageNet.
We chose 100 target images for each of the 9 classes and synthesized a robust and standard stimulus for 2 different noise seeds.
5 target images were later removed as they were grayscale and could not also be rendered as Texforms with the same procedure as the majority.
All stimuli were synthesized at a size of 256 × 256 pixels, this was equivalent to 6 . 67 × 6 . 67 degrees of visual angle (d.v.a.)
when performing the psychophysical experiments (See A.6 for calculation).
2. T EXFORM S TIMULI
Texforms (Long et al., 2018) are object-equivalent rendered stimuli from the Freeman & Simoncelli (2011)
; Rosenholtz et al.
(2012) models that break the metameric constraint to test for mid-level visual representations in Humans.
These stimuli – initially inspired by the experiments of Balas et al.
(2009) – preserve the coarse global structure of the image and its localized texture statistics (Portilla & Simoncelli, 2000).
Critically, we use the texform stimuli – voiding the metameric constraint – as a perceptual control for the robust stimuli, as the texforms incarnate a sub-class of biologicallyplausible distortions that loosely resemble the mechanisms of human peripheral processing.
As the texform model has 2 main parameters which are the scaling factor s and the simulated point of ﬁxation z , we must perform a perceptual optimization procedure to ﬁnd the set of texforms ˆ x that match the robust stimuli ˜ x as close as possible (w.r.t to the original image) before testing their discriminability to human observers as a function of eccentricity.
To do this, we used the accelerated texform implementation of Deza et al. (2019a) and generated 45 texforms with the same collection of initial noise seeds as the robust stimuli to be used as perceptual controls.
Similar to Deza & Konkle
[Figure]
we minimize the perceptual dissimilarity Z to ﬁnd ( s ∗ , z ∗ ) over this subset of images that we will later use in the human psychophysics ( ∼ 900 texforms):
[Equation]
(7) for an image quality assessment (IQA) function Q ( ◦ , ◦ ) .
We selected DISTS in our perceptual optimization setup given that it is the IQA metric that is most tolerant to texture-based transformations (Ding et al., 2020; 2021).
A cartoon illustrating the texform rendering procedure, the perceptual optimization framework and the respective results can be seen in Figure 3.
In our ﬁnal experiments (See Next Section) we used texforms rendered with a simulated scale of 0.5 and horizontal simulated point of ﬁxation placed at 640 pixels.
Critically, this value is immutable and texforms (like robust stimuli) will not vary as a function of eccentricity to provide a fair discriminability control in the human psychophysics.
For a further discussion on texforms and their biological plausibility and/or synthesis procedure, please see Supplement A.2.
H UMAN P SYCHOPHYSICS : D ISCRIMINATING BETWEEN STIMULI AS A FUNCTION OF RETINAL ECCENTRICITY
We designed two human psychophysical experiments: the ﬁrst was a an oddity task similar to Wallis et al.
and the second was a matching, two-alternative forced choice task (2AFC).
Two different tasks were used to evaluate how subjects viewed synthesized images both only in the periphery (oddity) and those they saw in the fovea (matching 2AFC).
The oddity task consisted of ﬁnding the oddball stimuli out of a series of 3 stimuli shown peripherally one after the other (100ms) masked by empty intervals (500ms) while holding center ﬁxation.
Chance for the oddity task was 1 out of(33 .
The matching 2AFC task consisted of viewing a stimulus in the fovea (100ms) and then matching it to two candidate templates in the visual periphery (100 ms) while holding ﬁxation.
A 1000 ms mask was used in this experiment and chance was 50%.
For both experiments, we also had interleaved trials where observers had to engage in a Original stimuli vs Synthesized stimuli task, or a Synthesized stimuli vs Synthesized stimuli discrimination task (two stimulus pairs synthesized from different noise seeds to match model representations).
The goal of these experimental variations (called ‘stimulus roving’ ) was two-fold: 1) to add difﬁculty to
[Figure]
the tasks thus reducing the likelihood of ceiling effects; 2) to gather two psychometric functions per family of stimuli, which portrays a better description of each stimuli’s evoked perceptual signatures.
We had 12 participants complete both the oddity and matching 2AFC experiments.
The oddity task was always performed ﬁrst so that subjects would never have foveated on the images before seeing them in the periphery.
We had two stimulus conditions (1) robust & standard model images and (2) texforms.
Condition 1 consisted of the inverted representations of the adversarially robust and standard-trained models.
The two model representations were randomly interleaved since they were synthesized with the same procedure.
Condition 2 consisted of texforms synthesized with a ﬁxed and perceptually optimized ﬁxation and scaling factor which yielded images closest in structure to the robust representations at foveal viewing (robust features have no known ﬁxation and scaling – which is why partly we evaluate multiple points in the periphery.
Recall Figure 3).
We randomly assigned the order in which participants saw the different stimuli.
More details found in A.6.
The main results of our 2 experiments can be found in Figure 5, where we show how well Humans can discriminate per type of stimuli class and task.
Mainly, human observers achieve near perfect discrimination rates for the Standard stimuli wrt to their original references, but near chance levels when discriminating to another synthesized sample.
This occurs for both experimental paradigms (Oddity + 2AFC), suggesting that the network responsible for encoding standard stimuli is a poor model of human peripheral vision given no interaction with retinal eccentricity.
However, we observe that Humans show similar perceptual discriminability rates for Robust and
Texform stimuli – and that these vary in a similar way as a function of retinal eccentricity.
Indeed, for both of these stimuli their perceptual discrimination rates follow a sigmoidal decay-like curve
Eccentricity
Robust Standard
Texform
P r opo r t i on
C o rr e c t
Chance oddity
Chance 2AFC
N=12 A.
O dd i t y T
a s k 2 A F C T a
s k 5 10 15 20 25 30 0 0.
Eccentricity
Robust vs Texform
Chance oddity
Chance 2AFC
B. when comparing the stimuli to the original, and also synthesized samples.
The similarity between the blue and magenta curves from Figure 5 suggests that if the texform stimuli do capture some aspect of peripheral computation, then – by transitivity – so do the adversarial stimuli which were rendered from an adversarially trained network.
These results empirically verify our initial hypothesis that we set out to test in this paper.
A superposition of these results in reference to the Robust stimuli for a better interpretation can also be seen in Figure 5 (B.).
3. S IMULATED F OVEA /P ERIPHERY I MAGE Q UALITY A SSESSMENT (IQA) ACROSS STIMULI
Some distortions are more perceptually noticeable than others for humans and neural networks (Berardino et al., 2017; Martinez-Garcia et al., 2019) – so how do we assess which model better accounts for peripheral computation, if there are many distortions (derived from the synthesized model stimuli) that can potentially yield the same perceptual sensitivity in a discrimination task?
Our approach consists of computing two IQA metrics (DISTS & MSE) over the entire psychophysical testing set over 2 opposite levels of a Gaussian Pyramid decomposition (Burt & Adelson, 1987).
This procedure checks which stimuli presents the greatest distortion (MSE), and yet yields greater perceptual invariance (DISTS).
A Gaussian Pyramid decomposition was selected as it stimulates the frequencies preserved given changes in human contrast sensitivity and cortical magniﬁcation factor from fovea to periphery (Anstis, 1974; Geisler & Perry, 1998).
These two metrics were one that is texture-tolerant and perceptually aligned (DISTS), and another that is a non-perceptually aligned metric: Mean Square Error (MSE).
Both IQA metrics were computed in pixel space for both the
Original vs Synthesized and Synthesized vs Synthesized conditions.
Results are explained in Figure 6, where Standard Stimuli yields low perceptual invariance to the original image at both levels of the Gaussian Pyramid, but robust and texform stimuli have a similar degree of perceptual invariance.
Critically, robust stimuli are slightly more distorted via MSE than texform stimuli suggesting that the adversarially trained model has learned to represent peripheral computation better than the texform model by maximizing the perceptual null space and throwing away more useless low-level image features (hence achieving greater Mean Square Error).
[Figure]
D I S T S
O r i g i n a l v s S y n t
h e s i z
e d
S y n t
h e s i
z
e d v s
S y n t
h e s i z
e d
D I S T S
[Figure]
D i r e c t i o n o f
G
r
e a t
[Figure]
r
P [Figure] r
[Figure]
e p t
[Figure]
a l I n [Figure] a r i a n [Figure]
[Figure]
r
[Figure]
r d T r a i n i
[Figure]
the Standard trained network (that was used to synthesize the Standard stimuli), both on a family of different stimuli: Original, Standard, Robust and Texform.
The Adversarially trained network – similar to the human – can not distinguish between 2-class Standard
Stimuli (unlike the Standard Network that has a near perfect 2-class hit rate).
Most importantly, the
Adversarially trained network yields a near double hit rate on Texform classiﬁcation wrt the Standard trained network.
This suggests that the Adversarially trained network has a representation that is more perceptually aligned to models of Peripheral Computation than the Standard trained model.
D ISCUSSION
One of the fundamental questions in modern computer vision is understanding what are the principles that give rise to adversarial stimuli – which show a striking perceptual divergence between man and machine (Feather et al., 2019; Golan et al., 2020; Geirhos et al., 2021;
Funke et al., 2021).
While it may seem theoretically impossible to escape from adversarial stimuli (Gilmer et al., 2018) – perhaps our efforts in the community should focus on understanding the biologically plausible mechanisms (if any) of the solutions that grant some level of adversarial robustness aligned with human error.
In this paper we focused on potentially linking the representations learned from an adversarially trained network and human peripheral computation via a series of psychophysical experiments through a family of stimuli synthesized from these models.
[Figure]
We found that stimuli synthesized from an adversarially trained (and thus robust) network are metameric to the original stimuli in the further periphery (slightly above 30 deg ) for both Oddity and 2AFC Matching tasks.
However, more important than deriving a critical eccentricity for metameric gaurantees across stimuli in Humans – we found a surprisingly similar pattern of results in terms of how perceptual discrimination interacts with retinal eccentricity when comparing the adversarially trained network’s robust stimuli with classical models of peripheral computation and
V2 encoding (mid-level vision) that were used to render the texform stimuli (Freeman & Simoncelli, 2011; Long et al., 2018; Ziemba et al., 2016; Ziemba & Simoncelli, 2021).
Further, this type of eccentricity-driven interaction does not occur for stimuli derived from non-adversarially trained (standard) networks.
More generally, now that we found that adversarially trained networks encode a similar class of transformations that occur in the visual periphery – how do we reconcile with the fact that adversarial training is biologically implausible in humans?
Recall from the work of Ilyas et al.
that performing standard training on robust images yielded similar generalization and adversarial robustness as performing adversarial training on standard images; how does this connect then to human learning if we assume a uniform learning rule in the fovea and the periphery?
We think the answer lies in the fact that as humans learn to perform object recognition, they not only ﬁxate at the target image, but they also look around, and can eventually learn where to make a saccade given candidate object peripheral template – thus learning certain invariances when the object is placed both in the fovea and the periphery (Cox et al., 2005; Williams et al., 2008; Poggio et al., 2014; Han et al., 2020).
This is an idea that dates back to Von Helmholtz (1867) as highlighted in Stewart et al.
(2020) on the interacting mechanisms of foveal and peripheral vision in humans.
Altogether, this could suggest that spatially-uniform high-resolution processing is redundant and sub-optimal in the o.o.d.
regime in the way that the visual representation which is computed is independent of point of ﬁxation – as seen classically in adversarially-vulnerable CNNs that are translation invariant and have no foveated/spatially-adaptive computation.
Counter-intuitively, the fact that our visual system is spatially-adaptive could give rise to a more robust encoding mechanism of the visual stimulus as observers can encode a distribution rather than a point as they move their center of gaze.
Naturally, from all the possible types of transformations, the ones that are similar to those shown in this paper – which loosely resemble localized texture-computation – are the ones that potentially lead to a robust hyper-plane during learning for the observer (See Fig.
7 and 8).
Finally, we’d like to add a disclaimer – we use the term biological plausibility at a representational level through-out this paper.
However, current work is looking into reproducing the experiments carried out in this paper with a physiological component to explore temporal dynamics (MEG) and localization (fMRI) evoked from the stimuli.
While it is not obvious if we will ﬁnd a perceptual signature of the adversarial robust stimuli in humans, we think this novel stimuli and experimental paradigm presents a ﬁrst step towards the road of linking what is known (and unknown) across texture representation, peripheral computation, and adversarial robustness in humans and machines.