Discriminating image representations with principal distortions
Authors: Jenelle Feather
David Lipshutz
Sarah E Harvey
Alex H Williams
Eero P Simoncelli
Venue: ICLR 2025 Poster (2025)
Keywords: representational similarity metric; Fisher information; information geometry; perception
Primary area: applications to neuroscience & cognitive science
TL;DR: We propose a metric for comparing the local geometry of image representations and use it to synthesize image distortions that separate models.

Abstract: Image representations (artificial or biological) are often compared in terms of their global geometric structure; however, representations with similar global structure can have strikingly different local geometries. Here, we propose a framework for comparing a set of image representations in terms of their local geometries. We quantify the local geometry of a representation using the Fisher information matrix, a standard statistical tool for characterizing the sensitivity to local stimulus distortions, and use this as a substrate for a metric on the local geometry in the vicinity of a base image. This metric may then be used to optimally differentiate a set of models, by finding a pair of "principal distortions" that maximize the variance of the models under this metric. As an example, we use this framework to compare a set of simple models of the early visual system, identifying a novel set of image distortions that allow immediate comparison of the models by visual inspection. In a second example, we apply our method to a set of deep neural network models and reveal differences in the local geometry that arise due to architecture and training types. These examples demonstrate how our framework can be used to probe for informative differences in local sensitivities between complex models, and suggest how it could be used to compare model representations with human perception.

Published as a conference paper at ICLR 2025
DISCRIMINATING IMAGE REPRESENTATIONS WITH PRINCIPAL DISTORTIONS
Jenelle Feather*1
David Lipshutz
Sarah E. Harvey!
Alex H. Williams 1.3
Eero P. Simoncelli
1 Center for Computational Neuroscience, Flatiron Institute, Simons Foundation
2 DDepartment of Neuroscience, Baylor College of Medicine Centee for Neural Science, New York University
ABSTRACT
Image representations (artificial biological) are often compared in terms of their global geometric structure; however, representations with similar global structure can have strikingly different local geometries. Here, we propose framework for comparing set of image representations in terms of their local geometries. We quantify the local geometry of a representation using the Fisher information ma- trix, a standard statistical tool for characterizing the sensitivity to local stimulus distortions, and use this as a substrate for metric on the local geometry the vicinity of base image. This metric may then be used to optimally differenti- ate set of models, by finding a pair of "principal distortions" that maximize the variance of the models under this metric. As example, we use this framework to compare a set of simple models of the early visual system, identifying a novel set of image distortions that allow immediate comparison of the models by visual inspection. In a second example, we apply our method to a set of deep neural net- work models and reveal differences in the local geometry that arise due to archi- tecture and training types. These examples demonstrate how our framework can be used probe for informative differences in local sensitivities between complex models, and suggest how it could be used to compare model representations with human perception.
INTRODUCTION
Biological and artificial neural networks transform sensory stimuli into high-dimensional internal representations that support downstream tasks, and these representations are often described in terms of their neural population geometry (Chung & Abbott. 2021). This idea has led to multitude of pro- posed measures of representational similarity (Kriegeskorte et al., 2008; Yamins & DiCarlo, 2016; Kornblith et al., 2019; Williams et al., 2021; Klabunde et al., 2023), which have been used to com- pare representations within a computational model the representations within a brain. However, despite differing architectures and training procedures, many computational models of perceptual or neural responses are equally performant on these representational similarity measures (Schrimpf et al., 2018; Tuckute et al., 2023; Conwell et al., 2024). Are these models functionally interchange- able, or are the datasets and methods that are used to test them simply insufficient to reveal their differences?
Often the similarity between two representations is quantified by measuring alignment of the rep- resentations
of natural stimuli that are relatively far apart in stimulus space. In this way, these measures capture notions of global geometric similarity between representations. However, systems with similar global structure can have strikingly different local geometries. These local geometries can be investigated by measuring the sensitivity of system to small image distortions. For example, Szegedy et al. (2013) found image distortions that were imperceptible to humans but
"Equll contribution, ordered alphahetically. Correspondence: jfeather fl ron nst tute. org, david. ppshutzuben edu.

---

Published as a conference paper at ICLR 2025
A
Model Representations
Level sets of FIMs
Log ratios sensitivities
p(rA|s + 62) (ra(s)
Eigen-distortions (Berardino 2017) teat modela sequentially
d
log d(ca)
Model A
5 p((n
choose oistortions that maximize magnitude
Stimulus Space
61
Model B
d
Generalized Eigen-distortions (Zhou et al. 2023) compare models
-1 d(Ei)
log
d((a)
chonse distortions that maximize separation
unit
distortions
Model C
Principal distortions (this paper) compare models
-2 d
log d(ea)
choose diisoorions that maximize variance
Figure I: Comparing the local geometry of image representations. A) Each model is assumed to map stimuli to stochastic responses in representation detee--dttmmiiistic models be inves- tigated by assuming additive Gaussian response noise. For example, Model A maps the stimulus (solid black circle +) to a conditional density p(r is) in Model A's representation space (solid blue circle and surrounding transparent blue ellipse). B) Model sensitivity at the base image s to local distortions can be mapped back to the stimulus domain via the model's positive semidefinite FIM. In the top panel "Eigen-diste diigen-distortions (Berardino et al., 2017)", the blue ellipse represents the unit level set (v (n) = }} of the norm induced by Model A's FIM LA. which the set of distortions of the base stimulus s that appear equally distorted according to Model A's representation. The cigenvectors of the FIM (E1, E2) can equivalently be interpreted as the distortions that maximize the magnitude of the log ratio of the model's sensitivities (solid blue circle middle panel "Generalized Eigen-distortions (Zhou et al., 2023)", the blue ellipse is copied from the top panel and the orange ellipse is the level set of Model B's FIM Ig. The generalized eigenvectors of IA and I (cI, E2) can equivalently be interpreted as the vectors that maximize the difference between the log ratios of the models' sensitivities. In the bottom panel "Principal distortions (this paper)", the blue and orange ellipses are as in the above panels and the green ellipse represents the level set of Model C's FIM Ic. Here, we see two stimulus distortions (E₁, E2) that maximize the variance of the log ratios of model sensitivities.
the number line). In the
led artificial neural networks to misclassify images, which motivated methods for training artificial neural networks so as to minimize their susceptibility to these adversarial examples (Goodfellow al., 2014; Madry, 2017). These observations suggest need for metrics that compare the local ge- ometries of image representations and, in particular, highlight the differences between systems even when global structure seems similar. Furthermore, such metrics may also aid the development of network interpretability and explainability tools.
How can we quantify and compare the local geometry of different image representations? brute- force comparison clearly is prohibitive: the space of images is extremely high- nigh-dimensional, and the set of potential distortions equally high-dimensional. Estimating the local geometry of representa- tions over moderately dense sampling of this full set of possible distortions is impractical, and estimating human sensitivity to such set is essentially impossible. As such, it is worthwhile to develop method for judicious selection of stimulus distortions that can be used when comparing set of models.
We take inspiration from Zhou et al. (2023). For pair of models and base image, they synthesize distortions along which the two models' sensitivities maximally disagree. This bears conceptual similarity to other methods that construct stimuli to optimally distinguish pair of models (Wang & Simoncelli, 2008; Golan et al., 2020) or cluster neurons by cell type (Burg et al., 2024), and builds on earlier work that examined "eigen-distortions" along which individual models are maxi-
2

---

Published as a conference paper at ICLR 2025
mally/minimally sensitive (Berardino et al., 2017). Specifically, Zhou et al. measure the local sen- sitivity of a model in terms of its Fisher Information matrix (FIM, Fisher, 1925), a classical tool from statistical estimation theory, and choose the pair of "generalized eigen-distortions" that max- imize/minimize the ratio of the two models' sensitivities. Once these image distortions have been computed, they may be added in varying amounts to base image to determine the level which they become visible to a human. These measured human sensitivities can then be compared to those of the models, with the goal of identifying which model is better aligned with the local geometry of the human visual system. The distortions can also be used as model interpretability tools. For instance, visualizations of the distortions may offer insight into how complex models systematically differ, providing practitioners with better understanding of model vulnerabilities. However, there is no principled method for selecting image distortions for comparing more than two models.
Here we define novel metric for comparing model representations in terms of their relative sensi- tivities to image distortions. We then use this metric to generate pair of distortions that maximize the variance across two more models under this metric. In analogy with principal component analysis, our method can be viewed as a dimensionality reduction technique that preserves as much of the variability in the local representational geometry as possible. As such, we refer to these the "principal distortions" of the set of models.
We apply our method to nested set of hand-crafted models of the early visual system to identify distortions that differentiate these models and can potentially be used to evaluate how well these models predict human visual sensitivities. We then apply method networks (DNNs) with varying architectures and training procedures, We find distortions that allow for visualization of differences the sensitivities between layers of the networks and neural network architectures. We further explore differences between standard ImageNet trained networks and their shape-bias enhanced counterparts, and between standard networks and their adversarially trained counterparts. In all cases, we illustrate how the method generates novel distortions that highlight differences between models. Portions of this work were initially presented in (Lipshutz* et al., 2024).
set of visual deep neural
2 PROBLEM STATEMENT AND EXISTING METHODS
Given collection of image representations, our goal is develop method for comparing the local geometries of these representations in the vicinity of some base input image. this section, define the local geometry of an image representation in terms of the FIM and review existing methods for selecting image distortions based on model FIMs.
2.1 LOCAL INFORMATION GEOMETRY OF STOCHASTIIC IMAGE REPRESENTATIONS
We assume that each image representation has an associated conditional density p(r[s), where s is K K-dimensional vector of image pixels and is a vector of stochastic responses (e.g.. biological neuronal firing rates or deterministic model activations with additive response noise). Fig. 1A de- picts a two-dimensional stimulus space and three models mapping stimuli + 61. and s + e2 to conditional densities. Note that the dimensions of the responses may vary across representations.
The sensitivity of the representation to a small distortion depends on the overlap between the conditional distributions p(rls) and p(ris + e), with less overlap corresponding to higher sensitivity (Green & Swets, 1966). This sensitivity can be precisely quantified terms of the Fisher-Rao metric (Rao, 1945; Amari, 2016), a Riemannian metric on the stimulus space (Fig. 1B) that is defined in terms of the positive semi-definite FIM (Fisher, 1925):
I (s) = Er-p(rls) [V logp(r(s)) log p(rl())
where V. log p(rls) denotes the gradient of logs(r(ss with respect to s. The FIM is standard tool in statistical estimation theory that locally approximates the expected log-likelihood ratio (or KL divergence) between the conditional distributions p(r(s) and p(r(s + E). and provides lower bound on the variance of any unbiased estimator of s (Cramér, 1946; Rao, 1945). The FIM has also been used to link neural representations to perceptual discrimination (Paradiso, 1988: Seung & Sompolinsky, 1993; Brunel & Nadal, 1998; Averbeck & Lee, 2006; Seriès et al., 2009; Ganguli & Simoncelli, 2014; Wei & Stocker, 2016). Given the FIM, we can define the sensitivity of the
3

---

Published as a conference paper at ICLR 2025
representation of stimulus to a distortion
d(s:c) Ve (()),,
(1)
which quantifies how well an ideal observer could detect small perturbations of the base stimulus in the direction
As tractable example, suppose that the conditional response r is Gaussian with stimulus-dependdnt mean (s) and constant covariance Σ: that is, p(r(s) (s),, 2). Then the FIM
T(s) = J
where (s) is the Jacobian of f-) representation to a distortion e depend on how the mean response is changing in the direction e relative the noise covariance. This example is relevant experimental results, where the function (s) denotes the output of deterministic model evaluated at a stimulus s.
From this expression, we see that the sensitivities of
2.2 EIGEN-DISTORTIONS OF AN IMAGE REPRESENTATION
Given a model image representation, Berardino et al. (2017) proposed the use of extremal eigen- vectors of the model FIM (termed "eigen-distortions"" Fig. IB, top panel) as model predictions of the most- and reast-noticeable image distortions. For set of early vision models and deep neu- ral networks whose parameters were optimized to match database of human image quality ratings (Ponomarenko et al., 2009), they computed the eigen-distortions each model. Despite the fact that these models all fit the image quality data equally well, their rigen-distortions were quite different, and human perceptual judgements of the severity of these gegen-iistortions varied substantially. The eggen-distortions of each image representation correspond to its most distinct distortion predictions, which can then be tested with human perceptual experiments. However, the eigen-distortions of two models similar, they will not be useful in distinguishing the models, since this method is insensitive to differences the non-extremal eigenvectors.
2.3 GENERALIZED BIGEN-DISTORTIONS FOR COMPARING TWO IMAGE REPRESENTATIONS
Zhou al. (2023) proposed comparing two image representations A and along distortions in which their local sensitivities maximally differ, which is conceptually similar to previous meth- ods that construct stimuli that maximize disagreement between models (Wang Simoncelli, 2008; Golan et al., 2020). Specifically, they chose distortions to extremize the generalized Rayleigh quo- tient:
u [ u (s(s)
E (s) arg V IA(S)V
e (s)s arg max
v IB(s)V
Since these distortions correspond to the extremal eigenvectors of the generalized eigenvalue prob- lem (s)E (s) we refer to them as "generalized eigen-distortions (Fig. IB, middle panel). However, this method limited to comparisons of pairs of models, or of single model against the average of other models.
3 PRINCIPAL DISTORTIONS OF IMAGE REPRESENTATIONS
We propose a natural extension of generalized eigen-distortions that allows for comparisons among more than two image representations. We show that the generalized eigenvalue problem suggests metric on the local geometry of image representations, which can be used to optimally choose image distortions that distinguish more than two models at base stimulus.
3.1 METRIC ON THE LOCAL GEOMETRY OF IMAGE REPRESENTATIONS
We can re-express the generalized eigen-distortions defined in Equation 2 as the solutions of single optimization problem:
log du(u) d d
log dv((v
{2), = arg max
u.v
4

---

Published as a conference paper at ICLR 2025
where we' ve used the definition of the sensitivity in equation 1 and simplified notation by omitting the dependence of sensitivities and distortions on the stimulus, s. We can regroup the numerators and denominators of the log quotients by model rather than by distortion, re-express the optimization problem as follows:
(ei, = arg max u.v. (IA.IB) n1.
where
log da(u) da(v) dB
log d
mu.y (IA. IB)
(3)
For any pair of distortions u, the function (..v(...)) defines a (pseudo- )merric." Specifically, it is non-negative, symmetric, obeys the triangle inequality, and zero when IA IB- This metric has several appealing properties:
Invariance scaling of the FIMs by positive constants CA. cp 0:
Mu, (A IB) (ca.l(cA...cplp
This a desirable property since we are interested identifying relevant image distortions that depend the shape of the FIMs, independent of scaling factors.
Invariance to permutation of the distortions u and V:
mu.v (LA, 1B)= = M...(LA,
When Ej and E2 are the generalized eigen-distoriies of LA and Lg. mei (IA, IB) is an approximation of the Fisher-Ree Rao distance between mean-zero Gaussian distributions with covariances IA and IB (up to scaling factors, see Appx. A). This interpretation suggests principled extension of the metric more than two distortions.
The metric compares stochastic representations back in stimulus space via their FIMs, which avoids having align the stochastic representations (Duong et al., 2023b).
3.2 PRINCIPAL DISTORTIONS FOR COMPARING MULTIPLE IMAGE REPRESENTATIONS
To optimize pair of image distortions for distinguishing N > 2 representations, Ai.... AN. we choose e 62 maximize the sum of the squares of all pairwise distances between the FIMs under the metric defined in Equation 3:
(E1, E2) arg max
mu, (IA,; I.m.
This is equivalent to maximizing the variance of the image representations' log sensitivity ratios:
N
dA. 1 (v)
N d.A. (u)
{aL, E2) = arg max U.V
Σ
log
Σ log
N
m=1 d
We refer {er, E2 the "principal distortions" of the models, analogous principal component analysis (Fig. IB). For a gradient-based optimization algorithm, see Appx. B.
There are several other natural extensions of genecalizeeeeeggadissottoooo when considering N models. For example, for any p 2 1, one can choose distortions E1. E2 that maximize the sum of the th power of all pairwise distances. Here we focus on the case maximizing the variance (p 2) and leave an exploration of other moments to future work.
4 EXPERIMENTAL RESULTS
As a demonstration of our method, we generated principal distortions for computational models previously proposed to capture aspects of the human visual system. All models were implemented PyTorch (Ansel al., 2024) and simulations were performed on NVIDIA GPUs (RTX A6000 and A100 models). As the models are deterministic, we calculate the FIM by assuming the network output is corrupted by additive Gaussian noise, as (Berardino et al., 2017). this case, ((s) =
J (s) Js))), where (s) is the Jacobian of the model f(-) input s and the induced geometry on stimulus space is the pullback via f-) of the Euclidean geometry on representation space.
"Tehhniallyy, m..( -) is psendometric because mu. ( Ia) does not imply that IA Ig.

---

Published as a conference paper at ICLR 2025
Principal 61 62 O Random 61
B
Image S
Principal E1
Principal 62
LN
Model
LG
LGG
LGN
d(t1)) log e((
C
LG
LGN 29.3E
LN
LGG $ + 10.0c
8 + 99.5c1
$ + 18.5c
8 0.56,
8 70.7.77,
81.5E2
$ 5 90.062
Figure 2: Principal distortions of four early visual models. A) Log sensitivity ratios of the two principal distortions and two random distortions for each model (Berardino al., 2017). Models are nested (LN is the most basic, LGN is the full model). Principal distortions (filled circles) give rise to diverse log ratios, while random distortions (hollow circles) do not. B) Natural image s and corresponding optimized principal distortions {EI, E2). C) Natural image corrupted by principal distortions, with each pair scaled so as to be equally detectable by one model (as indicated above). Models ordered by the log ratio of their sensitivities (panel A). If model's thresholds are proportional to human thresholds, the corresponding pair scaled distortions should be equally visible in the top and bottom images. Note: Images are best viewed at high resolution.
4.1 EARLY VISUAL MODELS
We generated principal distortions for nested family of models designed capture the response properties of early stages (specifically, the lateral geneiculate nucleus, LGN) of the primate visual system (Fig. 2). The full model (LGN) contains two parallel cascades representing ON and OFF center-surround filter channels, rectification, and both luminance and contrast gain control nonlin- earities. The other models are reduced versions of this model. LGG removes the OFF channel, LG additionally removes the contrast gain control, and LN removes both gain control operations. The filter sizes, amplitudes, and normalization values of each model were previously fit separately to predict a dataset of human distortion ratings (Berardino et al., 2017, details in Appx. C).
As these models were explicitly trained to predict human distortion thresholds, we provide a qual- itative comparison of each model's sensitivities to human distortion sensitivity (Fig. 2C). For each model, we adjusted the relative scaling of each principal distortion until the model was equally sensitive the scaled distortions while constraining the sum of the Fuclidean norms of the two dis- tortions to be a fixed value of 100; that is, we chose positive scalars kj, ko such that the sensitivities were equal, d(kjeı)
di d(k22))) with kj + K2 = 100. If a model's thresholds are comparable to human thresholds, then the pair of images should also appear equally distorted to human observer. Visual inspection of these images reveals that both distortions are visible when rescaled for the LGN model and the LN model, suggesting that these models closest to human distortion thresholds. For LG, the scaled e, distortion is not visible, while the scaled E distortion is immediately apparent, suggesting strong mismatch with human observer thresholds. The same is true of the LGG model,

---

Published as a conference paper at ICLR 2025
with the roles of the two distortions swapped. These qualitative observations are consistent with the results of (Berardino et al., 2017), in which experiments on eigen-distortion distortions suggested that the LGN model was the best of these models in terms of consistency with human distortion sensitivity. Future work with analogous human perceptual experiments could directly quantify the visibility of the principal distortions arising from our analysis (see Supp. Fig. SI.1 for an illustration).
An advantage of our framework is that it can dramatically reduce the number of distortions needed to differentiate set of models. For instance, to judge how well these models of the early visual system capture human perceptual discrimination thresholds, Berardino et al. (2017) computed the extremal bigen-distortions for each of the four models, and then measured human perceptual discrimination thresholds to all eight distortions. In general, their method requires assessing visibility of 2N distor- tions, for N models. Zhou et al. (2023) considered pair of generalized digen-distortions for each pair of models, for total of N + 1) distortions. In contrast, our method always selects the two distortions that maximize the variance across the models, independent of N. Human sensitivities to these distortions can then be estimated in perceptual discrimination experiments to judge which msdel(s) are closest in terms of the metric we defined Equation 3. The models whose sensitivities are far from human sensitivities can be discarded and this procedure can be repeated to best differen- tiate the remaining models, and so on. one could reduce the number of models by, say, a factor of two each iteration of this process, the total number of stimuli to be assessed scales as 2 loga (N). This dramatic improvement in efficiency could enable the comparison of significantly larger sets of models than feasible with previous methods.
4.2 DEEP NEURAL NETWORKS
Deep Neural Networks (DNNs), originally developed for object recognition, have also been ex- amined models of the primate visual system (Yamins & DiCarlo, 2016; Schrimpf et al., 2018; Lindsay, 2021). A plethora of models, varying in architecture and training techniques, have been proposed, but many of these models perform quite similarly on behavioral tasks neural bench- marks (Schrimpf et al., 2018; Tuckute et al., 2023; Conwell et al., 2024). This situation offers a well-aligned opportunity for use of our principal distortion method. Here, we investigate previously trained sets of models (no additional training is required to apply our method) and generate principal distortions to differentiate the image representations.
We first investigated set of layers from two architectures trained on the ImageNet object classifi- cation task-ales -eee-eleNet (Krizhevsky et al., 2012) and ResNet50 (He et al., 2016) and generated the principal distortions that maximally separate these models (Fig. 3, see Sec. D of the supplement for layer choices and model details). Although these architectures are not currently state-of-the-arr at image recognition or neural prediction, they have been widely used and trained with various opti- mization strategies, making them well suited for controlled experiments. (See Supp. Fig. SI.2 for a similar test of the method on the modern architectures EfficientNet and ViT.)
Notably, the hierarchical structure of the models is reflected in the log ratios of the sensitivities, where early layers of the models are closer together in the metric space (Fig, 3D), and late layers of AlexNet are always more sensitive to E1 when late layers of ResNet50 are more sensitive to 62- There is additional structure revealed by these principal distortions eeeex is more sensitive to the principal distortion that generally appears concentrated on regions of the image that have more variability, i.e., the "stuff" in the image (distortion E in Fig. 3B), while ResNet50 is more sensitive to distortions that occur in the relatively constant regions of the image (distortion 62 in Fig. 3B). The separability of the models and the sensitivity of the networks to the two distortion types was remark- ably consistent across set of 100 base images chosen from the ImageNet dataset (Fig. 3E, see Supp. Fig. SI.3 for additional examples). These distortions differentially affected the classification deci- sions of the two architectures (Supp. Fig. SI.4). The distortions also replicated in set of six models (3 randomly initialized AlexNet models and randomly initialized ResNet50 models, each trained on ImageNet-Ik, see Supp. Fig. S1.5). This distinction was also found in images designed explicilly to test for possible differences between models due to edge artifacts or contrast sensitivity (Supp. Fig. SI.6). As far tortions in different portions of the image has not been documented, demonstrating that our method can reveal interpretable differences in the local sensitivities of complex computational models.
know, this qualitative difference in sensitivities of the architectures dis-
7

---

Published as a conference paper at ICLR 2025
A
S
B
20c1
20c₂
+ 20c1
20c2
AloxNet ResNoc50
10
10
-10
-5
5
10
-10
-5
5
10
lug d(c1) d(ez)
E
AlexNet vs. ResNet (N=100 Base Images)
Layer
AlexNet
ResNet50
4
100.0
-7.5
-5.0
-2.5
0.0
2.5
5.0
7.5
10.0
Early
d(c1)
More sensitive to 62 log
More sensitive to E1
d(E2)
Figure 3: AlexNet versus ResNe150. A) Example base images. B) Optimized principal distortions (scaled by a factor of 20 for visibility, and using the convention ||<|| 1 here and in other fig- ures). C) Base image plus principal distortions. D) Log sensitivity ratios of principal distortions when comparing image representations at multiple layers of AlexNet and ResNet50. Assignment of and 62 was chosen so that the final tested layer of AlexNet has positive log ratio. E) Log sensitivity ratios averaged across 100 base images (error bars are standard deviation). The prin- cipal distortions organize the networks by architecture the log sensitivity ratios of AlexNet and ResNet50 are separated and early layers have smaller log ratios than late layers. AlexNet is more sensitive to distortion C1. which is concentrated (often the foreground object). ResNet50 is more sensitive to distortion E2 which concentrates power relatively smooth parts of the image, such regions of constant intensity/color.
higher contrast textured parts of the image
Networks trained to reduce texture bias The architectural difference observed between ImageNet-trained AlexNet and ResNet50 suggested that the texture of the image may be driving some of the differences in local geometry. Previous work demonstrated that standard DNNs ex- hibit strong "texture bias" (Geirhos et al., 2019) and proposed models that explicitly reduce the texture bias by training on Stylized ImageNet (SIN), a set of images that retains the content of each ImageNet image but overlays details matched to particular texture. Training these images reduce the model's reliance on texture for classification. If this training set strongly affected the local geometry of the networks, of architectures and training sets would be driven by this training set distinction. We find evidence that this the case. We generated principal distortions for 100 base images using set of layers from two architectures (ResNet50 and AlexNet) and two different training datasets (ImageNet and SIN, Fig. 4A). The principal distortions appeared qualitatively similar those generated when we investigated only the standard networks (Fig. 4B, more examples in Supp. Fig. SI.7). Both AlexNet architectures, regardless the training type, were more sensitive to the perturbations of higher vari- ability parts of the image, while both ResNet50 models were more sensitive to the distortions that were mainly targeting constant areas the image. To quantify this observation and dissociate the role of foreground and background information from the high- and low-frequency content in the image, we ran an experiment using the ImageNet-9 dataset with foreground and background masks, where generated principal distortions after applying low-pass filter to either the foreground
might expect that principal distortions generated for mixture
8

---

Published as a conference paper at ICLR 2025
Figure Comparison of AlexNet and ResNet50 variants trained to increase "Shape Bias" of net- works. A) Log sensitivity ratios of principal dis- tortions for networks trained on standard Ima- geNet and Stylized ImageNet (SIN), Average log sensitivity is computed across 100 base images (error bars are standard deviation) and the choice of E is set such that d(E1) 2 0 for the last tested layer of the standard AlexNet. B) Example hase image and the optimized principal distortions. Similar to the analysis of the ImageNet-trained
A
Comparison with networks
Layer Late
AlexNet
ResNet50
AlexNet (SIN)
ResNet (SIN)
Earty
10
-5
5
10
log ddee)/d(ez)
20 4:
+ 61
Alexreeer more sanstiive Et.
ResNet50 and AlexNet (Fig. 3), E is concentrated on parts of the image with higher spatial frequen- cies while Eg is concentrated on parts of the im- age with relatively solid patches. Both AlexNet architectures are more sensitive to 61 while both ResNet50 architectures are more sensitive to 62. suggesting that the differences in local sensitivi- ties of these networks depend more on differences in architecture than training procedure.
regardhess onegeraec Syyodd coagarler traiiig
20 E/
+ 206g
more sonotiiv Fg. regandees
imaganist stytced imageher franing
or the background of the image (Supp. Fig. SI88. The principal distortion that AlexNet more sensitive to consistently concentrated on portions of the image with higher spatial frequencies (i.e., parts of the image that are not smoothed), while ResNet50 is concentrated on parts of the image with only low-frequency information (the blurred parts of the image, regardless of whether this is in the foreground or background).
Networks trained to reduce adversarial vulnerability Another well well-known example of the use of training set modifications to achieve robustness in object recognition is that of adversarial training (AT). Adversarial examples are generated at each step of model training and these stimuli are used to update the model weights, with the "true" category label used for the update. Previous work has found that adversarially trained -raaiidd networks are more aligned with those of biological systems (Madry,
2017: Feather et al., 2023: Gaziv et al., 2023). As adversarial examples are constrained to be very small perturbations, it seems plausible that the local geometry for AT models would differ from their standard counterparts. Indeed, we see that the principal distortions generated from the set of models that included standard and AT trained AlexNet and ResNet50 models reliably separate the model classes by training type, rather than architecture (Fig. 5A). Most layers of the adversarially trained models are more sensitive to relatively smooth changes of patches of color in the image, or to shading around edges, while most layers of the standard models are more sensitive to what appears as unstructured noise (Fig. 5B, additional examples in Supp. Fig. SI.9). This is in stark contrast to the SIN-trained networks of the previous section, for which principal distortions reliably separated the models by architecture. These examples demonstrate that our method can be used to separate collections of similar models, and points to its utility in probing complex high-level representations.
DISCUSSION
We have introduced a metric on image representations that quantifies differences in local geometry, and used it to synthesize "principal distortions" that maximize the variance of the metric over a set of models. When applied to hand-engineered models of the early visual system and to DNNs, our approach produced novel distortions for distinguishing the corresponding models. In particular, with the DNN analysis, we revealed that there are qualitative differences local geometries of ResNe150 and AlexNet architectures, that adversarial training dramatically changes the local geometry, but stylized ImageNet training does not. Although our qualitative examples in this targeted set of neural networks do not fully elucidate the recent observations that many different models are equally good at capturing brain representations, our method provides a direct approach to begin to tease apart the interplay between local geometry and global structure.
9

---

Published as a conference paper at ICLR 2025
Comparison with Networks
Figure 5: Comparison of AlexNet and ResNet50 variants trained to reduce adversarial vulnerabil- ity. A) Log sensitivity ratios of principal distor- tions for different layers of standard- and adversarially-traine (AT) models. Average log sensitivity is computed across 100 base images (error bars are standard deviation) and the choice of 61 is set such that d(Ei) 2 O for the last tested layer of the standard AlexNet. B) Exam- ple hase image and the generated principal dis- tortions. Distortion Ei appears as less structured noise, and both AlexNet and ResNe150 standard networks are more sensitive this perturbation, while the AT DNNs are more sensitive to E2 which focuses color changes around the content of the image, suggesting that the differences in local sen- sitivities of these networks depend more on differ- ences in training procedure than architecture.
Model Layer Late
AlexNet
H H HAM
ResNet50
AlexNet (AT)
ResNet50 (AT)
MHI
Firly
10
0 log d(tr)/A(es)
TO
B
20
5 20
Swndlee ChaMee mave seralive to Cj. resaicioss of anchitectone
20
+ 206g
regendiess Ea anchiadture
There are several natural methodological extensions. The metric is closely related to the Fisher- Rao metric between mean zero Gaussians and this relation suggests a natural extension to synthesizing more than two distortions (Appx. A). analogous to using additional principal components to capture more variance within of high-dimensional vectors. Additionally, while we foeus on computing principal distortions that differentiate a finite set of models, there is a natural extension to computing the principal distortions that differentiate continuous families of models with a prior distribution over models.
There are several limitations in our formulation of principal distortions. First, our framework is based on local differential analyses of model at a base image, so the sensitivity estimates we ob- tain via these analyses can only be guaranteed to hold in an infinitesimally small neighborhood of the base image. If model is highly nonlinear in the vicinity of a base image, then the local lin- ear approximation may not accurately reflect model sensitivities. Second, to compute the FIM of a deterministic model, we assume additive Gaussian response noise, which is not generally repre- sentative of neural responses in the brain. Poisson variability could be used to better capture neural responses. Another approach is to fit the model noise structure to measurements of neural system; see (Ding et al., 2023; Nejatbakhsh et al., 2023) for work along these lines.
Principal distortions provide efficient method for comparing computational models with human observers, for whom experimental time for acquiring responses to stimuli is generally severely lim- ited, Although we only presented qualitative comparisons and examples related to human perception in this paper, the optimized distortions are a parsimonious choice of stimuli that can be readily in- corporated into psychophysics experiments. For example, the distortions generated from the early visual models could be used for perceptual discrimination experiment with human observers simi- lar to those performed by Berardino al. (2017), to compare the human log-ratio sensitivity for the optimized distortions to that predicted by the models.
The results with DNNs reveal some intriguing properties: some models have stronger sensitivity biases in the local geometry for perturbing higher frequency regions of the space, while others have more sensitivity to perturbations in relatively low frequency regions. This also suggests an interesting question for future distortion detection: in what contexts are humans more sensitive to perturbations of the "stuff" of the image compared to perturbations in empty parts of the image? And how do these observed differences relate to previous work investigating how human and neural networks rely on spatial frequencies for classification (Subramanian et al., 2023)? Finally, beyond direct comparisons with human observers using psychophysies experiments, our method may be useful in the domain of neural network interpretabilityy where it may be useful to have direct com- parisons of the local distortions that will maximally differentiate sets of models. Principal distortions are complementary to many currently used model interpretability tools (e.g., saliency maps, atten- tion weight visualization) since they are explicitly designed to generate distortions that differentiate a set of models, as opposed to interpreting one model at a time.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The Flatiron Institute division of the Simons Foundation. 'he computations reported in this paper were performed using resources made available by the Flatiron Institute. We thank David Brainard, Thomas Yerxa, and the Laboratory for Computational Vision at NYU and the Flatiron Instiute for their feedback.
REPRODUCIBILITY STATEMENT
The code used to generate principal distortions, details of loading the models, and example principal distortion generation code is available at
We aim for the distortion generation to be easy for others to use to probe new models. The mathe- matical derivation of the algorithm is provided in Appx. B. We have included details in Appx. C with the parameters of the early visual models, and Appx. D details where checkpoints were obtained for the tested deep neural networks.
REFERENCES
Shun-ichi Amari. Information Geometry and its Applications, volume 194. Springer, 2016.
Jason Ansel, Edward Yang. Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky,
Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary Vetioo, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang. Kshiteej Kalambarkar, Laurent Kirsch, Michael Lazos, Mario Lezcano, Yanbo Liang, Jason Liang, Yinghai Lu, CK Luk, Bert Maher, Yunjie Pan, Chris- tian Puhrsch, Matthias Reso, Mark Saroufim, Marcos Yukio Siraichi, Helen Suk, Michael Suo, Phil Tillet, Eikan Wang, Xiaodong Wang. William Wen, Shunting Zhang, Xu Zhao, Keren Zhou, Richard Zou, Ajit Mathews, Gregory Chanan, Peng Wu, and Soumith Chintala. PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation. In 29th ACM International Conference on Architectural Support for Programming Languages and 0t555//6620665.3644666
Operating Systems, Volume (ASPLOS 24). ACM, April 2024. doi:
URL https:ttp//.omm httpt::httttttoeeeeeee /pyttrch..cccccccco org/ asset wptttrcct-cccc 22222... pddf.
Bruno B Averbeck and Daeyeol Lee. Effects of noise correlations on information encoding and decoding. Journal of Nearophysiology, 9366)336333664, 2006.
Alexander Berardino, Johannes Ballé, Valero Laparra, and Eero Simoncelli. Figen-distortions of hierarchical representations. Advances in Neural Information Processing Systems, 30:3531-3540, 2017.
Nicolas Brunel and Jean-Pierre Nadal. Mutual information, Fisher information, and population coding. Neural Compulation, 10(7):1331-1757, 1998.
Max F Burg, Thomas Zenkel, Michaela Vystrčilová, Jonathan Oesterle, Larissa Höfling, Kon- stantin Friedrich Willeke, Jan Lause, Sarah Müller, Paul G. Fahey, Zhiwei Ding, Kelli Restivo, Shashwat Sridhar, Tim Gollisch, Philipp Berens, Andreas S. Tolias, Thomas Euler, Matthias Bethge, and Alexander S Ecker. Most discriminative stimuli for functional cell type clus- tering. In The Twelfth International Conference Learning Representations, 2024. URL https: htttss://pperees ew net forum? i d= 976aaAcY
Yeo Chung and Larry F Abbott. Neural population geometry: An approach for understanding biological and artificial neural networks. Current opinion in neurobiology. 70:137-144, 2021.
Colin Conwell, Jacob S Prince, Kendrick N Kay, George Alvarez, and Talia Konkle. A large-scale examination of inductive biases shaping high-level visual representation in brains and machines. Nature Communications, 15(1):9383, 2024.
Harald Cramér. Mathematical Methods of Statistics, volume 26. Princeton University Press, 1946.

---

Published as a conference paper at ICLR 2025
Xuchao Ding, Dongsoo Lee, Joshua B Melander, George Sivulka, Surya Ganguli, and Stephen A Baccus. Information geometry of the retinal representation manifold. Advances in Neural Infor- mation Processing Systems, 4432222--4432,.220 2023.
Lyndon Duong, Kathryn Bonnen, William Broderick, Pierre-Etienne Fiquet, Nikhil Parthasarathy.
Thomas Yerxa, Xinyuan Zhao, and Eero Simoncelli. Plenoptic: platform for synthesizing model-optimized visual stimuli. Journal of Vision, 23(9):5822-582) 2023a.
Lyndon Duong, Jingyang Zhou, Josue Nassar, Jules Berman, Jeroen Olieslagers, and Alex H Williams. Representational dissimilarity metric spaces for stochastic neural networks. In The Eleventh International Conference Learning Representations, 2023b.
Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness robustness. (python library), 2019. URL https: / htt.hc...ooooo.ccoommmmc..o.... Cacoo//Maii reassurssssssssssssssss...
Jenelle Feather, Guillaume Leclerc, Aleksander Madry, and Josh H McDermott. Model metamers reveal divergent invariances between biological and artificial neural networks. Nature Neuro- science, 26( 20220722004, 2023.
Ronald Aylmer Fisher. Theory of statistical estimation. Mathematical Proceedings of the Cambridge Philosophical Society, 2205):700-725, 1925.
Deep Ganguli and Eero P Simoncelli. Efficient sensory encoding and bayesian inference with het- erogeneous neural populations. Neural Computation, 26(10):21332113 2014.
Guy Gaziv, Michael Lee, and James DiCarlo. Robustified anns reveal wormholes between human category percepts. arXiv preprint arXiv:2308. 06887, 2023.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019.
Tal Golan, Prashant C Raju, and Nikolaus Kriegeskorte. Controversial stimuli: Pitting neural net- works against each other as models of human cognition. Proceedings of the National Academy of Sciences, 117(47):99330-29333 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. International Conference on Learning Representations, 2014.
D.M. Green and J.A. Swets. Signal Detection Theory and Psychophysics. Wiley, New York, 1966.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Max Klabunde. Tobias Schumacher, Markus Strohmaier, and Florian Lemmerich. Similarity of neural network models: survey of functional and representational measures. arXiv preprint arXiv:2305.06329, 2023.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In International conference on machine learning, pp. 3519- 3529. PMLR, 2019.
Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandetrini. Representational simiiarr analysis- connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2:249, 2008.
Alex Krizhevsky, Ilya Sutskever. and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Advances neural information processing systems, 25, 2012.
Grace W Lindsay. Convolutional neural networks as a model of the visual system: Past, present, and future. Journal of cognitive neuroscience, 33110)3007--003,, 2021.
12

---

Published as a conference paper at ICLR 2025
David Lipshutz*, Jenelle Feather*, Sarah E Harvey, Alex H Williams, and Eero P Simoncelli. Com- paring the local information geometry of image representations. In UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models, 2024.
Aleksander Madry. Towards deep learning models resistant to adversarial attacks. arXiv preprint NaXtv:7706.00083, 2017.
Amin Nejatbakhsh, Isabel Garon, and Alex Williams. Estimating noise correlations across continu- ous conditions with wishart processes. Advances in Neural Information Processing Systems, 36: 54032-54045, 2023.
MA Paradiso. A theory for the of visual orientation information which exploits the columnar structure of striate cortex. Biological Cybernetics, 58(1):35-49, 1988.
Julianna Pincle, João E Strapasson, and Sueli IR Costa. The Fisher-Rao distance between multivari- ate normal distributions: Special cases, bounds and applications. Entropy, 22(())404, 2020.
Nikolay Ponomarenko, Vladimir Lukin, Alexander Zelensky, Karen Egiazarian, Marco Carli, and Federica Battisti. Tid2008-a database for evaluation of full-reference visual quality assessment metrics. Advances of modern radioelectronics, 10(4):30-45, 2009.
CR Rao. Information and accuracy attainable in the estimation of statistical parameters. Bulletin of the Calcutta Mathematical Society, 37(3):81-91, 1945.
Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib Majaj, Rishi Rajalingham, Elias Issa, Ko- hitij Kar, Pouya Bashivan, Jonathan Prescott-Roy, Franziska Geiger, et al. Brain-score: Which artificial neural network for object recognition is most brain-like? BioRxiv, pp. 407007, 2018
Peggy Seriès, Alan A Stocker, and Eero P Simoncelli. the homunculus "aware" sensory adap- tation? Neural Computation, 21(12):3271-300 2009.
HS Seung and H Sompolinsky. Simple models for reading neuronal population codes. Proceedings of the National Academy of Sciences of the United States of America, 90:10749-10753, 1993.
Ajay Subramanian, Elena Sizikova, Najib J. Majaj, and Denis G. Pelli. frequelyfrequency chan- nels, shape bias, and adversarial robustness. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL
net. nertforumm dd=kywwXessl..
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Greta Tuckute, Jenelle Feather, Dana Boebinger, and Josh H McDermott. Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions. Plos Biology, 21(12):3000336, 2023.
Zhou Wang and Eero P Simoncelli. Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual quantities. Journal of Vision, 8(12):8-8, 2008.
Xue-Xin Wei and Alan A Stocker. Mutual information, Fisher information, and efficient coding.
Neural Computation, 28(2):305 2016.
Ross Wightman. models, 2019.
Pytorch image models. pytorch- imaleiimae--aae iwwgammmdeleeeeeee
Alex H Williams, Erin Kunz, Simon Kornblith, and Scott Linderman. Generalized shape metrics on neural representations. Advances in Neural Information Processing Systems, 34:4738-4750, 2021.
Kai Yuanqing Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. Noise or signal: The role of image backgrounds in object recognition. In International Conference on Learning Rep- resentations, 2021. URL https: //opp///eeew......//
Daniel LK Yamins and James DiCarlo. Using goal-driven deep learning models to understand sensory cortex. Nature Neuroscience, 19633335-365, 2016.
13

---

Published as a conference paper at ICLR 2025
Jingyang Zhou, Chanwoo Chun, Ajay Subramanian, and Eero P Simoncelli. Comparing neural models using their perceptual discriminability predictions. In Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models, pp. 170-181. PMLR, 2023.
14

---

Published as a conference paper at ICLR 2025
A RELATION TO THE FISHER-RAO METRIC
The Fisher-Rao distance between two mean zero K -ddimensional Gaussian distributions with posi- tive definite covariance matrices A and B is defined
K Σ ==1
AA B) := || log B 1/2 AB |/
(log x;)
where {):} denote the eigenvalues of the generalized eigenvalue problem Ar XBe (Pinele et al., 2020). We'd like metric that's invariant to arbitrary scalings A + CAA or B + cp B for
CA. > 0, which suggests the following definition:
K
(A, B)
min (cA.A. min >
(c
log A.)² K KVar((Log A:)),
(5)
cAlea50
2=1
where the final equality uses the fact that the optimal e the mean of {- log );}.
Using the facts that
k
K Var(ffog xx3)
Σ>
>
(log
A log 2y)3
2K
jml
and (log > - log 2,)² (log A log )² for i,j. have
K 1
(log A1 log Ax)² (2, =
(log A1 log xx)²
K
2
with equality holding when K = 2. When A IA and B I, then da(e) VET AE and dc(c) Bc. If с₁ and cx denote the extremal generalized eigenvectors associated with X and λx, respectively, then
da(EI) dg(E))"
log Ax 2og dA(EK) dø(ek)
log \1 2og
Therefore,
2
log dx(E))
dy(ex) llog dn
dx(E1)
log da(ex)
I ST(A, S V2(K - 1) log dØ(Ei) dØ(EK)
VK
(EI)
and
2
me (I,, IB) (AA B) S 11(K-))me (IA, IB).
VK
Extension to more than two distortions Equation suggests a natural extension for defining a metric between positive definite matrices using J > 2 distortions. Specifically, it suggests choosing the distortions to maximize the variance across the log ratios of the sensitivities. To this end, we can define the squared distance between the local geometries to be the variance (across distortions) of the log ratio of the the sensitivities:
(A, = MVar (fre
da(o,)) dz (2(1))))
m
A set of J principal distortions then be chosen to maximize the variance across models under this metric.
B COMPUTING THE TOP TWO OPTIMAL DISTORTIONS
Suppose we have models with sensitivities (d)( The optimal distortions (EI, E2) are solu- tions to the optimization problem
log (u) d, (u) d (v) N
arg max L(u, v).
L(u,v)
log
dm (v)
15

---

Published as a conference paper at ICLR 2025
Differentiating L with respect to yields
log (u)
log dn (u) dm (v)
(n)s
N
Im (s)
Vul(u, = Σ
dn(v)
d2 (2((u)
de
n=1
m=1
m==1
log
(u)
1
N
log
den (u)
(s)u
Σ
NV
I (s)
d
N
de (v)
d2 (u)
N
d
==
m=1
where we have used the fact that
Iu Iu
Vulog
Vu log(u (iu
Iu)
2
u Iu d'(u)
Similarly, differentiating L with respect yields:
(u 1 (
N
(u) de, (v)
[s)v N d2(v)
(n)
V.L(u,v) = - Σ n=l
log
Σ
log
m
Σ m=1 d (v)
N
N
m=1
Combining, we have the following gradient- based optimization algorithm.
Algorithm 1: Computing the principal distortions via projected gradient descent
1: Input: Positive definite x D matrices Ly,..., , IN. learning rate 7j > 0, target distortion size
> Q
2: Initialize: distortions u, v E R
3: while not converged do
4: for n = 1,...,,N N do
5: (i(n) I,
6:
(y(n) + I,
7:
dn(n) (u, (n))
8:
dz(n) - (v.02(n)
9:
u1 u1(n) oi(n)/d
10:
u2(n) (2(n)/d{(n)
11:
r(n) f log de log (n)
12:
end for
13:
U1 + mean( un (n)
14:
U2 f mean( U2 (n)
15:
F + mean(r(n)
16:
Au + En=
(r(n) r] (ui(n) (n) -ūi] (r(n) (2)(n) ū2
Av
Em=
18:
u - u Au
19:
V v nāv
20:
au/ll |||]
21: v av/ ||]
22:
end while
C METHODS: EARLY VISUAL MODEL EXPERIMENTS
PyTorch implementations of the early visual models were obtained from Duong al., 2023a). The early visual models were adopted from Berardino et al. (2017), with parameters chosen to maximize the correlation between predicted perceptual distance and human ratings of perceived distortion, for wide range of im- ages and distortions provided in the TID-2008 database (Ponomarenko et al., 2009). Parameters are reported in Table 1. Note that although the models are nested in their construction and parameter- ization, the model parameters differ across tested models since they are optimized for each model independently.
C.I DATASET
Distortions generated for images from the Kodak TID 2008 dataset (Ponomarenko al., 2009).
16

---

Published as a conference paper at ICLR 2025
Model
center-surround, center std
0.5339
center-surround, surround std center-surround, amplitude ratio
6.
1.25
LG Model
luminance, scalar
14.95
luminance, std
4.235
center-surround, center std
1.962
center-surround, surround std
4.235
center-surround, amplitude ratio
1.25
LGG Model
luminance, scalar
2.94
contrast, scalar
34.03
center-surround, center std center-surround, surround std center-surround, amplitude ratio
0.7363
48.37
1.25
luminance, std
170.99
contrast, std
2.658
LGN Model
(Two channels)
luminance, scalar contrast, scalar
[3.2637, 14.3961]
[7.3405, 16.7423]
center-surround, center std
[1,237, 0.3233]
center-surround, surround std
[30.12, 2.184]
center-surround, amplitude ratio
1.25
luminance, std contrast, std
[76.4, 2.184]
[7.49, 2.43]
Table 1: Parameters for early visual models, obtained from (Berardino et al., 2017).
D METHODS: DEEP NEURAL NETWORK EXPERIMENTS
With the exception of the experiment described in the next paragraph, we analyzed DNNs were obtained from the model loading code and checkpoints available https://githui benellefeather/ heteetteee/mmmeeeeeeee model__met aareee__mmtmrrssss amers. www.rrsspptttohoc which were used in (Feather et al., 2023), and allowed for easy loading and selection of the intermediate layer stages for many models that had previously been proposed as models of human visual perception. The checkpoints for the stan- dard ResNet50 and AlexNet models were obtained from the public PyTorch checkpoints (Ansel et al., 2024). The Stylized Image Net AlexNet felexnet.traiieed.on
com/
and ResNet50 were obtained from https :/t::://gtttie tith.b...htt..g com/ r rhos/ texture-ves shape associated with (Geirhos al., 2019). The checkpoint for the ResNet50 (e(Curin = 3.0) adversarially trained model was obtained from (Engstrom et al., 2019), and the checkpoint for the Alexnet (2( 3.0) adversarially trained model was obtained from (Feather et al., 2023). For all experiments, we only included intermediate layers before the final classification stage in the principal distortion analysis, and the subset of layers followed those chosen Feather et al. (2023). Specifically, for the AlexNet models we included layers reluO, relul, re relu2, re relu3, relu4, fc0.relu, and relurelu in each of anlayses. For the ResNet50 models we included convi.relus layerl, layer2, layer3, layer4, and avgpool in each set of analyses.
5resmntt50.triinede
To demonstrate that our approach holds for more modern architectures, we compared a ViT vs. and EfficientNet model. These models were obtained from the PyTorch Image Models (timm) repository (Wightman, 2019). We chose a version of EfficientNet and ViT that were trained on the size of Images in ImageNet-9 (224 224 images), and versions of each archi- tecture trained with the same image dataset (ImageNet-Ik). For the EfficientNet, model EfficientNet-BB (timm model tf.effiiientnet... bb) and included layers conv_stem, blocks. 0. blocks. 1. blocks. blocks. 3, blocks. 4, blocks 5. blocks. 6, and global pool in the analysis. For the VIT, we tested model VIT (Base-Patchl6-221 (timm model vit base patch1 6 224 augreg in1k) and included layers patch embed, bl blocks.
tested
17

---

Published as a conference paper at ICLR 2025
blocks. 1, blocks 2, blocks. blocks 4, blocks 5, blocks. blocks.
blocks. 8, blocks. 9, blocks. 10, blocks. 11 and head in the analysis.
D.1 DATASETS
For the deep neural network experiments we had two sets of images that were used to generate the principal distortions. For the main experiments, we use a subset of 100 ImageNet images where each image was chosen from a unique class (randomly chosen from the set of images at https: //github. com/ E1 wwwsshhwwttt/maaaaaeeeeeeeeaeeee iiwgi/eettmmiieeemee -aaaaaagnneetttallleeeeeeee images).
For the experiments with background and foreground blur, used the ImageNet-9 dataset with foreground/background masks (Xiao et al., 2021). We chose eight random images from each cat- egory of images, resulting in 72 total images. As this is relatively small subset of images, removed images that had blank backgrounds (by eye) or that seemed to have incorrect masks (as the masks automatically generated in the original dataset and not validated with human labeling) and replaced them with new random images so that the backgrounds would always be affected by the smoothing procedure. This procedure was done before running the images through the principal distortion analysis. For the foreground and background blurred images, we blurred the image with a Gaussian filter with standard deviation of pixels.
D.2 PRINCIPAL DISTORTION OPTIMIZATION
For each image and each comparison, we ran the gradient descent procedure for principal distortion optimization for 2500 iterations, using an exponentially decaying learning rate that started 10.0 and decayed to 0.001 by the final step. The exception to this is for the experiment with ViT (Base- Patch16-224 vs. EfficientNet-BO, where we ran the optimization for only 500 iterations.
We used a target distortion size of a = 0.1 and at each step of the optimization, we also scaled e so that the image + 1000c would not be clipped when the RGB value was represented between 0-1; that is, we scaled so that 0 < s[i] + [000c(t] < 1 for each value i in the image. This constraint could potentially bias the perturbations amplitude for the perturbation cannot be focused
be more spread out across the image (because all of the
small set of pixels). However, removing this constraint did not lead to quantitatively different results (Supp. Fig. SL.10,, but the inclusion allows for more viable comparisons with human perception (as the perturbation can be scaled while maintaining valid values in the image gamut). Finally, one or more of the FIMs is degenerate (i.e., not full rank), this potentially lead issues in the optimization procedure (though this issue did not arise the experiments presented here). To address this, one option is to regularize the FIM by adding small constant times the identity matrix.
D.3 SUPPLEMENTAL FIGURES
18

---

Published as a conference paper at ICLR 2025
A
C1
C2
IN
LG
LGN
M
dl41
B
5
5
15
5 20
5-+ 25
5- 5
5- 35
5 40.4
4
(-2.197)
(-1.386)
(-0.847)
(-0.405)
(0.000)
(0.405)
(0.847)
H (1.386)
(2.1997)
Figure S1.1: Illustration of method for comparing early visual models to human perception. A) An example base image (s) and the principal distortions (E1, E2) generated for the four early visual mod- els (as in Fig. 2). B) Base image corrupted by the two principal distortions (top and bottom row), scaled by amplitudes that sum to 50 and correspond to different log sensitivity ratios (parenthesized values below the pairs, also indicated with grey arrows in panel C). A perceptual experiment be designed to test which pair of corrupted images (indexed A through I) appear equally distorted to human observer. This could be done directly, by showing various pairs of distorted images (i.e.. s + 6161 and s + keg9 with varying amplitudes k1, k2) and asking an observer which of the two appears more distorted with respect the original (s). The pair of images for which an observer cannot decide (i.c., gives random answers) corresponds to the observer's 's log sensitivity ratio. Alter- natively, this could be assessed by measuring the observer's detection thresholds for each distortion independently, and then taking their log ratio. This estimated human log sensitivity ratio can then be compared to the models' log sensitivity ratios (colored dots, panel C), to assess which model best aligned with human behavior. Images are best viewed at high resolution.
19

---

Published as a conference paper at ICLR 2025
A
Base Image Example
Base Image Example 2
Base Image Example 3
Base Image Example
Base Image Example
ElficientNot
10
-10
5
-10
-5
5
10
-10
10
-10
10
log d(c))/((ra)
log d(c))/d(ra)
log d(cr)/d(raa)
lag d(ts)//(rz))
log d(ci)/Mi(ss)
G1
EfficientNet
more
8 25c
22
62 VIT more sensitive
B
(N=72 Base Images)
Model Layer
Proporton mainnariing sarne Sibal original image
62
EfficientNet
#
Late
VIT
[o]
4H
EfficientNet
9
6
-4
2 log d(E1)/d((e)
22
4
8
Early
WT
0.0
0.5
1.0
0.0
0.5
1.0
K
Figure S1.2: Comparison of ImageNet-1k 1 trained EfficientNet and Vision Transformer. A) Example principal distortions and associated log sensitivity ratio plots when comparing layers of EfficientNet- BO and VIT Basse-Patch1--222 B) When measured over 72 images from the ImageNet validation set (chosen from ImageNet-9, Xiao et al., 2021), the obtained principal distortions reliably separate the models by architecture the layers of EfficentNet are more sensitive distortion e while the layers of VIT are more sensitive to The principal distortions are qualitatively different than those obtained when comparing AlexNet and ResNet50-ffo for instance, the principal distortions e2 (i.e., the distortion that the VIT layers are more sensitive) have notable grid artifacts corresponding the size of the input patches to the ViT model. C) The distortions modify or preserve the label computed by each model, consistent with their predicted sensitivity (see Supp. Fig. SI.4 for detailed explanation).
20

---

Published as a conference paper at ICLR 2025
Base Image Example
Base Image Example 2
Base Image Example
Base Image Example 4
Base Image Example 5
Base Image Example
AlexNet ResNet50
5
-10
-10
log d(c,) d((e)
log d(ss)
log d(ci)/lles)
log d(ss)/M(ss))
log d(cs)/iles)
log d(ci)/dtea)
61
S 20c
10c1
S 1062
20
Figure SL.3: Principal distortions for ResNet50 and AlexNet for six example base images (s, middle row). Each row shows base images with differently scaled additive perturbations of 6 and 62- Distortions (2) are shown in isolation in the top/bottom rows, respectively. The log ratio plot for each image is at the top of the column.
21

---

Published as a conference paper at ICLR 2025
Proportion of images maintaining same label as original image
C1
1.0
0.8
AlexNet ResNet50
0.6
0.0
0.5 k
1.0
0.0
0.5
1.0
K₂
Figure S1.4: Adding a principal distortion to the base image changes the prediction of one DNN but not the other. Principal distortions were generated for intermediate layers of the AlexNet and ResNet architectures and were computed for a set of 72 base images from the ImageNet validation set (chosen from images included in ImageNet-9. Xiao et al., 2021). For each image and principal distortion e, the distorted images s + k,E,, with k varying between and I, were classified by ResNet50). The left plot shows
the DNNs used for the principal distotion generation (AlexNet the proportion of images for which each DNN had the same classification for the distorted image + K1 the base image s, for h1 varying between 0 and 1. The right plot is the same, but with 62 and k2 in place of E and k).
Comparison with multiple training seeds (N=72 Base Images)
A
Figure S1.5; Principal distor- tions generated for multiple ran- dom initializations of AlexNet and ResNet50 consistently sep- arate models by architecture type. A) Log sensitivity ratios of principal distortions for net- works trained on standard Im- ageNet when comparing image representations at multiple lay- ers of AlexNet and ResNet50, using three random initializa- tions of each architecture. Prin- cipal distortions were computed for set of 72 base images from the ImageNet validation set (chosen from images in- cluded ImageNet-9, Xiao al., 2021), and the mean ratio across these 72 images is plot- ted (error bars are standard de- viation). B) Example base im- age, principal distortions, and distorted images. The princi- pal distortions are qualitatively similar to those observed from a single architecture in Fig. 3. C) Proportion of images with changed prediction class (com- pared to image s) for the dis- torted images s + kçe,, Results are similar to trends for sin- gle seeds of models from Supp. Fig. SI.4.
AlexNet (Seed 1)
Model Layer Late
AlexNet (Seed 2)
AlexNet (Seed 3)
ResNet50 (Seed 1)
ResNet50 (Seed 2)
ResNet50 (Seed
Early
-10
-5
5
10
log d(<r)/d(ec)
20
20
AlewNei sensitive to L regardless of seed
20 62
S 20
ReeNer50 more sensitive to regardiess
Proportion of images maintainning sanee label ass original image
1.0
AlexNet (Seed
AlexNet (Seed
AlexNet (Seed 3)
0.8
ResNe150 (Seed
ResNet50 (Seed 2)
0.6
ResNet50 (Seed 3)
0.0
0.5
1.0
0.0
0.5 k,
1.0
K
22

---

Published as a conference paper at ICLR 2025
A
Base Image Texture Surround
B
Base Image
C Base Image Full Texture
Texture Center
AlexNet
HI00
HIOD
ReeNer50
-10
-10-5
10 110
5 10 110 -6 10
-00
-10 -00-500
10 10
10 10 10
log dies)/d(ea) log d(ee)/A(ea)) log d(cs)/d(es)
log d(es)/d(es)
log ((ts)/d(es))
log d(es)/d(e))
log d(ss)/
61 AlexNict eensitive
20k1
10c
$ 60
8 62
62 ResNat50 more
Figure SL.6: Principal distortions for ResNet50 and AlexNet for base images that have been con- structed to have either (A) texture only in the periphery and blank center (B) texture only in the center and blank surround and (C) a full image of texture. The log ratio plot for each image is at the top of the column. Panels A and B highlight that the AlexNet architecture is more sensitive to the perturbation with power around the "stuff" of the image (i.e., the non-blank areas), while ResNet50 more sensitive to a distortion focused on blank areas of the image, and these perturbations are not sensitive to the choice of low contrast ("black") or high luminance ("white"") for the blank part of the space. Panel C highlights that in the case where there is not obvious blank area of the image, the perturbations become harder interpret, and this reflects the locally adaptive property of the to the input image.
23

---

Published as a conference paper at ICLR 2025
Base Image Example
Base Image Example 2
Base Image Example
Base Image Example 4
Base Image Example 5
Base Image Example
AlexNet
ResNet550
AlexNet (SIN)
ResNat50
-10
D
10
-10
10
-10
5
10
-00
-5
10
-10
-5
-10
log d(es)/d((.)
log ((as)
log d(rs)/d(ca)
log d(ss)/dd(s))
log d(c))/4(a))
log d(ri)/H(oa)
S 20k
8+ 10c1
1062
s 20
62
Figure SI.7: Principal distortions for standard ImageNet trained and Shape Image Net Trained (SIN) AlexNet and ResNet50 architectures for example base images.
24

---

Published as a conference paper at ICLR 2025
A
Principal distortion amplitudes in foreground images (ImegeNat vs. SIN Experiment)
brdd
primate
dog
Original (Orig)
Blur Background Blur Foreground (Blur BG)
(Blur FG)
10
2 czz
Base
Image
Amplitude (per pixel) in Foreground
Amplitude pixel) Foreground
10
musical instrument
reptie
fish
COD
014
to
wheelled vehicle
carrivore
insect
10
2009
10
Reabac50
Orig
Blur
Blur
10.0
Resfect 10
BG
FG
10
log ((t1) (8(e)
C (AlexNet more sensltive)
Org OrigiurBur
62 (ReeNel50 more seneltive)
Figure SL88: Control experiment dissociating foreground and background from high- and low-spatial frequency regions. (A) Example natural base image, image with a blurred background, and image with blurred foreground. Images were chosen from ImageNet-9 (Xiao al., 2021), which contains masks of the "foreground" parts of the image. Images are 224 x 224 pixels. To construct the blurred variants, a Gaussian filter with a is was applied to the full image, and we used the provided image masks to construct the Blur Background (Blur BG) condition by adding the original image foreground to the blurred image background, and the Blur Foreground (Blur FG) condition by adding the original image background to the blurred image foreground. The associated optimal distortions are shown for each image, generated from the layers from AlexNet, ResNet50, SIN- Trained AlexNet, and SIN-Trained ResNet50. In the example image, the distortion AlexNet is most sensitive to (c1) is biased towards higher frequency (non-blurred) parts of the image, while the distortion ResNet50 is most sensitive (E2) is biased towards low-frequency (blurred) parts of the image. We quantified this observation in (B) by masking the perturbation with the foreground mask, and measuring the average squared value of each perturbation this region. If perturbation is localized to the foreground, it will have a high perturbation magnitude on this plot. Each point on the plot corresponds to one
randomly selected images from ImageNet-9 (Xiao et al., 2021), I), where 8 images were selected from each category, and the box and whisker plots are defined with the median given by the line across the box, the box extends from Q1 to Q3, and the whiskers extend to 1.5 IQR. Although there is slight bias for distortion E (defined the perturbation AlexNet is more sensitive to), to have more power the foreground biases of natural images where often times the background is blurred. If blur the background, we see that this difference is exaggerated, while we blur the foreground, the AlexNet perturbation shifts be less concentrated in the foreground while the perturbation ResNet50 is more sensitive (E2) becomes more concentrated in the foreground. Points for individual categories are shown in (C), which show the same trends, and results look similar when just including AlexNet and ResNet50 models in principal distortion generation (not shown).
the image, this is likely due to the general
25

---

Published as a conference paper at ICLR 2025
Base Image Example
Base Image Example 2
Base Image Example
Base Image Example 4
Base Image Example 5
Base Image Example
AlexNiot ResNer50
AlexNeet
0-000
ResNet50
-10
10
-10
10
-10
5
10
-5
10
-10 -5
-10
log dlcs)/d(c.)
log (((s)/
log d(ri)/d(ea)
log d(es)/dd(s))
log d(c))/((s))
log d(si)/H((a)
S 20k
8+ 10c
10
+ 2
Figure S1.9: Principal distortions for standard ImageNet trained and adversarially trained AlexNet and ResNet50 architectures for example base images.
26

---

Published as a conference paper at ICLR 2025
B
20 C1
$ 61
Principal Distortions: Optimization constraint removed
Alewee more sensilve
Model Layer
AlexNet
ResNet50
20 Cg
+ 20Cg
Early
log ((())/(((2))
ResNatto more senestier
Figure 10. Example principal distortions for ResNet50 vs. AlexNet comparison where the pixel- wise min/max value for the perturbation has not been constrained to avoid clipping once the pertur- bation is scaled. Similar to the results in the main text, we see that the principal distortion to which AlexNet more sensitive focused on the part of the image with high-contrast texture features, while ResNet50 is more sensitive to the perturbation targeting relatively smooth parts of the image.
27


Reviews:
[{"review_id": "ZWvclA96cJ", "reviewer": "Reviewer_WQar", "summary": "This paper introduces a novel metric on image representations to measure differences in local geometry. The authors then leverage this metric to generate \"principal distortions\" that maximize variance across a set of models. Experimental results of the proposed method reveal qualitative differences in the local geometry of ResNet50 and AlexNet, which is quite interesting.", "questions": "- How is the stimulus-dependent function $f(s)$, defined after Eq. (1), computed in practice?  \n- If $I(s)$ is positive semi-definite, this seems to cause issues with the metric defined in Eq. (3) when it approaches zero.   \n- How are two image representations obtained from a single image $S$? Are $I_A(S)$ and $I_B(S)$ learned by different neural networks? Please clarify. \n- How are the coefficients for $\\epsilon_1$ and $\\epsilon_2$ determined in the proposed approach?  \n- The interpretation and justification of the estimated principal distortions are difficult to justify. For instance, in Fig. 3, while the finding that AlexNet and ResNet are more sensitive to complementary parts of the images is interesting, its validity is unclear. Providing additional justification would strengthen the claims. One possible approach is to introduce adversarial noise to different parts of the images and evaluate its impact on downstream classification tasks, e.g., comparing the performance drop across different parts of the images.  \n- This reviewer finds it difficult to connect the identified distortions with the concept of local geometry. The method appears to focus more on differences in image intensities and textures. However, it is possible that this reviewer has misunderstood some aspects of the proposed approach. Please clarify.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- This is well-written and organized paper; hence it is easy for readers to follow. \n \n- The concept of exploring the local geometry of deep networks and examining the interplay between local geometry and the global structure of images is compelling.\n\n- Developing a novel metric to compare two image representations is highly innovative.", "weaknesses": "- While the proposed method of using the Fisher Information Matrix to measure the sensitivity of a representation to a stimulus distortion seems reasonable, its effectiveness as a metric is unclear, or difficult to justify.\n\n- Identifying the types of principal distortions to which the network is most sensitive is an interesting idea. However, the proposed method lacks a good validation plan to confirm the accuracy or reliability of these findings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HoheaoPrv2", "reviewer": "Reviewer_7znR", "summary": "The paper proposes a novel framework to compare local geometry within image representations, arguing that image representation encompasses both global and local geometry information. The authors introduce a method to quantify the local geometry of image representations using the Fisher information matrix and statistical techniques sensitive to local stimulus distortion. This approach aims to identify \"principal distortions pairs,\" which maximize model variances and serve as optimal discriminative tools for evaluating model performance.", "questions": "Model Selection: The authors demonstrate the metric’s functionality using older architectures, specifically AlexNet and ResNet. Given that AlexNet is largely obsolete in current practical applications, the paper would benefit from extending the evaluation to more contemporary and widely-used networks (e.g., EfficientNet, Vision Transformers). Demonstrating the framework's effectiveness across a variety of modern architectures would strengthen the claim that the metric is universally applicable and capable of distinguishing models.\n\nPractical Utility and Generalizability: The manuscript does not clearly establish the practical relevance and utility of this framework in real-world applications. A more explicit discussion on the potential benefits of this metric in actual deployment scenarios, or in improving model interpretability and selection, would add significant value. Additionally, empirical results supporting the framework’s effectiveness across diverse, practical model architectures are essential to substantiate the generalizability and robustness of the proposed approach.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The framework's focus on comparing local geometry is innovative, and the use of the Fisher information matrix and sensitivity to local distortions represents a unique approach to quantifying image representation. This novel method to derive principal distortion pairs that maximize model variance offers a potentially valuable tool for model discrimination.", "weaknesses": "Model Selection: The authors demonstrate the metric’s functionality using older architectures, specifically AlexNet and ResNet. Given that AlexNet is largely obsolete in current practical applications, the paper would benefit from extending the evaluation to more contemporary and widely-used networks (e.g., EfficientNet, Vision Transformers). Demonstrating the framework's effectiveness across a variety of modern architectures would strengthen the claim that the metric is universally applicable and capable of distinguishing models.\n\nPractical Utility and Generalizability: The manuscript does not clearly establish the practical relevance and utility of this framework in real-world applications. A more explicit discussion on the potential benefits of this metric in actual deployment scenarios, or in improving model interpretability and selection, would add significant value. Additionally, empirical results supporting the framework’s effectiveness across diverse, practical model architectures are essential to substantiate the generalizability and robustness of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yhko97LgMR", "reviewer": "Reviewer_s6Hs", "summary": "This paper proposes a framework for comparing image representations regarding their local geometries through the Fisher information matrix by finding a pair of “principal distortions” that maximize the variance of the models under this metric. The experiments include (1) comparing a set of simple models of the early visual systems and 2) comparing a set of deep neural network models to reveal differences in the local geometry that arise due to architecture and training types.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea is simple but effective -- as the first approach to compare more than two models.\n2. The problem statement and method description are well-written and clear to understand.\n3. The experiment result is interesting, with meaningful discussions about texture bias and adversarial vulnerability. Visualizations are very helpful. \n4. The supplementary is very informative.\n\n.", "weaknesses": "1. It is not clear how this paper's principal distortions relate to human sensitivity. There seem to be no experiments to prove this statement, such as using human observers for evaluation of the principle distortions [1].\n2. For example, in 4.1 Early Vision Models, it is not stated how to determine the effectiveness of this approach. There is no quantitative evaluation, and it is unclear how to understand the visualizations of the principle distortion images. \n3. Are there some practical applications of this approach? Or how to use visualizations of principle distortions?\n4. What kind of networks can be compared with this approach? Only networks for classifications? How about the models for detections, segmentations, and other applications?\n5. This paper is over nine pages.\n\n[1] Alexander Berardino, Johannes Balle, Valero Laparra, and Eero Simoncelli. Eigen-distortions of ´\nhierarchical representations. Advances in Neural Information Processing Systems, 30:3531–3540,\n2017.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "O72GFz3WuI", "reviewer": "Reviewer_zmdX", "summary": "The paper introduces a new method for comparing vision models that map input images to stochastic representations, leveraging the Fisher Information Matrix (FIM) to analyze the local geometry of image representations. This approach allows for the comparison of multiple models, including computational models of the visual system and deep neural networks (DNNs). By focusing on relative sensitivities to principal distortion directions in the stimulus (image) space, the method reveals differences between models that are not captured by global geometric approaches such as representational similarity analysis (RSA). The authors demonstrate the utility of their method through computational studies, showing consistent findings with previous research on early visual system models and uncovering differences in local image representations between models like AlexNet and ResNet50.", "questions": "1. How do you plan to extend this work to achieve more quantitative results? For instance, do you have ideas on quantifying the distinction between \"noisy\" versus \"smooth\" image distortions?\n2. The authors essentially present a metric between models, which is based on the model’s representations for a single stimulus. It seems to me that the results and conclusions drawn from them could potentially disagree for different stimuli. How do you either choose a representative stimulus or how do you choose multiple and how could the results be aggregated?\n3. How does your approach compare to, or how could it be integrated with, explainability methods based on saliency or relevance maps?\n4. Could you clarify how you scaled the relative sensitivities between models of the early visual system (as mentioned on pages 5 and 6)? A revision of this explanation might enhance understanding.\n5. Can you speculate on how the observed differences in sensitivity to noisy distortions between AlexNet and ResNet50 relate to their architectural differences or inductive biases?\n\nAdditional feedback:\n- For deterministic representations in DNNs, the application of the FIM might seem less direct. It would be beneficial to elaborate on this aspect, perhaps by relating it to concepts like the pullback of the Euclidean metric onto the image space.\n- Strengthening the discussion of experimental results and their key takeaways would enhance the paper. Specifically, clarifying how principal distortions inform us about a model's alignment with human visual representation, responses to adversarial examples, and contributions to interpretability would be valuable.\n- Introducing more quantitative analyses or metrics could solidify the results and provide stronger evidence of the method's effectiveness.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is clearly written and easy to follow, with a strong motivation that outlines how it builds on previous work.\n- Presents a new method to compare the local geometry of image representations between models, generalizing prior approaches to accommodate multiple models.\n- The method uncovers differences invisible to global geometric approaches, providing deeper insights into model behaviors.\n- Includes several computational experiments that support the method's effectiveness and relevance.\n- Discusses potential applications in studying biological vision, DNNs, and the interplay between them.\n- Offers a well-rounded discussion that contextualizes the findings within the field.", "weaknesses": "- The technical advancement may be seen as an incremental improvement over existing metrics.\n- The method primarily applies to stochastic representations, which are uncommon in DNNs; applying it to deterministic models may seem arbitrary.\n- It is not entirely clear how the proposed local metric can be aggregated to provide global insights across different stimuli or datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RwaGD7jlXn", "reviewer": "Reviewer_2NDt", "summary": "The paper proposes a method to compare image representations by analyzing local geometries using the Fisher information matrix, enabling differentiation of models based on local sensitivity to distortions. For comparison, it has been applied to both early visual system models and deep neural networks, the method enables comparison of multiple models (>2) and can be used to compare model representations with human perception.", "questions": "mentioned everything in weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "I found the paper's topic interesting and overall the language of the paper was clear. The paper targets an important problem in cognitive science and is a step toward understanding visual models better. The figures and qualitative results were sufficient to understand the concept and the differences. The paper is also based on a sound experimental framework to investigate the source of variability in results from both training data and architectural design.", "weaknesses": "I have a few questions and would like to know the authors' response to them:\n\n1. (minor) It has been claimed in Figure 4 that the differences between AlexNet and ResNet come from the architecture than the training procedure. There is always randomness in the training where initialization of the model's weights and also the random split of data can effect one single model's performance. In fact, the model output may change if we train it on different seeds. I am wondering wouldn't that be a better approach to replicate the results on models trained on at least three different seeds? (basically 3 different versions of a model)\n\n2. (minor) I expected to see a candidate from Transformers in the experiments, however, there was no candidate from those (currently) popular models. Can authors justify why they have not included any transformers such as ViT or Swin?\n\n3. With Transformers, it has been a trend to show the attention maps (on tokens of an image) and show the behavior of the model. I am wondering if the authors can comment on this. Can the distortion maps be correlated to attention maps? This can ideally (if correlated) help with the explainability of attention-based models.\n\n4. (major) Currently the manuscript is heavily populated with qualitative results. I strongly suggest the authors add a per-class sensitivity distribution for the dataset they are using for different models. This helps to understand on a population level and over different classes which models are more sensitive to high freq. localities and which ones are not.\n\n5. I could not find a Dataset section in the manuscript :) Adding an independent dataset section would make it easier to understand the scale of experiments. currently, it is hard to find within the text", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)