Dimensionality Mismatch Between Brains and Artificial Neural Networks
Authors: Santiago Galella
Maren Wehrheim
Matthias Kaschube
Venue: NeurIPS 2025 poster ()
Keywords: representational geometry
fMRI
convolutional neural networks
brain-model alignment
Primary area: neuroscience_and_cognitive_science
TL;DR: In human vision, representational dimensions continually expand to enable abstraction, whereas artificial networks undergo late-stage collapse that may constrain their capacity for flexible generalization.

Abstract: Biological and artificial vision systems both rely on hierarchical architectures, yet it remains unclear how their representational geometry evolves across processing stages, and what functional consequences may arise from potential differences. In this work, we systematically quantify and compare the linear and nonlinear dimensionality of human brain activity (fMRI) and artificial neural networks (ANNs) during natural image viewing. In the human ventral visual stream, both dimensionality measures increase along the visual hierarchy, supporting the emergence of semantic and abstract representations. For linear dimensionality, most ANNs show a similar increase, but only for pooled features, emphasizing the importance of appropriate feature readouts in brain–model comparisons. In contrast, nonlinear dimensionality shows a collapse in the later layers of ANNs, pointing at a mismatch in representational geometry between the human and artificial visual systems. This mismatch may have functional consequences: while high-dimensional brain representations support flexible generalization to abstract features, ANNs appear to lose this capacity in later layers, where their representations become overly compressed. Overall, our findings propose dimensionality alignment as a benchmark for building more flexible and biologically grounded vision models.

Dimensionality Mismatch Between Brains and Artificial Neural Networks
Santiago Galella
Maren Wehrheim
FIAS & Institute of Computer Science Goethe University Frankfurt galella@fias ias uni-frankfurt.dde
Mila & Department of Biology York University
maren.. www.wewwwwweml..aeeeeeeee...meee quebec
Matthias Kaschube FIAS Institute Computer Science
Goethe University Frankfurt
kaschube@fias uni-frankfurt.co
Abstract
Biological and artificial vision systems both rely hierarchical architectures, yet it remains unclear how their representational geometry evolves across processing stages, and what functional consequences may arise from potential differences. In this work, we systematically quantify and compare the linear and nonlinear dimen- sionality of human brain activity (fMRI) and artificial neural networks (ANNs) during natural image viewing. the human ventral visual stream, both dimension- ality measures increase along the visual hierarchy, supporting the emergence of semantic and abstract representations. For linear dimensionality. most ANNs show similar increase, but only for pooled features, emphasizing the importance of appropriate feature readouts in brain-model comparisons. In contrast, nonlinear dimensionality shows collapse in the later layers of ANNs, pointing mismatch in representational geometry between the human and artificial visual systems. This mismatch may have functional consequences: while nigh-dimensional brain repre- sentations support flexible generalization to abstract features, ANNs appear to lose this capacity in later layers, where their representations become overly compressed. Overall, our findings propose dimensionality alignment as a benchmark for building more flexible and biologically grounded vision models.
Introduction
The geometry of functional representations has been linked to core computational properties of both biological and artificial systems, including accuracy, flexibility, and generalization [e.g.. [2 21 25
43 61 [7]]. One key geometric property is representational dimensionality. which quantifies how
neural activity embedded low- or high-dimensional subspaces in response to different stimuli or Various metrics have been proposed to quantify the representational dimensionality Linear measures such as the effective dimensionality describe how variance spread globally, while nonlinear estimates (intrinsic dimensionality) capture the number of degrees of freedom needed to locally approximate the data manifold. Understanding how these geometric properties evolve along the visual hierarchy is critical for identifying functional distinctions across processing stages and for aligning representational principles between brains and models.
tasks 2
[e.g.,
67 58
In machine learning, the manifold hypothesis posits that high- high-dimensional inputs lie on low- dimensional manifolds embedded in ambient space (6) ANNs leverage this by compressing task-
Joint supervision
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

relevant information into compact, discriminative subspaces 4 Prior work has shown that neural networks typically exhibit an expansion expansion-compresse compression pattern of dimensionality across depth: early layers increase dimensionality to represent diverse features, while deeper layers compress repre- sentations to facilitate categorization [1 61 67]. While such compression is linked to improved classification performance (23)). high intrinsic dimensionality has been associated with improved generalization to novel categories [21], raising the possibility that over-compression may impair semantic abstraction.
In the brain, it remains unelear how representational geometry changes along the processing hierarchy, While early sensory representations are traditionally considered to occupy low-dimensions subspaces (26) [0) [40], recent work using naturalistic conditions reveals that neural activity often resides on
high- righ-dimensional manifolds across the whole brain, even in early sensory areas [3] [00 [2) [88 62) (7).
Distinct dimensions were shown to encode not only sensory input but also abstract semantic and
linguistic information [39 38 14] Moreover, several studies have produced diverging hypotheses
about how representational dimensionality evolves along the visual hierarchy. While studies in
mice and zebrafish found continuous expansion of the dimensionality 17 77). others reported a compression-expants expaassionn pattern in macaques [67]. These contrasting patterns highlight the open question of how the human brain organizes information across processing stages in the visual cortex and how this compares to state-of-the art ANN representations.
Here, we investigate commonalities and differences in representational geometry between biological and artificial vision systems, and examine possible functional consequences of such differences for visual abstraction and generalization. We use the THINGS dataset, which contains fMRI data from humans viewing naturalistic images spanning multiple object categories, along with human ratings of these images on abstract attributes such as 'natural' and 'heavy' 34 69]. We compute both linear (effective) and nonlinear (intrinsic) dimensionality across the human visual hierarchy,
taking cortical distance from the central visual field representation in the primary visual cortex as proxy for position within the hierarchy, and directly compare these profiles to those we obtain for state-of-the-art vision models.
We find that in the human ventral stream, both effective and intrinsic dimensionality increase along the hierarchy, suggesting an expansion of representational capacity to support semantic abstraction. In vision models, we find a consistent increase in effective dimensionality, but only if we assess representations in these ANNs using global average pooled features, possibly because this may better reflect the coarse spatial resolution of fMRI. In contrast, ANNs exhibit collapse in intrinsic dimensionality in deeper layers, even when using pooled features, revealing a fundamental mismatch in representational geometry between brains and models. This mismatch suggests functional implica- tions: we find that in the brain, high-dimensional regions are better at encoding abstract, behavioral attributes, whereas in ANNs, despite their increase in categorization, the deeper layers that display the collapse in intrinsic dimensionality tend to show decline in performance on the abstract concepts. These results suggest that nonlinear dimensionality is an important characteristic of representational geometry and that its collapse in ANNs could be associated with limited generalization to abstract concepts.
Overall, our contributions are as follows:
We demonstrate that global average pooled ANN features best align with fMRI-derived linear geometry, highlighting the importance of population-lewe readouts in model- IMI comparisons.
We reveal functional mismatch in representational geometry between brains and ANNs: while both effective and intrinsic dimensionality increase along the human ventral stream, ANNs exhibit marked compression of intrinsic dimensionality in their deeper layers.
We link this mismatch to function by showing that nigher-dimensional representations in the brain are accompanied by superior generalization to abstract, behaviorally rated concepts, whereas nonlinear compression in ANNs tends correlate with lower generalization abilities despite high categorization performance.
Our findings suggest that geometric properties such as intrinsic dimensionality carry functional significance, and aligning these between ANNs and the brain may be key to improving model generalization and interpretability.

---

2 Methods
2.1 Dimensionality
Dimensionality refers to the number of independent dimensions required to describe the data. Here, we define dimensionality in terms of effective dimensionality (ED), linear measure for dimensionality, and intrinsic dimensionality (ID), which is widely used nonlinear measure of dimensionality.
The effective dimensionality (ED), also known as participation ratio, estimates the number of dimensions that capture most of the variance of the data [27 18 21]]. For data matrix A E IR x.M with N images and M features (number of model units vertices), the ED is calculated as:
Σ 1= X₂ 72
ED
=
1i=1 X
(1)
where X; are the eigenvalues of the covariance matrix of A, and n = min(N, M).
The intrinsic dimensionality (ID) estimates the dimensions of the manifold in which the data is embedded [7 18. i.e., the minimal number of dimensions that capture the data distribution locally. can be estimated via maximum-likelihen radius around each data point as a homogeneous Poisson process and relating their expected number the volume of d-dimensional hypersphere 47 51):
(MLE) by modeling neighbor counts within small
log To (x,)) Ty
ID
(2)
n(k
where n is the number of samples in the dataset, k is the h nearest neighbor of the data point, and T3 (1x) is the Euclidean distance from point 2; to its k-th nearest neighbor. Here, we set l to be 20.
Hence, whereas ED reveals the effective number of global variance components, ID reflects local structures complex, curved manifolds that may not be apparent in linear projections. Note that ID can be smaller than ED and vice versa. For instance, a one-dimensional helix spiraling through three-dimensional space has ID 1 but exhibit ED close 3 if its variance is spread roughly evenly across the x. y, and axes; by contrast, filled 3-ball that is stretched along one axis can have ED close despite its true intrinsic dimension being ID =
Note that both of our dimensionality measures are unsupervised, hence they do not require explicit 13 or random categorical [58] information.
2.2
IMRI data
We used the THINGS IMRI dataset [34] to study dimensionality of brain activity while viewing natural images. In the THINGS fMRI dataset, three participants (S1-S3) completed 12 scanning sessions each, while viewing a total of 9,840 images. From these, 8,640 were unique image repetitions of 720 object categories, and the remaining 1,200 images comprised 100 repeats. All stimuli were selected from the THINGS database, which contains 26,107 distinct images spanning 1,854 object categories [35].
Additionally, we use BOLDS000 dataset, which contains fMRI data of four different subjects (CSI1-CS14) while viewing natural images of objects (from ImageNet-1K KK and COCO datasets) and scenes (from Scenes dataset) 77]. The total number of images observed by the subjects were 5234, from which 4916 were unique (for the fourth subject, the total number of images was 3108).
For both datasets, we performed surface-based analysis by projecting the beta values linearly from functional volumes onto the cortical mid-thickness surface, between the pial and white matter surfaces, to minimize noise from superficial and deep cortical layers [9. We use the HCP parcellation to subdivide visual cortex into five large processing stages: the primary visual area, early visual area, dorsal stream visual area, ventral stream visual area, and MT+ Complex and Neighboring visual areas, which we term the lateral stream; each larger area is then further subdivided into smaller subregions (see A2.2 In all analyses, we used a searchlight approach where fixed-size spherical (or surface-basse neighborhood mask of contiguous vertices is moved across the cortex. For each
3

---

vertex, we selected a neighborhood of the 50 closest vertices (including the vertex itself). On average, these neighborhoods have a maximal distance close to 7 mm. To compute cortical distances between brain areas, we first identify the vertex for each area that has the minimal sum of distances to all other vertices in that area (the surface centroid). We then measure the distance from the V1 centroid in primary visual cortex to the centroids of the other visual areas.
We calculate local dimensionality using searchlight analysis. While this allows for a local dimension- ality estimate, it also ensures an upper bound of dimensionality across areas, preventing larger regions from artificially inflating dimensionality estimates. We obtain area-specific dimensionality estimates by averaging across all local dimensionality estimates in a specific area. Hence, our approach ensures comparability across cortical areas, while avoiding biases introduced by varying area sizes.
We repeat all analyses using searchlight neighhorhoods of 25 (5 mm) and 100 (10 mm) nearest neighbors (Figs. A14 A15). Additional information on the methods and results for the different subjects are included in Appendix[A4
2.3 Artificial Neural Network Models
In addition to the fMRI data, we study artificial neural networks (ANNs), as potential models of visual processing in the brain. We analyze 36 different ANNs, including convolutional neural networks and vision transformers trained with supervised self-supervised objectives as well as autoencoders to test the influence of unsupervised objectives (see Table A3 for a list of all models and more details on training objectives). Since our study also aims bridge biological and artificial representations, we further include models inspired by biological constraints, such as CORNet-S [45), which is explicitly designed to approximate the primate ventral stream hierarchy. Moreover, to evaluate the role of training data in shaping representational structure, we also include models trained on more ecologically valid datasets, such as Ecoset (54), which mimics human object category statistics, and Ego VLP [48). which learns from egocentric, first-person visual input. This range of models enables us to ask not only how dimensionality differs between ANN architectures, training data, and objective functions, but also whether models with more biologically plausible features (in terms of architecture or data) show greater alignment with the hierarchical dimensionality profiles observed in human ventral visual cortex.
For all models, we extract convolutional feature activations after ReLU activations (GELU for transformer-bast models) for unique THINGS images that were viewed by the subjects in the fMRI scanner. We then apply global average pooling (GAP) to each feature map to focus on conditionally independent image-based features and remove spatial variance (21). To generate comparable results across layers, we repeatedly subsample 50 GAP features. Additionally, we include analyses using i) random projections, that compress the feature dimensions onto 50 random sparse vectors while preserving relative distances between samples [67], and ii) 50 randomly subsampled unit activations (without GAP) to preserve each unit's 's tuning properties [41) in Appendix A7.II We define the depth of layer in the model's processing hierarchy by dividing the layer's position by the total number of
all processing steps in the model [ 67
3 Results
3.1 Diverse dimensionality throughout cortex
We first examine how representational dimensionality varies across the human visual cortex by analyzing whole-brain IMRI data from the THINGS database, where participants passively viewed thousands of naturalistic object images. Specifically, we calculate the local dimensionality for all vertices across visual cortex using a surface-basee searchlight approach D. We computed both effective dimensionality (ED) and intrinsic dimensionality (ID) of the vertex-wise responses as complementary measures to quantify the geometry of fMRI representations: While ED captures how the variance is distributed across linear response dimensions, ID estimates the underlying degrees of freedom required to describe the nonlinear manifold the data resides in. We found that dimensionality is highly diverse across regions in the visual cortex (Fig. [a.. Notably, we find that all visual areas show a lower dimensionality compared to areas in the ventral stream (Fig. [T].. Hence, the higher dimensionality in these regions might be associated with their involvement in handling rich, complex

---

a
b
Subject S1
Effectuue Dimansionallyy
Effective Dimensionality
Intrinsic Dimensionality
Smmut
Primary
Early
Ventral
Dorsal
Lateral
Intrinsic Dimensionallty
Soream
Winga
11
10
15
Primary
Early
Ventral
Dorsal
Lateral
Entccrv Dimardionsilyy
VIA
LOT
Intrerto Dimenderratty at
12
10-
V1
V0
PRO VMN
VO
IP81
VBA
PST
L03
Phtrrry
Earty
Ventral
Donsal
Lateral
Figure I: Dimensionality varies across cortex. a) Effective (ED) and intrinsic (ID) dimensionality across the visual cortex for subject S1 from THINGS fMRI dataset. Dimensionality computed using a searchlight approach with 50 vertices (see Methods). Dimensionality averaged across larger brain regions (b) and smaller sub-regions (c). The boxes show the quartiles of dimensionality across searchlight discs in an area.
representations of static objects, and not feature of the processing hierarchy as the dorsal stream's 'ss more specialized role in action and movement exhibits a low dimensionality.
Zooming into these coarse regions reveals dimensionality patterns be spatially specific and re- producible across finer cortical areas, indicating a stable organizational structure in how visual information is represented across the brain (Fig. [T]). Specifically, regions like the Fusiform Face Complex (FFC), responsible for face processing, the Ventromedial Visual Area (VMV), associated with spatial scene processing, and the Ventral Visual Complex (VVC), commonly known for object recognition, show high dimensionality. The higher dimensionality observed these regions may reflect greater differentiation between various object categories and more abstract features com- pared to other visual areas. Despite potential variations in the processing of visual inputs between hemispheres, both ED and ID were highly correlated across hemispheres (Figs. A4 A5 Also. ED and ID in fMRI data were strongly correlated (Figs. A6 A7). suggesting common underlying changes in the geometric properties of the BOLD response along the visual processing hierarchy. Overall, these differences in dimensionality for different areas are consistent across subjects (Fig. A8). Early visual areas exhibit local retinotopic organization, which may reduce dimensionality estimates due to spatial correlations among neighboring vertices. To control for this, we repeated the analysis using randomly sampled, non-contiguous vertices within each region. Despite expected shifts absolute dimensionality values, the hicrarchical trend across regions remained robust (Fig, A17).
5

---

a
function Subject $1
ResNer-S0
Swin B
0.2-
50
Quadratic coefficient, ID
0.0-
20-
-0.2
50 Distance from V1 (mmm)
100
0.5
1.9
0.6
-0.4
Relative Desth
Ra ve Depth
Subject S1
RestNes-00
Swin-B B
-0.8
36-
25-
-0.8
20-
-1.0
-1.2
Dissance from V1 (mmm)
00 100
Rolative Depth
0,5
1.5
0.6 Relatiee Depth
1.0
-1.0
Quadratic coefficient, ED
-0.5
0.0
0.5
Figure 2: Hierarchical changes in dimensionality reveal a mismatch between brains and ANNs a) Effective (ED, top) and intrinsic (ID, bottom) dimensionality for human fMRI data along the ventral visual stream. Values for different areas (using the data from Fig. [bb) are plotted
of geodesic distance along the cortical surface from the primary visual cortex (see Methods). The error bars show the standard deviation across 100 bootstrapped samples from the results of 100 randomly selected vertices per region. b) Dimensionalities for two example ANNs (ResNet50, ViT) with global average pooled features. Relative depth indicates the normalized index of the network layer. The error bars represent the standard deviation of the dimensionality calculated for 100 random subsets of 50 globally pooled features per layer. e) Quadratic coefficients of second-order polynomial fit for ED and ID along the visual hierarchy in humans (blue dots) and models (gray dots).
3.2 Diverging dimensionality trends highlight fundamental differences between brains and ANNs
3.2.1
Dimensionality increases along the ventral stream
To quantify how dimensionality changes across the ventral visual stream, examined how both ED and ID evolve as a function of distance from primary visual cortex (V1). Specifically, we defined visual hierarchy depth as the geodesic distance from the center of V1 each region of interest (see Appendix A6.2 for other measures of visual hierarchy depth). We found that both ED and ID increase with depth (Fig- 24. top and center), often in superlinear manner. This increase might correspond to functional expansion of the brain's 's representation space to more diverse and selective tuning, reflecting untangled, abstract feature dimensions.
3.2.2 Global-average pooling aligns the linear geometry of artificial networks with the brain
Contrary to what we observe in the brain, previous work has consistently reported expan- nion-compression pattern of dimensionality ANNs, characterized by an initial increase followed by decrease across layers [e.g.. 61 67]. To investigate whether this pattern is robust and holds under more naturalistic conditions, we analyze ED using the same large-scale image dataset (THINGS) as for fMRI in 36 different pretrained ANNs, varying in architecture and training objective. When ED is computed using randomly projected or subsampled units, we replicate the previously observed trend across all ANN layers (see Fig. A18). In contrast, when we compute ED using global average pooled (GAP) feature maps, which emphasize image-level over spatial variance, we observe an increase in dimensionality with depth in most models (Fig. 2p top row ResNet-50, Fig. AIT), mirroring the trend observed in the human ventral stream. GAP preserves high-level semantic content and feature identity while discarding spatial detail, potentially paralleling the spatial integration performed by fMRI voxels across millimeter scale cortical areas. We confirm that this increase ED emerges only during training, and not present randomly initialized networks. However, for ViT-based models we again observe a decrease in ED in later layers (Fig. D top row Swin-B) indicating a fundamental difference between geometric representations in the ViT architecture compared to CNNs and the brain. Overall, these results highlight critical consideration

---

for cross coos--dmminn alignment: coarse- grannedd pooled features may more accurately reflect population- level signals in the brain, and thus serve as a better proxy for fMRI recordings in representational geometry analyses.
3.2.3 Divergent intrinsic geometry in brains and artificial networks
Having observed that global average pooling can resolve the mismatch in effective dimensionality between the brain and many ANN architectures, we next ask whether the same holds for ID which more sensitive to the local, nonlinear structure of the representational manifold. As before, we apply global average pooling across feature maps to reduce spatial variance. However, in contrast to the brain, where ID increases along the ventral stream, we find that in all ANNs, ID follows the canonical hunchback shape defined by an initial increase followed by a continuous decrease (Fig. 2p bottom row, Fig. A12 consistent with previous findings [ 61 67]. The dimensionality collapse is not limited to task-optimized layers but persists in those most aligned with late-stage visual areas, including IT (Fig. A20)), revealing a fundamental geometric mismatch between biological and artificial representations that cannot be resolved by pooling alone.
To systematically compare representational trends, in particular the strength of dimensionality collapse, we fit second-order polynomial (i.e., model including both a linear and quadratic term) to the progression of both ED and ID along the visual hierarchy in brains and ANNs. For ED, the coefficients of the quadratic term obtained for the human subjects overlap with those of the ANNs, suggesting overall comparable trends (Fig. 2 However, for ID, we observe a clear divergence: whereas all subjects show positive quadratic coefficient, indicating an upward curved trend where high-level regions maintain a higher ID than other regions, all ANNs show a negative quadratic coefficient, reflecting the mid mid-psscsssing peak and sharp decline (collapse) in ID in their deepest layers. This pattern persists across all tested ANNs, including those with brain- insiind architectures (e.g., CORNet-S [45),, models trained on more ecologically valid datasets such as Ecoset (54), and those trained with egocentric visual experiences designed to approximate human perception [48] Despite these biologically motivated design choices, all exhibit similar late-stage decline in intrinsic dimensionality. Taken together, these findings highlight the relevance of studying the nonlinear representational structure by revealing fundamental difference in how biological and artificial systems represent information towards the higher levels of their processing hierarchies.
In summary, we observe that ED increases along the processing hierarchy in both fMRI data and most ANNs, reflecting growing diversity of linear subspaces. However, while fMRI representations retain high ID in higher-order regions, ANNs exhibit sharp reduction in ID in their final layers. We hypothesize that this reflects task task-specific compression, where the network reduces representational complexity to optimize performance on single objective. To test this possibility, next examine how different types of properties, ranging from visual features to abstract semantics, can be decoded across brain and model hierarchies.
3.3 Low-level image features are represented in early layers
Having shown that dimensionality trends differ in brains and ANNs in higher-order regions, we next examine what type of information is actually represented across the processing hierarchy in both systems. While dimensionality metrics describe the structure of representational space, they do not reveal its content. To address this, we perform decoding analyses test whether regions differ in their ability to represent low-level visual features as well as semantic and categorical content. Specifically, for fMRI data, we first decode the presented stimuli using leave-one-session out cross validation using logistic regression. In line with prior work [e.g., decoding accuracy is highest in primary and early visual areas, especially in regions with low eccentricity (Fig. [4, left). To assess the representation of low-level features more directly, we extract and predict the first principal components of the images (see Supplementary Material), which capture contrast, spatial frequency, and color from the fMRI data using ridge regression. These low-level features are again predominantly represented in early visual areas (Fig. 33,, right), except color (component 4), which is more relevant in later regions. Since stimulus decoding is not applicable in deterministic ANNs, we only evaluate how well low-level image features are encoded across layers. In line with the brain, we find that early ANN layers most strongly encode these features and the performance continuously drops with layer depth (Fig. (3)). similar drop performance for the color component highlights mismatch in how color information is encoded in brains versus models. Together, these results confirm that carly stages
7

---

a
Subject S1 Secand Component
Stimuli
First Component
Third Camponent
Fourth Component
0 235
5,028
0:16
3.01
800
0.077
0.72
0.0
046
2.023
0.015
0.02
0.04
0.012
2.00-
0.01-
01G
0.000
5,009
25
tio
10c
Distance from (mm)
Distance
Distonce
Distaacce
Distance from
b
ResNet-50
375
0.22
332
0.25
Roladie
Rolutie Dosth
Relutve
Figure 3: Decoding low-level features (image components) from brains and ANNs. Decoding stimuli (i.e. the image, left) and image components (2nd left to right) of images for subject S1 from THINGS iMRI dataset (a) and ANNs (b). For the fMRI data, the scatter plots indicate the mean and standard deviation obtained by repeatedly sampling the results (100 times) for 100 vertices per region across the early, primary, and ventral streams (11 regions; see Fig. Te). Cortical distance is measured as in Fig. 2a. For ANNs, the plots show the mean and standard deviation across 10 train-test splits (80/20), each using a subset of 50 averaged feature maps.
in both brains and models reliably encode low-level visual features, supporting purpose, high-fidelity representations of sensory input.
3.4 Decoding performance of abstract concepts aligns with nonlinear dimensionality in the brain and artificial networks
Having established that early stages in brains and ANNs represent low-level image features, we next assess whether higher-order regions encode more abstract, semantically meaningful information and whether this aligns with the observed differences in representational geometry.
In fMRI, we first decode object categories, and find that decoding performance increases continuously along the ventral stream, with category-level information being strongest in higher-order ventral areas, consistent with their established role in object recognition (Fig. Fpb left). To move beyond categorical labels, we next predict human-rated semantic attributes of each image, such as heavy natural, from brain activity [69]. Again, find high prediction performance in ventral but also lateral regions (Fig. b right) with a continuous increase along the ventral stream, highlighting that high-level areas flexibly represent abstract features that generalize beyond categorization.
In contrast, while ANNs generally improve in categorization accuracy across layers, for abstract concepts, this performance trend often plateaus or even declines in deeper layers (Fig. 4D.. This is reminiscent of the collapse in ID (Fig. 2 and we quantify this trend, as before, by fitting a second- order polynomial to the change in prediction performance. For most ANNs we observe a negative quadratic coefficient for abstract feature regression (Fig. Fi), confirming that indeed most ANNs show performance decline in higher layers. Additionally, we find the quadratic coefficients for ID
8

---

Subject S1
Categories
Heavy
Moves
Natural al
Size
b
D.014
0.15
0.15
0.15
0.10
9.10-
0.10
0.10
0.011
100
0.00
0
20
100
60
T00 (mn)
from VI (mm)
Distance from VI (mmm)
(mm)
(mmm)
ResNet-50
0.75
0.
0.50
6--
0.22
0:00
0.5
1.0
8.5
10
2.5
10
Relatvve Depth
Pillaive Desh
Reletive Dooth
Ruladive Dupth
Ralative Deaft
d
All Models
Qui Curt. Canggres
Ouas coots Regreszton
Quad Regmesbon
G.adt coeff Regreteion
Quat coeff Regressoon
3.5
0.5-
3.6
0.6
60--
a..
20
-0.5-
-3.
Quid. Coof.
Quad.
-1.8
coeff
-2.5
Quad, ocef. ID
-1.0
-2.5
Qlad, coef.
-10
25.
Quad, coet.
-15
-0.5
Figure 4: Decoding high-level features (abstract attributes) from brains and ANNs. Decoding categories (left) and abstract attributes from images (2nd left right) for subject SI from THINGS fMRI dataset (a, b) and ANNs (c). For the fMRI data, the scatter plots indicate the mean and standard deviation when repeatedly sampling the results (100 times) for 100 vertices per region across the early, primary, and ventral streams. Cortical distance is measured as in Fig. 2 For ANNs, the plots show the mean and standard deviation across 10 train-test splits (80/20), each using a subset of 50 averaged feature maps. d) The coefficients of the quadratic term of second-order polynomial fit for ID and categorization (left) and abstract feature decoding performance (right) along the visual hierarchy in the ANNs from Fig.
to correlate positively with those of several abstract feature decodings, but not with categorization (Fig. [i), potentially indicating a connection between the strength of ID collapse and the inability to form ordered abstract representations across models. This suggests that late-stage compression in ANNs supports task-specific optimization at the cost of rich, semantically grounded representations that the model was not explicitly trained for.
Our results, thus, suggest a functional divergence: while ANNs compress their representations to optimize for narrow tasks, the brain preserves high-dimensional geometry in higher-order areas, enabling semantic abstraction and behavioral generalization. This underscores the value of aligning not only representations but also geometric properties when building biologically inspired models.
Discussion
This work highlights a fundamental mismatch in representational geometry between the human visual cortex and ANNs. We report continuous increase in both effective and intrinsic dimensionality

---

along the human ventral stream, while ANNs exhibit collapse of intrinsic dimensionality in deeper layers that best align with higher visual areas in the brain. This mismatch appears functionally meaningful: high intrinsic dimensionality in the brain is accompanied by high generalization to abstract, behaviorally rated features, whereas in most ANNs, abstraction performance declines in the deeper layers, despite improved categorization, and, as trend, this decline seems stronger in those models that also show a more pronounced ID collapse.
How ANNs align with the brain is an active field of research, see e.g. [5
35 76
Here, we show that global average pooled features in ANNs align more closely with the population- level geometry observed in fMRI than unit-level features or random projections. However, these pooled representations may not capture fine- fine-grained neural dynamics and are therefore less suitable for comparison with clectrophysiological recordings, where single-unit activations may provide a closer match. While the brain-alignment of effective dimensionality relies on pooled features, the drop in intrinsic dimensionality remains across all preprocessing methods employed. This drop also appears self-supervised contrastive models, suggesting that dimensionality collapse can emerge independently of supervised objectives. These results point to deeper difference in how abstraction is supported across systems: Whereas the brain may promote high-dimensional structure to support semantic flexibility, ANNs tend to optimize for compression, potentially limiting their generalization.
Notably, the use of high- digh-dimensional manifolds in the brain may facilitate flexible readouts in downstream areas, as embeddings into higher-dimensiona spaces can make complex classification problems more likely be linearly separable [6 25]. While nonlinear dimensionality reflects intrinsic representational degrees of freedom, linear decodability assesses how readily features can
be linearly extracted an approach also used to estimate manifold capacity [13 [88. The observed drop in nonlinear dimensionality in deeper ANN layers, which appears to correlate with reduced linear decodability for some abstract features, suggests that representations become increasingly compressed along task-relevant dimensions, limiting access to non-task-related features.
Additionally, it would be interesting to study how the reported dimensionality mismatch emerges during training and whether deeper ANN layers better align with higher-order cortical regions (e.g. prefrontal cortex) [60]. These insights could inform model design strategies that avoid premature compression and better preserve the geometry needed for generalization [29]. At the same time, previous work reports that intermediate layers maximize alignment with measured brain responses 772 50 and can exhibit stronger generalization [75] 66 This is consistent with findings linking higher dimensionality to improved generalization 21 In our results, intrinsic dimensionality also
peaks at intermediate layers, suggesting link between this regime and generalization performance.
Together, our results suggest that the increase in representational dimensionality is not only an architectural feature but a functional signature of abstraction and generalization in vision systems. By uncovering geometric mismatches between brains and ANNs, we argue that future models could benefit by aiming to preserve and increase intrinsic dimensionality throughout their hierarchy. Incorporating dimensionality-ave aware objectives into model training may be key better approximate brain-like flexibility and semantic richness, and thus offer principled benchmark for the next generation of biologically inspired artificial systems.
5 Limitations
One limitation of this study is our use of anatomical distance between region centroids as a proxy for hierarchical processing depth. Anatomical distance does not necessarily reflect functional or structural connectivity, particularly given the recurrent, non-feedforward organization of cortex. Future work could incorporate tractography or connectivity-based metrics to better capture hierar- chical relationships, especially between non-adjacent regions. In addition, the BOLD signal can be influenced by anatomical factors such as sulcal depth [46]. which may bias dimensionality estimates. Another limitation is that the ANNs we analyze are primarily feedforward and static, lacking the recurrent and temporally dynamic computations characteristic of biological vision; this could be addressed by evaluating recurrent architectures trained on temporal stimuli, such as video. Finally. early visual regions in the brain (e.g. VI-V3) are strongly structured by topographic features such as eccentricity and polar angle, which are not explicitly modeled in our approach. Future work could address this by testing topographic ANNs that better reflect the spatial organization of early visual cortex [53].
10

---

Acknowledgments
We thank Bradley Love, Heida Sigurdardóttir, Paolo Muratore, Gemma Roig and all members of the ARENA unit for helpful discussions. This research was supported by the German Research Founda- tion (DFG)- DFG Research Unit FOR 5368 ARENA. MW received funding from the Connected Minds Postdoctoral Fellowship (supported by CFREF), and the Deutsche Forschungssemeinsscatt (DFG, German Research Foundation) 414985841.
References
[1] Alessio Ansuini, Alessandro Laio, Jakob H Macke, and Davide Zoccolan. Intrinsic dimension of data representations in deep neural networks. Advances in Neural Information Processing Systems, 32, 2019.
[2] David Badre, Apoorva Bhandari, Haley Keglovits, and Atsushi Kikumoto. The dimensionality of neural representations for control. Current Opinion in Behavioral Sciences, 38:20-28, 2021.
(3] Silvia Bernardi, Marcus K Benna, Mattia Rigotti, Jérôme Munuera, Stefano Fusi, and C Daniel Salzman. The geometry of abstraction in the hippocampus and prefrontal cortex. Cell, 183(4):954 2020.
[4] Pratik Prabhanjan Brahma, Dapeng Wu, and Yiyuan She. Why deep learning works: A manifold disentanglement perspective. IEEE transactions on neural networks and learning systems, 27(10): 2015.
[5] Charles Cadieu, Ha Hong. Daniel LK Yamins, Nicolas Pinto, Diego Ardila, Ethan A Solomon, Najib J Majaj, and James J DiCarlo. Deep neural networks rival the representation of primate it cortex for core visual object recognition. PLoS computational biology, 10(12))e100396,, 2014.
[6] Lawrence Cayton et al. Algorithms for manifold learning. eScholarship, University of California, 2008.
[7] Nadine Chang, John A Pyles, Austin Marcus, Abhinav Gupta, Michael Tarr, and Elissa M Aminoff, Bold5000, a public fmri dataset while viewing 5000 visual images. Scientific data, 6(1):49, 2019.
[8] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pages 1597-1607. PMLR, 2020.
[9] Yi Chen, Praneeth Namburi, Lloyd T Elliott, Jakob Heinzle, Chun Siong Soon, Michael WL Chee, and John-Dylan Haynes. Cortical surface-based searchlight decoding. Neuroimage, 56(2):582-592, 2011.
[10] Zirui Chen and Michael F Bonner. Universal dimensions of visual representation. Science Advances, 1((7))cedw77697, 2025.
[11] Chi-Ning Chou, Royoung Kim, Luke A Arend, Yuan Yang, Brett D Mensh, Won Mok Shim, Matthew Perich, and Yuo Chung. Geometry linked untangling efficiency reveals structure and computation in neural populations. bioRxiv, pages 2024-02 02, 2025.
[12] Yeo Chung and Larry F Abbott. Neural population geometry: An approach for understand- ing biological and artificial neural networks. Current opinion in neurobiology, 70:137-144, 2021.
[13] Sue Yeon Chung, Daniel D Lee, and Haim Sompolinsky. Classification and geometry of general perceptual manifolds. Physical Review X, 8(3):031003, 2018.
[14] Oliver Contier, Chris I Baker, and Martin N Hebart. Distributed representations of behaviour- derived object dimensions in the human visual system. Nature Human Behaviour, pages 1-15, 2024.

---

[15] Colin Conwell, Jacob S Prince, Kendrick N Kay, George A Alvarez, and Talia Konkle. A large-scale examination of inductive biases shaping high-level visual representation in brains and machines. Nature communications, 15(1):9383, 2024.
[16] Thomas M Cover.
Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE transactions electronic computers, (3):326-334, 2006.
[17] David Dahmen, Stefano Recanatesi, Gabriel K Ocker, Xiaoxuan Jia, Moritz Helias, and Eric Shea-Brown. Strong coupling and local control of dimensionality across brain areas. Biorxiv, pages 2020-11, 2020.
[18] Marco Del Giudice. Effective dimensionality: tutorial. Multivariate behavioral research, 56(3):527-542, 2021.
[19] Jia Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. leee, 2009.
[20] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 1 16x 16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11922 2020.
[21] Eric Elmoznino and Michael F Bonner. nigh-performing neural network models of visual cortex benefit from high latent dimensionality. PLOS Computational Biology, 20(1) 17011922 2024.
[22] Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robust- ness (python library), 2019.
[23] Elena Facco, Maria d' E'rricoo, Alex Rodriguez, and Alessandro Laio. Estimating the intrinsic dimension of datasets by a minimal neighborhood information. Scientific reports, 7(1): 12140, 2017.
[24] Jeremy Freeman, Corey M Ziemba, David Heeger, Eero P Simoncelli, and Anthony Movshon. functional and perceptual signature of the second visual area in primates. Nature neuroscience, 16(7):974-981, 2013.
[25] Stefano Fusi, Earl Miller, and Mattia Rigotti. Why neurons mix: high dimensionality for higher cognition. Current opinion in neurobiology, 37:66-74, 2016.
[26] Juan Gallego, Matthew Perich, Lee E Miller, and Sara A Solla. Neural manifolds for the control of movement. Neuron, 94(5):978 984, 2017.
[27] Peiran Gao, Eric Trautmann, Byron Yu, Gopal Santhanam, Stephen Ryu, Krishna Shenoy, and Surya Ganguli. A theory of multineuronal dimensionality, dynamics and measurement. BioRxiv, page 214262, 2017.
[28] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-traimg trained enns are biased towards texture; increasing shape bias improves accuracy and robustness. In International conference on learning representations, 2018.
[29] Naghmeh Ghanooni, Waleed Mustafa, Dennis Wagner, Sophie Fellenz, Anthony Widjaja Lin, and Marius Kloft. Mitigating spurious features in contrastive learning with spectral regularization. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.
[30] Matthew F Glasser, Timothy S Coalson, Emma C Robinson, Carl D Hacker, John Harwell, Essa Yacoub, Kamil Ugurbil, Jesper Andersson, Christian Beckmann, Mark Jenkinson, et al. multi-modal parcellation of human cerebral cortex. Nature, 536(7615) 2016.
12

---

[31] Gregory Griffin, Alex Holub, Pietro Perona, et al. Caltech- 256 object category dataset. Technical report, Technical Report 7694, California Institute of Technology Pasadena, 2007.
[32] James V Haxby, J Swaroop Guntupalli, Andrew C Connolly, Yaroslav O Halchenko, Bryan R Conroy, M Ida Gobbini, Michael Hanke, and Peter J Ramadge. A common, high-dimensional model of the representational space in human ventral temporal cortex. Newron, 72(2):404-416, 2011.
[33] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[34] Martin N Hebart, Oliver Contier, Lina Teichmann, Adam H Rockter, Charles Y Zheng, Alexis Kidder, Anna Corriveau, Maryam Pariri-Pashkam, and Chris I Baker. Things-date data,, multimodal collection of large-scale datasets for investigating object representations in human brain and behavior. Elife, 12:882880, 2023.
[35] Martin Hebart, Adam H Dickter, Alexis Kidder, Wan Y Kwok, Anna Corriveau, Caitlin Van Wicklin, and Chris Baker. Things: A database of 1,854 object concepts and more than 26,000 naturalistic object images. PloS one, 14(10):a0223792, 2019.
[36] Martin N Hebart, Charles Y Zheng, Francisco Pereira, and Chris Baker. Revealing the multidi- mensional mental representations of natural objects underlying human similarity judgements. Nature human behaviour, (11):117331118) 2020.
[37] Horizon2333. Auto-encoder trained on imagenet. imagenet- autoencoder Accessed: 2025-07-25.
https://github. htt.huubb.cm/oooozoo223333333
[38] Alexander G Huth, Wendy A De Heer, Thomas L Griffiths, Frédéric E Theunissen, and Jack Gallant. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600):453-458, 2016.
[39] Alexander G Huth, Shinji Nishimoto, An T Vu, and Jack Gallant. A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron, 76(6):121--122, 2012.
[40] Mehrdad Jazayeri and Srdjan Ostojic. Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity. Current opinion in neurobiology, 70:113-120. 2021.
[41] Meenakshi Khosla, Alex H Williams, Josh McDermott, and Nancy Kanwisher. Privileged representational axes in biological and artificial neural networks. biorxiv. 2024.
[42] Nikolaus Kriegeskorte, Rainer Goebel, and Peter Bandettini. Information baseed functional brain mapping. Proceedings of the National Academy of Sciences, (03(10):3863-3866 2006.
[43] Nikolaus Kriegeskorte and Rogier A Kievit. Representational geometry: integrating cognition, computation, and the brain. Trends in cognitive sciences, 17(8):401-44 2013.
[44] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
[45] Jonas Kubilius, Martin Schrimpf, Aran Nayebi, Daniel Bear, Daniel LK Yamins, and James J DiCarlo. Cornet: Modeling the neural mechanisms of core object recognition. BioRxiv, page 408385, 2018.
[46] Jan W Kurzawski, Omer Faruk Gulban, Keith Jamison, Jonathan Winawer, and Kendrick Kay. Non-neural factors influencing bold response magnitudes within individual subjects. Journal of Neuroscience, 42(38):7256-725 2022.
[47] Elizaveta Levina and Peter Bickel. Maximum likelihood estimation of intrinsic dimension. Advances in neural information processing systems, 17, 2004.
13

---

[48] Kevin Qinghong Lin, Jinpeng Wang, Mattia Soldan, Michael Wray, Rui Yan, Eric Z Xu, Difei Gao, Rong- Cheng Tu, Wenzhe Zhao, Weijie Kong, al. Egocentric video-language pretraining. Advances in Neural Information Processing Systems, 35:7575-7586, 2022.
[49] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEF/CVF international conference on computer vision, pages 10012-10022, 2021.
[50] Ângela López-Cardona, Sebastián Idesis, Mireia Masias-Bruns Sergi Abadal, and loannis Ara- pakis. Brain-language model alignment: Insights into the platonic hypothesis and intermediate- layer advantage. arXiv preprint arXiv:2510. 17833, 2025.
[51] David JC MacKay and Zoubin Ghahramani. Comments on 'maximum likelihood estimation of intrinsic dimension by e. Levina and P. Bickel, 2005.
[52] Najib J Majaj, Ha Hong, Ethan A Solomon, and James J DiCarlo. Simple learned weighted sums of inferior temporal neuronal firing rates accurately predict human core object recognition 2015.
performance. Journal of Neuroscience, (3339):13402-1333
[53] Eshed Margalit, Hyodong Lee, Dawn Finzi, James J DiCarlo, Kalanit Grill-Spector and Daniel LK Yamins. unifying framework for functional organization in early and higher ventral visual cortex. Neuron, 1 12(14))2435-2455 2024.
[54] Johannes Mehrer, Courtney Spoerer, Emer C Jones, Nikolaus Kriegeskorte, and Tim C Kietzmann. An ecologically motivated image dataset for deep learning yields better models of human vision. Proceedings of the National Academy of Sciences, 118(8)=2001 14171118, 2021.
[55] Gavin Mischler, Yinghao Aaron Li, Stephan Bickel, Ashesh Mehta, and Nima Mesgarani. Contextual feature extraction hierarchies converge in large language models and the brain. Nature Machine Intelligence, ((12):1677-44 2024.
[56] Lukas Muttenthaler and Martin N. Hebart. Thingsvision: A python toolbox for streamlining the extraction of activations from deep neural networks. Frontiers in Neuroinformatics, 15:45, 2021.
[57] Sabine Muzellec and Kohitij Kar. Reverse predictivity: Going beyond one-way mapping to compare artificial neural network models and brains. bioRxiv, pages 2025-08, 2025.
[58) Lorenzo Posani, Shuqi Wang, Samuel Muscinelli, Liam Paninski, and Stefano Fusi. Rarely categorical, always high-dimensional: how the neural code changes along the cortical hierarchy. bioRxiv, pages 2024-11. 1, 2025.
[59] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 9748-8763. PMLR, 2021.
(60] Joséphine Raugel, Mare Szafraniec, Huy V Vo, Camille Couprie, Patrick Labatut, Piotr Bo- janowski, Valentin Wyart, and Jean-Rémi King. Disentangling the factors of convergence between brains and computer vision models. arXiv preprint 150.::2500..822200 2025.
[61] Stefano Recanatesi, Matthew Farrell, Madhu Advani, Timothy Moore, Guillaume Lajoie, and Eric Shea-Brown. Dimensionality compression and expansion in deep neural networks. arXiv preprint /9Xiv 00443, 2019.
[62] Mattia Rigotti, Omri Barak, Melissa R Warden, Xiao-Jing Wang, Nathaniel D Daw, Earl K Miller, and Stefano Fusi. The importance of mixed selectivity in complex cognitive tasks. Nature, 497(7451):585-599) 2013.
[63] Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib J Majaj, Rishi Rajalingham, Elias B Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy. Franziska Geiger, et al. Brain-score: Which artificial neural network for object recognition most brain-like? BioRxiv, page 407007, 2018.
14

---

[64] Shuai Shao, Zeming Li, Tianyuan Zhang. Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale. high-quality dataset for object detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8430-8439, 2019.
[65] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXivv1409 144409955,, 1556, 2014.
[66] Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Patel, Jalal Naghiyev, Yann LeCun, and Ravid Shwartz-Ziv. Layer by layer: Uncovering hidden representations in language models, 2025. URL https://arxiv. org/abs/25002.02113 1.
[67] Ben Sorscher, Surya Ganguli, and Haim Sompolinsky. Neural representational geometry underlies few-shot concept learning. Proceedings of the National Academy of Sciences, 119(43))=220880011, 2022.
[68] Anuj Srivastava, Ann B Lee, Eero P Simoncelli, and S-C Zhu. On advances in statistical modeling of natural images. Journal of mathematical imaging and vision, 18(1):17-33, 2003.
[69] Laura M Stoinski, Jonas Perkuhn, and Martin N Hebart. Thingsplus: New norms and metadata for the things database of 1854 object concepts and 26,107 natural object images. Behavior Research Methods, 56(3):1583-1 1603, 2024.
[70] Carsen Stringer, Marius Pachitariu, Nicholas Steinmetz, Matteo Carandini, and Kenneth D Har- ris. High-dimensional geometry of population responses in visual cortex. Nature, 571(7765):361- 365, 2019.
[71] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.
[72] Mariya Toneva and Leila Wehbe. Interpreting and improving natural-language processing (in machines) with natural wingaage-proeessing (in the brain). Advances in neural information processing systems, 32, 2019.
[73] Antonio Torralba and Aude Oliva. Statistics of natural image categories. Network: computation in neural systems, 14(3):391, 2003.
[74] Greta Tuckute, Jenelle Feather, Dana Boebinger, and Josh H McDermott. Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions. Plos Biology, 21(12):e300266, 2023.
[75] Arnas Uselis and Seong Joon Oh. Intermediate layer classifiers for ood generalization. arXiv preprint arXiv:2504.054611 2025.
[76] Aria Y Wang, Kendrick Kay, Thomas Naselaris, Michael J Tarr, and Leila Wehbe. Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset. Nature Machine Intelligence, 5(12):4415 1426, 2023.
[77] Zezhen Wang, Weihao Mai, Yuming Chai, Kexin Qi, Hongtai Ren, Chen Shen, Shiwu Zhang, Guodong Tan, Yu Hu, and Quan Wen. The geometry and dimensionality of brain-wide activity. eLife, 14, 2025.
[78] Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. cerformance-opimmized hierarchical models predict neural responses higher visual cortex. Proceedings of the national academy of sciences, 11 111(2))8661988))
2014.
[79] Thomas Yerxa, Jenelle Feather, Eero Simoncelli, and Sue Yeon Chung. Contrastive-equivrrinn self-supervised learning improves alignment with primate visual area Advances in neural information processing systems, 37:96045-9610 2024.
[80] KiJung Yoon, Michael A Buice, Caswell Barry, Robin Hayman, Neil Burgess, and Ila R Fiete. Specific evidence of low-dimensional continuous attractor dynamics in grid cells. Nature neuroscience, 16(8) 2013.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have included the paper's contributions in the abstract, introduction and discussion. In the Introduction we have enumerated the main contributions of the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have included and detailed the main limitations of this work in the discussion section.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes] All information necessary to reproduce the experiments and figures is provided the main text, supplementary materials, and accompanying code.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The data analyzed in this study is publicly available. The code used to reproduce the experiments and generate the figures included with the submission.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The main details of the experimental setup are provided in the main paper, with additional information included in the supplementary material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Ys]]
Justification: We report the use of enror bars and the methods used to compute them in the text or figure captions, both the main paper and the supplementary material.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
[Yes]
16

---

Justification: Details of the computational resources required to reproduce the experiments are provided in the supplementary material.
9. Code of ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Yes, have read and adhered to the code of ethics
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
The paper focuses fundamental research in cognitive neuroscience and artificial neural networks, and does not directly involve applications with immediate societal impact. We acknowledge that advances in understanding brain function may inform future work with broader societal relevance.
Justification:
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not involve the release of data models that pose high risk for misuse. As such, no specific safeguards were necessary.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external assets used in the paper, including data and models, are cited accordingly either in the main text, or the supplementary material.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide the code used to generate the experiments and results, accompanied by comments for clarity and reproducibility.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects. 15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
17

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this paper does not involve LLMs as any important component.
18

---

Supplementary Material
A1
Compute resources and code availability
All code and scripts necessary to reproduce the experiments and figures are publicly available at bttps://gitthb. www. wow.sallllllaa/ala// hyt.eelaaaaaaaaaasooaattttt-aaaa brans---baaiimmmmmllee-e
The code is designed to run on a local multicore machine (e.g., an 8-core setup) and supports parallel computing to accelerate processing. While running the analysis on computing cluster can significantly reduce runtime, most of analyses can still be completed within a few hours on standard local machine.
A2 Additional fMRI data information
We replicated the dimensionality results using the BOLD5000 dataset, which contains fMRI data of four different subjects while viewing natural images of objects (from ImageNet- K and COCO datasets) and scenes (from Scenes dataset) 77. The total number of images observed by the subjects were 5234, from which 4916 were unique (for the fourth subject, the total number of images was 3108). We refer the subjects using the same identifiers used in the original publication: CSII, CSI2, CSI3, CS14. We follow the same preprocessing details as for the THINGS fMRI dataset detailed in the main text. In total, we analyze the fMRI data of seven participants in two different datasets (see table A2
Name
Dataset
Sex
Age
Handedness
SI
THINGS
Female
29
Right
S2
THINGS
Male
24
Right
S3
THINGS
Female
23
Right
CSI
BOLD5000
Male
27
Right
CSI2
BOLD5000
Female
26
Right
CSI3
BOLD5000
Female
24
Right
CS14
BOLD5000
Female
25
Right
Table A2: THINGS and BOLD5000 subject information.
In the THINGS MMR experiment, images were cropped to square format and varied in size (average: 996x996 pixels; minimum: 480x480 pixels; <1.8% smaller than 500 pixels). During presentation human participants, images were shown on gray background 10 degrees, with 0.5-degree fixation crosshair overlaid [34]. Images were shown for 0.5 seconds followed by seconds fixation time. In BOLD5000, images were cropped size of 375 x 375
maximum visual angle of
pixels, with maximum visual angle of 4.6 degrees, and were shown for second, followed by seconds of fixation time 7
A2.1
Searchlight approach
To estimate local representational dimensionality in fMRI data, we employed searchlight analysis For each vertex (cortical surface space) or voxel (volume space). we defined a local neighborhood by selecting fixed number of spatially contiguous vertices within specified radius. then computed ED and ID over the multivariate response patterns within each local patch, yiclding vertex-wise or voxel-wise map of local geometry. This approach preserves spatial continuity and captures topographic structure, such as retinotopy in early visual areas.
42 9.
To assess the influence of spatial correlations on dimensionality estimates, we conducted control anal- yses using randomly sampled (non-contiguous) subsets of vertices within each region. While random sampling yielded higher absolute dimensionality values, likely due to reduced spatial autocorrelation, the relative hierarchy of dimensionality across regions remained intact, validating the robustness of our findings.
19

---

A2.2 Parcellation
Here we parcellate the surface-basee signal using the HCP atlas [see Fig. A1 These regions can be broadly subdivided into primary, early, ventral, dorsal, and lateral areas (see A4).
Dorssl
Shremt
Stroom
Eanly
Early
Misued
Kehimen Wisani
Latornl
80
Streant
Streem
Ventral Suraan
Ventrall
Sireem
Figure Al: Visual cortex parcellations from the HCP atlas 30
A3 Additional artificial neural networks information
We include 36 different ANN models in our analysis (Table A5 Most models were trained on ImageNet-IK Ecoset perform object classification (see Table Pretraining Dataset). While ImageNet-1K 1K is one of the most popular dataset to train models for object classification [19], it includes many technical subclasses that do represent how humans would classify objects (i.e.. many exact dog breeds, instead of simply the class dog). Ecoset, by contrast, is curated to reflect categories that are more ecologically relevant to humans, with a focus on naturalistic, commonly encountered objects and animals, aiming to provide more human-like visual learning environment for training neural networks [54 Hence, comparing models trained with both datasets allows for a comparison, how human-centrr categorization has effect dimensionality. We use the thingsvision implementation of Ecosct-trained models [56].
SimCLR was trained using a contrastive self-supervised learning objective on ImageNet-IK: where the model learns bring augmented views of the same image closer in the representation space while pushing apart views from different images 8 Finally, CLIP was trained using contrastive image-tex: alignment objective on large dataset of image cap--aatttnnnoo pairs, learning to associate images with their corresponding textual descriptions (59). Additionally, we analyzed autoencoders (AE) trained on ImageNet- K. Caltech256 [1., and Objects365 (64) to test how the dimensionality changes along the processing hierarchy for models with an unsupervised training regime. Moreover, we included biologically inspired models (CORnets [45]) to our analyses. These models were explicitly designed to mirror the hierarchical organization of the ventral visual stream, providing a direct computational analogue to cortical processing in the primate brain. Finally, we included model trained egocentric video data VEgoVLP; [48) to capture a mode of visual experience more closely aligned with how humans perceive the world, emphasizing naturalistic, first-person perspectives.
We additionally include a ResNet-50 model trained be robust against adversarial examples 222] referred to as ResNet-50 (Adversarial), and ResNet-50 models trained using data augmentation to be unbiased textures (28), referred to as ResNet-50 (Fine-tuned on IN), ResNet-50 (SIN) and ResNet-50 (SIN + IN).
20

---

Region ID
Region Name
Area
VI
Primary Visual Cortex
Primary Visual
V2
Second Visual Area
Early Visual
V3
Third Visual Area
Early Visual
V4
Fourth Visual Area
Early Visual
V8
Eighth Visual Area
Ventral Stream
FFC
Fusiform Face Complex
Ventral Stream
PIT
Posterior Infero-Temppe Temporall Complex
Ventral Stream
VMVI
Ventro-Medial Visual Area 1
Ventral Stream
VMV3
Ventro-Medial Visual Area 3
Ventral Stream
VMV2
Ventro-Medial Visual Area 2
Ventral Stream
VVC
Ventral Visual Complex
Ventral Stream
V6
Sixth Visual Area
Dorsal Stream
V3A
Area V3A Seventh Visual Area
Dorsal Stream
V7
Dorsal Stream
IPS1
Intra-Parietal Sulcus Area
Dorsal Stream
V3B
Area V3B
Dorsal Stream
V6A
Area V6A
Dorsal Stream
MST
Medial Superior Temporal Area
Lateral Stream
LOI
Area Lateral Occipital I
Lateral Stream
LO2
Area Lateral Occipital 2
Lateral Stream
MT
Middle Temporal Area
Lateral Stream
PH
Parahippocampal Area
Lateral Stream
V4t
Area V4t
Lateral Stream
FST
Fundus the Superior Temporal Sulcus
Lateral Stream
V3CD
Area V3CI)
Lateral Stream
LO3
Area Lateral Occipital 3
Lateral Stream
Table A4: Visual cortex regions and areas
For ViTs, we use either i) only the CLS tokens, ii) the average of the rest of the tokens which we also refer to as GAP 20
A4
Additional fMRI results
A4.1
Across subject comparison
While we mainly report detailed results for subject 1 of the THINGS database in the main text, here also show the results for the other two THINGS subjects, as well as all subjects in the BOLD5000 (Figs.
A2 A3 Across all, we observe very similar trends in increases in dimensionality along the visual processing hierarchy of the ventral visual cortex.
A4.2
Dimensionality in the left and right hemisphere
Since the two hemispheres contain different numbers of vertices, we compute the correlation of sampled dimensionality across regions (see Fig. (2)) by averaging over 100 randomly selected vertices per region, repeated 100 times. We observe that the hemispheres in the visual cortex are quite similar, they process slightly different parts of the visual field. We therefore here confirmed that across all subjects and datasets, the dimensionality measures are similar within the left and the right hemisphere (Figs. A4 A5).
A4.3 Effective vs. Intrinsic dimensionality in the brain
We observe highly correlated values for the linear (effective) and nonlinear (intrinsic) dimensionality estimation in the visual cortex for both datasets and subjects (Figs. A6 A7). This is different to
21

---

Model
Architecture
Modality
Training Type
Pretraining Dataset
Objective function
ImageNet- IK
AlexNet
44
54
CNN
Image
Supervised
Ecoset
Obj. Class
ImageNet-
Inception V3 0
CNN
Image
Supervised
Obj. Class
Beoset
ImageNet 1K
VGG
CNN
Image
Supervised
Obj. Class
Ecoset
ResNet-18
CNN
Image
Supervised
ImageNet-I
Obj. Class
ResNet-34 33
CNN
Image
Supervised
ImageNet- I
Obj. Class
ImageNet-IK
ResNet-50 B3
22
CNN
Image
Supervised
Benset
Obj. Class
ResNet-101 63
CNN
Image
Supervised
ImageNet- Ie
Obj, Class
ResNet-152
B3
CNN
Image
Supervised
ImageNet-IK
Obj. Class
SimCLR (ResNet-50)
CNN
Image
Superviue
ImageNet-Il
Contrastive
VIT-B/16 20
VIT
Image
Supervised
ImageNet-
Obj. Class
VTT-1/166 20
VIT
Image
Supervised
ImageNet 1K
Ohj. Class
Swin-S
VIT
Image
Supervised
ImageNet-
Obj. Class
Swin-B B 49
VIT
Image
Supervised
ImageNet- 1
Obj. Class
400M Image-
CLIP (Res(Net-50)
59
CNN
Image + Text
SuperSuuer
Text Pairs
Contrastive
400M Image-
CLIP (VIT-B/16)
VIT
Image Text
SuperSuue
Contrastive
Text Pairs
400M Image-
CLIP (VIT-L/16) 59
VIT
Image Text
Self-Supervised
Text Pairs
Contrastive
Caltech256
AB (VGG16)
CNN
Image
Unsupervised
ImageNet-1 1K
Reconstruction
Objects365
CORnet-Z
CNN
Image Image Image
Supervised
ImageNett
Obj.
CORact-S 45
CNN (Recursent)
Supervised
ImageNet- IK
Obj. Class
CORnet-RT 45
CNN (Recurrent)
Supervised
ImageNet-IK
Obj. Class
EgoVILP
Text
Self -upe-iupe
Ego41)
Contrastive
Table A5: Models information.
the ANNs, where for the layers with highest effective dimensionality (later layers), the intrinsic dimensionality dropped (ef. Fig. 2
A4.4 Dimensionality increase in the ventral stream across all subjects
We further quantified the increase in dimensionality along the processing hierarchy in the ventral stream from primary, to early, to the ventral areas. We quantified the hierarchy using the average geodesic distance from the center of the primary visual cortex. Generally, find increase along the hierarchy for all THINGS and BOLD5000 subjects (Fig. A8
A4.5 Replication of the representation of low- and high-level features in the brain
We repeated all prediction analyses, including stimuli, low-level image features, categories, and abstract concepts, for the other two subjects in the THINGS database. For category decoding, selected 100 of the 720 categories observed by each subject. Each category was represented by 12 different images, resulting
total of 1,200 images. For stimulus decoding, we used a subset of 100 images, each shown 12 times across sessions, also totaling 1,200 image presentations. For decoding categories we use logistic regression, and for decoding stimuli, components and concepts we use ridge regression. All decoding analyses were performed using lcave-one-session sessionnosiit cross-validation.
We again find reduced representation of stimuli and low-level features in later brain areas of the ventral stream (Figs. A9,, A10,, cf. Fig. [3p the main text). The decoding performance of the categories and abstract concepts increases with increasing distance to the primary visual cortex (Figs. A9p, A10p, cf. Fig. 4h in the main text).
A5
Additional ANN results
In the main text we only show the changes in ED and ID along the hierarchy for two example ANNs (ResNet50, ViT). We include the detailed ED and ID results (Figs. ATT A12). as well as for the categorization performance (Fig. A13.
22

---

S1
S2
S3
1000
10
10
10
15
15
15
Figure A2: Effective and intrinsic dimensionality for all three subjects of the THINGS database using 50 vertices.
A6
Robustness analyses in fMRI data
A6.1
Varying radii for the searchlight approach
To test the robustness of the dimensionality computations different radii of the searchlight, report the results with three different radii (25, 50, 100 vertices (see Fig, (A14) that approximately translate 5, 7, and 10 mm of cortex (Table As expected,
find that with smaller radii, the dimensionality is lower (Fig. A15), due to the reduced amount of possible dimensions. However, in both, effective and intrinsic dimensionality, we observe the same increasing trend towards the ventral regions. Again, effective dimensionality varies more than the intrinsic dimensionality.
23

---

CSI1
CSI2
CSI3
CSI4
10
10
10
Figure A3: Effective and intrinsic dimensionality for all three subjects of the BOLDD000 database using 50 vertices.
md 83 5.0
50.00 p-0,000-00
5.0
pi21000000
p-0.500-50
Right Hemisphere
Right Hemisphere
Right Hemisphere
4.5
4.5
4.0-
3.5
3.5
3.0
3.0
3.0
4.5
Left Hemisphere
Left Hemisphere
Left Hemisphere
12.0
p=0.000-00
P-200040
11.75
4-2000012
Right Heghi Hemisphare
Right Hemisphere
Right Hemisphere
11.50
11.5
11.25
11.0
11.00
10.8
10.75
10.8
10.5
10.50
10.4
10.5
11.0
11.5
10.5
11.0
11.5
10.5
11.0
11.5
Hemisphere
Hemisphere
Left Hemisphere
Figure A4: Correlation between the dimensionality of the left and right hemisphere across brain regions for all THINGS subjects. Top: effective dimensionality, bottom: intrinsic dimensional- ity.
24

---

4.50
Right Hemiiggh wer
Right Hemiisshre
lamitoherr
5.0
4.00
3.5-
9-6
4.5-
3.76
Rim
3.25
3,5
2.5
3.00
2.5
3.5
4.5
Homisphare
Left
Left Homisphore
Hemisshore
15.5
/3306-00
10.75
10.4
Right Humbsphere
11.0
Right Hemisphere
Right Humisphhue
10.00
Right Hemisshere
10.2
10.25
10.0-
10.5
10.0-
10.00
9.8-
6.76
9.5
9.50
8.4-
10.0
10.5
11.0
11.5
9.5
10.0
10.5
11.0
10.0
10.5
11.0
9.5
10.0
10.5
Left Hemisphere
Left Hemmsshhee
Loft Hemisphhree
Left Hemrishheee
Figure A5: Correlation between the dimensionality of the left and right hemisphere across brain regions for all BOLD5000 subjects. Top: effeetive dimensionality, bottom: intrinsic dimensionality.
S1
20
20
Intrinsic Dimensionality
Intrinsic Dimrensionellty
Intrinsic Dimeensionalltt
16
15-
15
10
10
10
12
Dimensionalily
Dimensionality
Effective Dimensionallly
Figure A6: Correlation between effective and intrinsic dimensionality vertices for THINGS subjects.
CSI1
CB12
CSIS
CS14
Ehective Oimenstonalty
Efeetiv
EReccbee Dimersionally
Eteccive Cimonsloraity
Figure A7: Correlation between effective and intrinsic dimensionality of vertices for BOLD5000 subjects.
25

---

51
52
93
CSR1
CSR2
CCR
CCM
3x
Drtt Dean prer
Cartial Deahior
Cartical Drclh pemo
Cantur pank
Cattea Digh yort
Deghh 2014
Centiall
Figure A8: Increase in effective (top) and intrinsie (bottom) dimensionality in all subjects of the THINGS and BOLD5000 databases. Cortical distance is computed as the average geodesic distance between the center of V1 to the other areas. Each dot represents area.
a
Subject S2
Stimuli
First Component
Second Component
Third Component
Fourth Component
0.024
0.20
0.000
0.08
cV Aocuraey
0.16
0.045
0.12
0.016
0.02
0.04
0.00
0.012
0.04
0.01-
0.02
0.008
50
130
0.00
50
100
-0.066
0
50
100
50
100
0.00
50
500
Distance from V1 (mm)
Dietenoa from V1 (mm)
Dialanos from (mm)
Dislance from V1 (mm)
Dislance from V1 (mm)
b
Categories
Heavy
Moves
Natural
Size
0.0185
0.3 po
03
0.3
Accurazy
0.0150
0.2
0.2
0.2
0.0188
0.0120
0.1
0.0105
0.0000
60
100
00
100
50
100
50
100
0
60
100
Distance from V1
(mm)
Datennaa from V1 (mm)
Distance from V1 (rm)
Distance from V1 (mm)
Dattance from V1 [mm]
Figure A9: Low-level (a) and high-level (b) feature decoding in THINGS Subject 2.
26

---

a
Subject S3
Stimuli
Component
Second Component
Third Component
Fourth Component
0.018
0.00
0.07-
Cy Accuracy
0.000
0.046
0.016
0.02
0.030
0.012
0.075
0.01
0.010
50
130
90
100
50
100
0.00
60
100
0,000
60
100
Distance from V1 (mm)
Distance from V1 (mm)
Distance from V1 (mm)
Distance from V1 (mm)
Distonce from (mm)
b
Categories
Heavy
Moves
Natural
Size
0.0150
0.3
0.3
Accuracy
0.2
0.2
0.2
02
000200
0.1
0.0128
50
100
50
100
50
100
00
0
50
100
Distanca from V1 (mm)
Distance from V1 (mm)
Distance from V1 (mm)
Distance from V1 (mm)
Dislance from V1 [mm)
Figure A10: Low-level (a) and high-level (b) feature decoding in THINGS Subject 3.
27

---

AJooNEt
Incoption
REsNNC-18
RosNec
RISNEE-S
RosNNE-10 10L
67-
40.0
30 edlr
50
83 10-
0
8
20
22
Depth
Teathe Copth
Dapty
Eadathe Dusth
Resket-152
VOG16
Swin-B
Swin-S
MTF//1 (CLS)
VITB/16 (GAP)
62--
(wotter at 40:25
66
pahooe1 lie 0.00
10:4
On 11
30-
30
20
31-
10
3.5
2.0
0.5
0.5 Redilet-DD (SNN + IN]
6.5
VIFU26 (CL.5)
VIFUI (GAP)
Reshae-5 (Adivenscie))
Reshei-50 [5:N]
Bendlai--D (SIN + IN, (N)
2012 42t 12.02
60
and 2012 s0.00 c20
1 -0 0.29
[ 12.00-2.00-----
43-
30
3.22
38
10-
35
1.0
05
1.
05
3.0
1.0
0.5
1.0
05
3.0
Depth
StmCLR RusMet 50
CLIP 50
CUP B/15 (GAM)
CLI VIT (GAP)
CUP B/25 ICLS)
CLIP VTTU4 (CL5)
42-
46-
32-
10
10
05
1.
0/5
16
64
25
05
16
65
3/
Tastie Depth
nedeiioo Dusch
Puladre Duath
Seltive Dopth
Addaiee Dapth
Fadatre Death
CoRnet-S 5
COR net-
CoRnet
Mexket (Ecoset)
Inception (Ecoset)
ResNet- (Eooset)
50-
20-
10
2.5
10
0.5
16
05
10
1.6
6.5
3.2
Reotive Zepth
Addaiee Doghh
Febiliv Desth
NeutieDDott
Ridaive Degth
Pebadive Peath
EgoVt.P (CL5)
EgoVLP
(Eellec2256)
(Iraggeleet-133
(18jects365)
51-
50-
50-
9
32
20-
20
22-
20
25 Aeadiv Jepeh
300
1.
65 Rotadhe Death
Addadin Dapt
Kotadie Death
Aalasie Depet
Retadian Depth
Figure All: Effective dimensionality changes along the hierarchy for all tested ANNs with global average pooled features. Relative depth indicates the normalized index of the network layer. The error bars represent the standard deviation of the dimensionality calculated for 100 random subsets of 50 globally pooled features per layer. The models are ordered by: generic convolutional neural networks, VIT', self-supervised models, biologically-inspire models, models with more human-liks like training data, and unsupervised autoencoders.
28

---

Incostion x3
Reshez 34
ReSMet 50
ReSNNC
25-
25.
22-
15
2.5
3.0
0.5
1.
1.0
0.5
1.
6.5
3.2
Reotice 2epth
Intariinn Depth
Kobatie Deyth
AeptiieDDpe
Intatiin Degth
Robadin Seath
Resket-152
VGG16
Swin-B
Swi--S
MTL/16
VITB/16 (GAP)
2.5
16
65
Restie Deptn
Addcion Dapth
Ealadre Detth
Seatie Dopeh
Addaies Dapti
Enlatre Douth
(CL5)
(Adieerentee)
BenBlet-DD
Raadiv Jeptt
Ratacion Daph
Koladho Seath
Aalatic Depeh
Adariis Depth
Robadee Death
SimCLA ResMet
ReaNet
CUP B/15 (GAP)
(GAP)
B/15 (CLS)
VIT (CLS)
data
2
Teutiv
Addeive Dapth
Fedddbre Death
Seabive Capth
nelazied Dapth
Tadathr Desth
CORnet-S
CoR
CoRnet
Weeket (Ecoset)
Inceptionn v3 (Ecoset)
Restiet- (Eoosed)
33
35-
+2.33
1- 40.30 1.0
p-oa 22 sour
25-
22-
3.5
2.0
0.5
055 Rokidhe
VEGIS (Booser)
Egoot. (CL5)
EgoVLP (GA15
AE VCGL6 [Caliee (256))
AL VECI6 (1mageeNeet-11)
At VDGI1 (Dojects:05)
purt 45.0
25
dw +0.00
25-
15
15
22-
10
22
35
1.0
035
1.0
05
3.0
1.0
055 1.0
65
3.5
Depeh
Figure A12: Intrinsic dimensionality changes along the hierarchy for all tested ANNs with global average pooled features. Relative depth indicates the normalized index of the network layer. The error bars represent the standard deviation of the dimensionality calculated for 100 random subsets of 50 globally pooled features per layer. The models are ordered as in Fig. A11
29

---

Alcxhet
Incoption
ResNer 18
Re2NoC 34
ReSNEE 50
ReSPNcE LOL
6.3-
GA-
3A-
02
0.00 6.75 0.50 077 Tio
0:03
025 0.6 6.7 1.00
C:0 025 653 3.5 1.0
3.00 025 s 035 Lio
0.c0 6.75 0.56 6.75 150
00s 025 6.9 0.75 lice
Telatrr Dash
TalAbi Dejh)
leetie Deper
Edaaied Dest
Ikluced Dupch
Red
VGG16
SwinB
Swin-5
VTTV11 [CLS)
VIFR/16 (GAP)
02
32
0.0
6.5 0.50 1.0
3.
022 025 oác 8.5 1:00
0.5
1.00 025 050 0.5 120
3.00 025
0.35 1.0
6.00 0.25
6.7 1:0
125 0.50 0.75 Liee
Rebaiten Death
Rebiive bighh
de Depin
Iedittnn Desth
Wetorie Dayth
telidee Pepth
and G.e.
30
alc kelabie bipth
0.7
o.5
6.75
Lic
Ralatie Bacth
seanve Deper
Ielation best
mebacis Depth
feladi Septh
Rush 50
Ro
VIT 8/16 (GAP)
(GA))
[CLS)
(CLS) Jactort CA
3.0
cen E.
eshgara 10.t 0nd
and 2e00
0:0
ase
5.00 052
025 071
0:25 6.7 1
125 0.2 Liee
kebatie
CoRnet:5
CoRnet-Z
CORnet-RT
Nlecket (Eooset)
Inception v3 (Ecoset)
Restiet (Ecosed)
3.0
dal
-aals eaar
1.0
2-02007 -227
0.5- KTro
3.
0.5 Te-de
0.3-pah
28-
2.
32
6.6% 8.50 1he
6:1 525
aic 6.75 1.0
5/0 625 653 a
9.0 025 5: 6:5 Lo0
6.c0 0.25 0.56
6.7 180
1 6.5 Liee
VGGI6 (Ecooct)
EgOVUFF (CLS)
EgoVEP (GAF)
AE VGGL6 (C5))
AE VSGL6 (mmggeeet 1K0
VEVG (ON)edt:365)
(4 2 4.-
ynd Nur axte
pn (o-sarr--000 -220 .133
38.
08.
6. Low
38.
0.6-
6.4-
3.6
26
24
D4
24
6.2
0.00 635 050 075 100
0.53 025 05c
6.75 1.00
500 025
653
375 1.26
0.00
ads 800 635 1.00
0.00
6.55
0.56
6.75 160
600 125
0.50 0.75
Life
Telatee Dashh
TelAbvi Dogth
teative Depty
Desth
Dagih
Figure A13: Category decoding performance changes along the hierarchy for all tested ANNs with global average pooled features. Relative depth indicates the normalized index of the network layer. The error bars represent the standard deviation of the performance caleulated for 100 random subsets of 50 globally pooled features per layer. The models are ordered as in Fig. AIl
30

---

25 vertices
50 vertices
100 vertices
Figure A14: Representation of the size of the searchlight on the cortical flatmap for Subject 1 from the THINGS fMRI dataset: 25, 50, and 100 vertices.
Subject S1
25 vertices
50 vertices
100 vertices
10
10
Devea
15
15
15
Figure A15: Lower dimensionality with smaller neighborhood sizes. For the neighborhood sizes shown in Fig. A14] we again compute the dimensionality across the whole visual cortex, and observe similar patterns reported in the main text.
31

---

Vertices
Left Hemisphere
Right Hemisphere
25
4.97 0.05 mm
4.96 0.05 mm
50
7.06 + 0.09 mm
7.04 - 0.08 mm
100
10.01 = 0.12 mm
9.97 + 0.12 mm
Table A7: Average size (in mm) and standard deviation of the searchlight area using 25, 50, and 100 vertices.
A6.2 Different cortical depth estimates
As the functional form of dimensionality increase can be affected by the type of cortical measure and order of the regions we additionally i) compute the Euclidean distance on the cortical surface and, ii) convert the geodesic distance into ranks instead of exact measures. We find Euclidean distance to be highly correlated with geodesic distance (Fig, [AT left). Similar to the results reported in the main text (Fig. 2). While the geodesic rank reveals a more linear increase ED, ID becomes highly variable with unclear structure (Fig. AT6b).
Geodesic distance (mm)
S1
Geodesic distance (mm)
S2
S3
100
100
100
Gecdesic distance
50-
50-
50-
0
20
40
20
40
0
10
20
30
40
Euclidean distance (mm)
Euclidean distance (mm)
Euclidean distance (mm)
b
Effective Dimensionaliyy
9)
Effective Dimensionalitt
6
H IM
Effective Dimensionallyy
6
4-
4
Ieft
Hn
left
3-
0.0
7.6
2.5
7.5
0.0
2.6
7.5
10.0
Rank
Rank
Rank
Effective Dimensionally,
Effective Dimensionality
Effective Dimensionaliy
mug D244
12-
He]
12
11
bib
Hnj
lift
11
left
left
right
right
10
10
10
7.5
10.0
Rank
Rank
Rank
Comparing different measures of cortical distance. a) Left: High similarity between geodesic and Euclidean distance to compute cortical distance. b) Using the rank of geodesic distances instead of the exact distances also shows an increase along the hierarchy.
Figure A16:
A6.3 Random sampling in fMRI
In the main text, we report results using searchlight approach to compute dimensionality. Both, ED, ID, and decoding accuracy are generally higher in ANNs than in fMRI data using the searchlight approach, partly due to spatial correlations fMRI vertices and noise reducing decoding accuracy. To driven by the correlative structure
test whether the differences in dimensionality are affected
32

---

of the fMRI signal in nearby regions, repeated the dimensionality computation sampling randomly vertex activity within an area (primary, early, ventral, lateral, dorsal). We find that even with this random sampling approach the ventral areas have higher dimensionality than all other areas (Fig. A17). confirming our results with the searchlight approach and suggesting that the dimensionality only slightly affected by the spatial correlative structure of the fMRI signal. Note, however, that random subsampling of vertices increases brain ED values, making them more comparable on an absolute scale to ANN results.
Subject S1
Left Hemisphere
Right Hemisphere
Both Hemispheres
Effecive Dimerstoralt
Lifectio Dimerstonnat
50-
20-
Eftectne
Varinal
Danad
Loaand
Exety
Viettal
Donal
Latand
Prinary
Earty
Vintral
Donal
Lafard
12
Intinaio Dimendoonalh
40-
86-
Infinntt Comernatonaai
40-
30
Primary
Figure A17: Calculation of dimensionality sampling random vertices of the different areas instead of using neighbors of vertices. Top: effective dimensionality, bottom: intrinsic dimen- sionality.
A7
Robustness analyses in ANNs
A7.1 Random sampling and random projections in ANNs
How features in an ANN are preprocessed when studying brain alignment can have a significant impact on the results [4 In dimensionality analyses, is important restrict the number of features across layers or regions to ensure valid comparisons. A common approach is to use random projections, which preserve the pairwise distances in the data while reducing dimensionality. While this method retains overall geometric properties, it also mixes unit activations through arbitrary linear combinations, potentially distorting unit-level selectivity, which may be critical for interpreting brain-model correspondence [1]
Previous work has linked individual ANN units to single neurons in the brain 78). supporting unit- level comparisons. However, the fMRI signal reflects a population-Ievel average across thousands of neurons within a voxel, suggesting mismatch in spatial scale. To address this, adopt global average pooling, which compresses feature maps into coarse-grained, image-level representations that potentially better approximate the aggregate nature of fMRI signals. This approach preserves semantic content while discarding spatial specificity, similarly to the spatial integration performed by millimeter-scale cortical sampling IMRI. To evaluate how preprocessing affects dimensionality estimates, we computed both effective and intrinsic dimensionality using three approaches: (i) global average pooling with feature map subsampling, (ii) randomly sampled units that preserve selectivities (unit-level features), and (iii) random projections that preserve geometric structure while reducing feature dimensionality for three example models (ResNet50, SimCLR, CLIP). To compare the methods, we subsample images from the dataset to reduce computational costs. We sampled 500 images from the THINGS dataset, selecting only those observed by the subjects, and repeated each analysis 100 times per layer. For subsampling, we selected 50 pooled feature maps or units, while for random projections, we reduced the dimensionality to 50. Note that due the smaller number of images used, the curves obtained here for global average pooling differ from those in Figure 2
33

---

For effective dimensionality, we find that only global average pooling produces the increase across layers that mirrors the pattern observed in the ventral visual stream of the brain (Fig. A18). In contrast, unit sampling and random projections both yield the previously reported oypansion-comprestie trend, even for effective dimensionality. For intrinsic dimensionality, this axpansio-coopressoe pattern persists across all preprocessing methods (Fig. (T9), suggesting that nonlinear representa- tional geometry in ANNs is shaped more by task constraints and training objectives than by feature selection alone.
ResNet-50
SimCLR (ResNet-50)
CLIP (ResNet-50)
(flectiee Diventoraayy
Efeedte Dmmenterary
Efodbre Omeenderat ora
20-
Global Average Pooling
-3
12
13
0.0
04 0.5
1.0
3.0
0.2
0.5
0.2
0.4
0.6
1.0
Relative Dooth
Relative
Relativ
Cffecive Cimerimaatty
Efeecive Cimensznnlly
ETessive Cimensennlityy
Unit-Level Features
13
20
16-
0.5
08
0.4
0.6
Eftective Damenstonsity
Efectre Cimeneznnlly
Efeetive timmerttnaltty
42.6-
40-
Random Projections
35.0-
0.8
0.4
Figure A 18: Effective dimensionality in ANNs using different strategies to estimate dimensional- ity. Note that differences other reported results in Fig. All stem from a subsampling of only 500 images.
A7.2 Identifying brain-mode correspondences to contextualize dimensionality trends
Previous work hypothesized ANNs to be models of the ventral visual stream showing that ANNs share many similarities with biological brains, including hierarchical processing [e.g.. [ 78). To test how dimensionality changes along the processing hierarchy in ANNs compared to the brain, we first performed two complementary analyses to identify which ANN layers align best with which cortical areas. First, used Scain (63 24!
[2 to identify the layers that best correspond to V1, V2, V4, and IT in macaques (Fig. A20 top). We find that the decreasing trend in ID is already present in layers that best align with IT cortex (Fig. A20 bottom, red arrow indicates best IT-layer). Second, we leveraged our human fMRI data from the THINGS dataset to train linear encoding models to predict [MRI vertex responses from ANN features, going beyond the four BrainScore regions (Fig. A20p). Both approaches yielded similar results: early visual areas (e.g.. V1-V4) are best predicted by early to mid layers, while higher ventral regions are best predicted by deeper layers. While these results were expected, the mapping between ANN depth and brain hierarchy allows us to make informed comparisons of dimensionality profiles across the two systems. These results further indicate that there is fundamental mismatch in ID even when explicitly aligning the model layers to specific brain regions.
A7.3 Effect of training on ANN dimensionality
To test whether the observed changes in dimensionality along the processing hierarchy of ANNs are artifact of the underlying architecture, we also test untrained models (ResNets) with random initialization. For both, effective and intrinsic dimensionality, we find dimensionality profile that strongly differs from the trained models. We observe flat profile for ED along the hierarchy.
34

---

ResNet-50
SimCLR (ResNet-50)
CLIP (ResNet-50)
Intrmt: Cimeetiimeey
Intente Inte onaly
12
12
Global Average Pooling
12
10
0.
04
0.5
5.0
02
0.6
02
04
06
10
Reletive Depth
Rolative Depth
Reiative Depth
losine Divendionallyy
Intere: Dimenstonally
Inimtee Diwentrrattyy
15-
Unit-Level
36.
10-
Features
0.0
52
04
0.5
1.0
0.0
0.2
04
0.5
0.5
1.0
0.2
0.
0.6
1.0
Relalive Depth
Rolativ Depth
Relalive Depth
Intieees Dimenstrrrily
Intrrtec aimendoooalty
12
23
12
Random Projections
12
12
0.
0.2
DA
D.
DE
50
03
041
06
08
0n 00
02
04
0.
Relative Depth
Relative Depth
Relatve Depth
Figure A19: Intrinsic dimensionality in ANNs using different strategies to estimate dimensional- ity. Note that differences to other reported results in Fig. A12 stem from a subsampling of only 500 images.
However, ID increases in deeper layers when global average pooling is applied. This contrasts with prior studies without global average pooling, which reported a flat ID profile [c.g..[6]], suggesting that pooling can enhance the expressivity of representations even the absence of learned weights by aggregating weakly correlated features across the spatial map.
A8
Robustness analyses of dimensionality computation
A8.1 Effect of data preprocessing on dimensionality
We repeated the dimensionality analyses with different preprocessing strategies: i) with and without z-scoring (Fig. A2Ia), and with and without mean-centering (Fig. A2Tp). Across both IMRI and ANN data, we Found that dimensionality estimates were highly correlated. ID was particularly robust, while ED showed minor variation, likely due to the impact of mean-centering and z-scoring on the covariance matrix. In contrast, ID relies on relative distances that appear less sensitive to such transformations. These robustness checks support the stability of our main conclusions.
A8.2 Effect of parameters on intrinsic dimensionality estimate
The the number of neighbors used to estimate the intrinsic dimensionality an important hyper- parameter that could affect the results. We therefore repeated the analysis with different k-values (k = 5, 10, 20) and found that all results hold when changing the number of neighbors (Fig. A23).
A Low-level feature representations
A9.1 Extracting low-level feature representations in images
We performed PCA on the THINGS images viewed by subjects during experiments to extract low-level visual features 73 68
Ordering the images by their loading on the first PCs reveals contrast, spatial frequency and color components (Fig. A24]. The first four components already explain 43.67% of the total variance. highlighting the low-dimensionality of the input data. Due to
35

---

a
Best Layer Alignment to Brain Areas
0.5
0.5-
0.3
02-
0.0
0.2
0.6 Relative Depth
b
ResNel-50
SimCUR ReeNet-50
CLP ReaNel-50
16-
16-
16-
0.5
1.0
05 Reladiv Depth
0.5 Relative Depth
Reletiv Depti
S1 ResNet-50
& SimCLR (ResNet-50)
S1 CLIP (ResNer-50)
1740
Ker 174
Far 5040
Relt (N6
100
Figure A20: Alignment between brains and ANNs. a) We use BrainScore to find the layer that best aligns with VI, V2, V4, and IT cortex (top). Additionally we indicate the layer that most closely aligns with IT cortex (bottom, red arrow). b) Additionally, we train linear regression models to map each ANN layers' GAP activations to the fMRI features. Red regions indicate high alignment, while blue shows low alignment performance.
copyright restrictions, we do not display the original images. Instead, we project and show images from the THINGSplus dataset [69].
36

---

THINGS subjects
BOLD5000 subjects
4
derrwared
C912
domeonod
domoamed
53
C513
12
12
derrardd
demcared
Figure A21: Effective dimensionality computations with and without z-scoring, and with and without mean-centering.
subjects
subjects
demeened
10
10
82
CBI
demelned
51
demeuned
RO0ec0oi me
12
12 25corcd
10
25corod
2500n0d
comcancd
2520000
domoanied
demarrdd
5
10
12
15
5
10
12
10
12
14
18
Semcared
demeaned
Figure A22: Intrinsic dimensionality computations with and without z-scoring, and with and without mean-centering.
37

---

S1
000100
16
0000000000
16
pno 00em00
14
14
14-
k=20
k=20
10
10
10
8-
8-
12
14
16
12
14
16
12
14
16
k=5
k=5
k=10
CSI1
16
006100
16
000000>000
16
000000
14
14
14
12
12
12
10
10
10
14
16
16
k=5
k=5
k=10
Figure A23: Different numbers of neighbors for the intrinsic dimensionality computation.
1st Component (24.83%)
2nd Component (7.74%)
3rd Component (6. 14%)
4th Component (4.96%)
Figure A24: Image-level PCA reveals low-level features in the images.
38


Reviews:
[{"review_id": "qU0IRq5PCt", "reviewer": "Reviewer_6Shz", "summary": "This submission investigates how dimensionality measures (particularly effective dimensionality and intrinsic dimensionality) vary across the fMRI responses measured from different visual regions. They compare the observed dimensionality from the fMRI responses with the dimensionality measured from the hierarchical layers of ANNs.  The paper shows that overall the dimensionality increases for deeper visual cortical regions, and features of ANNs tend to show a trend where the dimensionality begins to decrease for late layers. When using spatially averaged features, the authors observe that the trends in ANN dimensionality across layers begin to look more like the visual regions, however the ID still decreases for some late layers, suggesting a mismatch between the hierarchy of the brain and the tested ANNs.", "questions": "1. Is there any normalization (for instance, z-scoring) applied to the features of the neural networks or the measured fMRI activations before the dimensionality measure is computed? This preprocessing should be reported. Additionally, I would like to see the replication for Figure 1 (1) with and without z-scoring the features/fMRI responses across stimuli before dimensionality is computed. (2) with and without the per-feature/voxel mean subtracted across stimuli (but not normalizing the variance as you do in z-scoring).\nI am asking about this specifically because, for the fMRI data in particular, things like the overall amplitude of the response between different voxels can dramatically change the effective dimensionality, and this type of response variability can be dramatically different from one region to another due to the SNR of the fMRI response. It thus seems particularly important to make sure that the main trends of the paper are robust to the preprocessing strategy used. \n\n2. Is there a citation for referring to the effective dimensionality / participation ratio as the “linear dimensionality”, or is this something being used by the authors? I have not encountered it being referred to as such before. \n\n3. The sampling for the fMRI data is performed in a searchlight approach where neighboring voxels are included in the dimensionality, while for the ANN data it is sampled by randomly choosing 50 pooled feature maps. This potentially preserves topographical organization in the fMRI data. The authors present a control experiment for this in A10 and show that the overall pattern of responses is conserved, however it looks like the absolute values of the dimensionality are much higher for the random sampling.  Could the authors explain this substantial difference? \n\n4. Are the super-linear fits significantly better than linear fits? I am surprised that the authors are highlighting that the increase is super-linear because from looking at it, it seems to be that the V1 point (at zero) is driving this “super linear” effect. The fits with 3 points in the supplement A11 and A12 seem particularly concerning. \n \n5. The authors talk about the importance of global-average pooling for the dimensionality alignment (201-213), where the last sentence is about them serving as a better proxy for “neural recordings”, but I think that the authors are specifically talking about alignment with fMRI data where nearby neurons are pooled? This should be clarified, and a discussion about how there might be differences when looking at individual neural responses rather than bold activity should be included somewhere in the paper. \n\n6. The specific layers used for the deep neural network analyses and how this is mapped to the \"relative depth\" 0-1 scale should be more clearly explained (I couldn't find this, but maybe it is in a section of the supplement). \n\n7. Given that the authors main interpretation of the results is that higher-order regions remain higher dimensional because they allow for behavioral generalization, it seems appropriate to discuss work on data diet and training types that encourage the deep neural networks to maintain additional information. Some work that has demonstrated changes in the training environment/task to preserve additional information and lead to better neural alignment are: \n* https://openreview.net/pdf?id=AiMs8GPP5q (adds an equivariant loss and shows that this improves predictions for IT response). \n* https://www.nature.com/articles/s41467-024-53147-y (shows that diversity in the visual training diet is one of the main drivers of good predictivity)\n* https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3002366 (shows that training on multiple tasks results in the best overall predictions for audio fMRI data)   \n\n8. Please include references to the numbered supplemental figures / sections when they are referenced in the main text.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n* Investigating the local dimensionality of different cortical regions on a large-scale image dataset is interesting, and seems novel to report. \n* The comparisons with ANNs seem to be well thought out, and there are some critical control experiments presented in the supplement. \n* The paper is well structured, and the writing and logic are clear to follow\n\n**Weaknesses**\n\n* The claim about super-linearity in many places of the paper seems exaggerated, as it is driven primarily by one point (primary visual cortex) and might be dependent on the way that the “cortical distance” was determined.  \n\n* The authors claim that the ID decreases for late layers which diverges from the brain, however the actual depth of the models seems somewhat arbitrary not directly aligned with the neural responses.  It is very possible that the better comparison with these visual brain regions would be only the early/mid layers of the models (where there is a steady increase in dimensionality), and higher-order brain regions not included in the analysis would actually drop in dimensionality if they were included in the fMRI analysis. \n\n* Dimensionality measures can sometimes be dramatically dependent on the data processing, and this is often particularly true for noisy data like fMRI (for instance normalization of signals, or the noise in the data). There are some specific questions about this below.", "was_revised": true, "final_justification": "This paper performs a clear and well-thought-out set of analyses to examine the dimensionality of brains and computational models. The experiments are solid, and the results seem important for those who compare models with neural data (especially fMRI).  \n\nDuring the rebuttal, the authors addressed my concerns by stating they will make changes to the paper about the following: \n- Talk about the direction of dimensionality changes rather than emphasize super-linearity\n- Clarify the methods (e.g. layer depth normalization, linear dimensionality) \n- Report additional control analysis (data preprocessing, brain region/model layer alignment)\n- Add additional comments/discussion (searchlight vs. random sampling, fMRI vs. single units, additional references).", "rebuttal": "Thank you for your feedback. We appreciate the opportunity to improve our manuscript based on your observations. Please let us know if there is further clarification we can provide.\n\n**1. Clarifying the use of “super-linearity” in dimensionality trends and potential confounding effects of the cortical distance metric**\n\nThank you for this thoughtful comment, which was also raised by other reviewers, and appreciate the opportunity to clarify this point. We agree that in some cases, the observed super-linear increase in dimensionality seems to be influenced by individual data points, such as the primary visual cortex. We therefore repeated these analyses with a more fine-grained partition of visual areas and in all cases but one found that a super-linear function better fits this increase than a linear function. Additionally, we would like to point out that we tested multiple methods for computing cortical distance, including geodesic and Euclidean distance on the cortical surface as well as Euclidean distance in volume space. Across all of these distance metrics, the dimensionality increase in the human cortex was consistently better fit by a super-linear function than a linear one. That said, we would like to clarify that the precise shape of the increase (i.e., whether linear or super-linear) is not the main claim we intend to emphasize. Our core observation is that both effective dimensionality (ED) and intrinsic dimensionality (ID) increase across the human visual hierarchy, whereas in ANNs, dimensionality decreases in the later layers. To reflect this more clearly, we will revise the manuscript text to **de-emphasize the notion of \"super-linearity\"** and instead focus on the broader, more robust contrast in the **direction of dimensionality** change between biological and artificial systems. We will incorporate these clarifications and additional analyses into the revised manuscript.\n\n**2. Alignment between brain regions and model layers**\n\nThank you for raising this important point about the alignment between model depth and brain hierarchy. Similar concerns were also raised by other reviewers, and we have implemented three different analyses to directly address this point.\nFirst, we tested if the drop in dimensionality persists if we reduce our results to the layers that were identified by BrainScore [1] to best match V1, V2, V4, and IT. We found that both ED and ID increase up to the layers corresponding to V4, but ID notably drops in the IT layers, supporting our claim of a mismatch in representational geometry. However, as **BrainScore** relies on unit activations (without GAP) and is focused on macaque data, we additionally conducted two alignment analyses using the THINGS fMRI data.\nFirst, we trained an **encoding model to predict the vertex responses from the model’s activations** [1-3]. This analysis revealed that V1 is best predicted by early to intermediate layers, early processing regions (V2-V4) by intermediate layers (around peak ID), and higher ventral regions by late layers where ID has already declined, confirming that the drop in ID occurs in layers most similar to high-level brain areas.\nSecond, to remove noise from the data we projected both fMRI activity and ANN activations onto their respective first principal components and computed the correlation. Again, we found that early and intermediate model layers align best with primary and early visual areas, and later layers with higher areas, further validating our results.\nOverall, these results suggest that the observed divergence between brains and models cannot be explained by misaligned depth and supports our conclusion that current ANNs may lack key computational properties required to maintain high-dimensional, abstract representations observed in higher-order visual cortex. We will add these findings in the revised manuscript to contextualize the models’ ID drop with respect to model–brain alignment.\n\n**3. Data preprocessing**\n\nWe agree that the data preprocessing is an important aspect that might affect dimensionality. To address this, we repeated the dimensionality analyses with different preprocessing strategies: i) with and without z-scoring, and ii) with and without mean-centering. Across both fMRI and ANN data, we found that dimensionality estimates were highly correlated. ID was particularly robust, while ED showed minor variation, likely due to the impact of mean-centering and z-scoring on the covariance matrix. In contrast, ID relies on relative distances that appear less sensitive to such transformations. These robustness checks support the stability of our main conclusions, and we will include them in the revised supplement. \n\n**4. Referring to the participation ratio as linear dimensionality**\n\nWe use the term linear dimensionality to describe ED, as it captures the effective number of linear dimensions in the data based on the eigenvalue spectrum of the covariance matrix. While it is not specifically common to refer to ED as linear dimensionality most studies focus on a single measure of dimensionality. To highlight the differences between ED and ID, we chose to include the terms linear and nonlinear dimensionality. We are happy to make this more clear in a revised version of the text.\n\n**5. Differences in dimensionality values between the searchlight approach and random sampling**\n\nThank you for pointing out the differences in scale between the dimensionality values using the searchlight and the random sampling approach. Indeed, the searchlight approach in the fMRI data includes spatially contiguous vertices, which likely preserves local topographical organization, such as retinotopic mapping, that may contribute to lower observed dimensionality values in early visual areas. In contrast, the random sampling of vertices does not preserve spatial proximity and may yield higher dimensionality. We agree that this difference in sampling could contribute to the differences in absolute dimensionality values observed in Figure A10. However, our control analysis was designed to test the robustness of the relative **dimensionality trends rather than to compare the absolute values**. Importantly, although local correlation likely plays a role, the consistent increase in dimensionality across the hierarchy remains even when subsampling is applied, indicating that there are additional organizational principles beyond spatial locality driving these effects. While our primary focus will remain on the relative trends across layers and regions, which are robust across both sampling approaches, we will explicitly mention the differences in absolute values, and state that they are, to some extent, expected due to differences in spatial organization and sampling strategy. \n\n**6. Layer depth**\n\nThe definition of layer depth was indeed missing from the main text and was also raised by other reviewers. In short, we followed previous literature to compute the relative depth [4,5]: First, for models with a ResNet 50 backbone, we extracted activations from all ReLU layers (GeLu activations for transformer based models), we then normalize the position of these layers in the processing hierarchy by the number of all processing steps in the model. We will add this explanation to the Methods section to improve clarity and also highlight the extracted layers by including a list of layers in the supplement.\n\n**7. Miscellaneous**\n\nThank you for several comments regarding clarity. Indeed, our statement in lines 201–213 refers specifically to the **alignment of GAP model features with fMRI data**, and not with single-neuron recordings. We will revise the relevant paragraph to clarify this. Furthermore, we appreciate the suggestion to include a broader discussion of this distinction. We will add a note to the Discussion section acknowledging that while GAP features may align well with fMRI data, they may not be suitable proxies for electrophysiological recordings of individual neurons that may indeed best align with the direct unit activation in the models. This distinction will become increasingly relevant as advances in high-resolution fMRI methods (e.g., 7T imaging, improved coils) enable more precise measurement of neural activity across spatial and temporal dimensions, potentially enabling more direct comparisons with unit-level model activations.\n\nThank you for the recommended **literature**, and we will include this in a brief discussion. Following a similar suggestion from another reviewer, we extended our analysis to include models trained with visual diets that better match human experiences [6,7] and indeed observed a continuous increase in the model’s ID also in the later layers. We will report these findings in the revised manuscript.\n\nFinally, we are happy to include the exact **references to the supplemental figures** and section in the main text to further improve the clarity.\n\n**References**\n\n[1] Schrimpf et al., 2018. Brain-score: Which artificial neural network for object recognition is most brain-like?\n\n[2] Wang et al., 2023. Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset. \n\n[3] Naselaris et al., 2011. Encoding and decoding in fMRI.\n\n[4] Alessio et al., 2019. Intrinsic dimension of data representations in deep neural networks. [5] Sorscher et al., 2022. Neural representational geometry underlies few-shot concept learning\n\n[6] Mehrer et al., 2021. An ecologically motivated image dataset for deep learning yields better models of human vision. \n\n[7] Lin et al., 2022, Egocentric Video-Language Pretraining."}, {"review_id": "8I2GNKBc6Z", "reviewer": "Reviewer_TFdE", "summary": "The authors probe the representational geometry of the human visual cortex and convolutional neural nets (CNNs) by measuring their effective dimensionality (ED) and intrinsic dimensionality (ID). ED measures the linear expressivity of the neural code, while ID measures the number of independent parameters needed to describe the data locally. In both human fMRI data and CNNs with global-average pooling (GAP), ED grows monotonically in deeper layers. ID grows monotonically along the human visual hierarchy but shows a hunchback profile for CNNs with GAP, indicating a mismatch. The higher ID observed along the human cortex is interpreted as higher semantic abstraction, whereas the lower ID in CNNs indicates that the final layers of deep networks lose such information. Supporting this claim, the authors show that classification tasks can be solved better with deeper layers both in the human cortex and CNNs, while abstract concepts are decoded inceasingly well only in the human data.", "questions": "1. How does high ID relate to disentanglement/abstraction in the sense used in ref. \\[34]? Also, how does the ID defined here compare to SD in ref. \\[34]?\n2. How does changing $k$ affect the ID? Does setting $k = 20$ keep each term in Eq. 2 “local” when there are only 50 points in each searchlight sphere?\n3. What is the importance of super-linearity, and how does it differ from mere linearity? How does this relation change if the distances are the ranks rather than geodesic distance?\n4. Would you consider repeating the experiments for biologically inspired models like cornets?", "limitations": "The main claim holds only for CNNs with specific pooling, thus limiting the applicability of the proposed benchmark to other architectures such as ViTs.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\nThe paper was very fun to read; it is clearly written and easy to follow. Experimental methods are clearly defined with sufficient detail. Representational alignment between ANNs and brains is a hot topic. The proper metrics to measure representational geometry and interpretability are a huge debate in the NeuroAI community, and this work makes a solid contribution by providing a carefully designed and interpretable benchmark to distinguish ANNs from brains. The experiments are thorough and well-thought, e.g. testing dependence on pooling strategy, controlled experiments with randomly initialized networks, two different datasets (THINGS, BOLD), cross-validated analysis, etc.\n\n## Weaknesses\n\n1. I am not familiar with the searchlight approach. I could not completely understand the data pipeline and how exactly ED and ID are calculated. A short description for searchlight would be helpful for general audiance. \n2.  In addition to GAP, ANN layer features are randomly pooled to 50 (out of ~512) features to compute dimensionality, but this assumes the features are uniformly distributed in a layer.\n3. For ED/ID and accuracy plots, the y-axes have completely different scales between brain and ANNs. For ED, ANNs are significantly higher-dimensional and hard to compare with human data. For decoding, ANNs perform an order of magnitude better than fMRI data. The paper should at least comment on this. Also, since the paper does not compare dimensionalities between brains and ANNs, there is no need to restrict ANN dimensionality.\n4. The low ID in deeper layers is explained by task specificity of ANNs. Shouldn't SimCLR have larger ID since it is self-supervised? It is not clear how the training objective affects ANN–brain alignment. Including an unsupervised model might help test the paper’s main claim.\n5. It is not clear how high *non-linear* dimensionality relates to better *linear* decodability of abstract features. The authors cite ref. \\[16] (line 41) for this relation, but ref. \\[16] only considers linear dimensionality. Even then, the regression score cannot be explained solely by dimensionality; alignment with the task also matters ([https://arxiv.org/abs/2309.12821](https://arxiv.org/abs/2309.12821)). The authors should clarify this point.\n6. Some reported values—$R^2$ in Fig. 3 and accuracy in Fig. 4—are negative and should be fixed.", "was_revised": true, "final_justification": "This is an empirical paper that reports an interesting phenomenon: artificial neural networks seem to differ from biological networks in their non-linear dimensionality but not in their effective dimensionality. It is clearly written, and the experiments are thorough.", "rebuttal": "Thank you for your feedback, and we are pleased to hear that you enjoyed reading the paper. Please let us know if there is further clarification we can provide.\n\n**1. Searchlight approach**\n\nWe agree that a more detailed explanation of the searchlight approach [1] will be helpful and we will include this in the revised version.\n\n**2. Distribution of features within a layer**\n\nIn our analysis, we compute dimensionality based only on the feature maps (i.e., channels) after global average pooling, and not on spatially structured representations. In principle, this means that each feature map reflects a distinct, non-spatial channel, and we expect minimal correlation due to spatial proximity. Random subsampling is therefore applied to a set of relatively independent features and does not rely on any assumptions about uniform spatial distribution. We will revise the text to better convey this. \n\n**3. Differences in scale between brain and ANN dimensionality**\n\nED, ID, and decoding accuracy are generally higher in ANNs than in brain data, partly due to spatial correlations in fMRI vertices and noise reducing decoding accuracy. Random subsampling of vertices increases brain ED values, making them more comparable to ANN results (Fig. A10). We believe subsampling ANN features remains important, as it normalizes scale differences and enables consistent comparisons of dimensionality evolution across layers. However our main claim concerns the pattern of dimensionality change across the visual hierarchy in the brain and ANNs, rather than the precise values. We will revise the manuscript to clarify these points.\n\n**4. Different ANNs with diverse training paradigms**\n\nThank you for the suggestions to expand our analyses to other architectures and training objectives. In response, also to other reviewers, we extended our evaluation to include:\n- **Unsupervised models (Autoencoders [2]):** ID increases in the encoder but drops in the decoder, while ED remains relatively stable across all layers, revealing a different type of mismatch to the brain.\n- **Biologically inspired models (CORNets [3])**. All models show an increase in ED. For ID, CORNet-Z exhibits a continuous increase, while the recurrent models show the typical drop in late layers.\n- **Vision Transformers (ViTs, SWIN-ViT, CLIP-ViT [4-6]):** While CLIP-ViT shows the dimensionality pattern as observed in CNNs, ViTs and SWIN-ViTs show an increase-decrease pattern in ID and ED.\n- **Models trained on human-like visual diets (Ecoset, EgoVLP [7,8]):** Both models exhibit a continuous increase in ID and ED throughout the hierarchy, more closely matching trends observed in the brain.\n- **Other model architectures (AlexNet, Inception, VGG16, ResNets):** All models show a similar dimensionality behavior as ResNet50.\n\nThanks to these additional analyses we were able to strengthen our claim by showing that the observed mismatch in ID between human visual cortex and ANNs holds across many training paradigms and architectures. Moreover, we now show that other aspects, like biological alignment or visual diet can improve the model-brain correspondence. We will include these new insights in the revised text.\n\n**5. Dimensionality reduction in SimCLR**\n\nWe agree that SimCLR’s self-supervised objective might intuitively suggest higher ID in deeper layers, as it does not compress representations into class labels. However, prior work [e.g., 9] shows that contrastive models often exhibit dimensionality collapse, with highly correlated features spanning a lower-dimensional subspace. Hence, self-supervised training does not guarantee high ID. Consistent with this, both SimCLR and CLIP show the same increasing-decreasing ID pattern as classification trained models. We will clarify this point in the revised Discussion. \n\n**6. Relation between nonlinear dimensionality and linear decodability**\n\nThank you for raising the point about the relation between nonlinear dimensionality and linear decodability. Nonlinear dimensionality reflects the intrinsic degrees of freedom in a representation, while linear decodability measures how well specific features can be extracted with a linear readout. Note that prior work has studied the capacity of representational manifolds also through the ability to linearly decode (pseudo-)classes [10, 11]. In our results, a drop in nonlinear dimensionality in deeper ANN layers correlates with reduced linear decodability of abstract features, suggesting that representations become increasingly compressed along task-relevant axes, limiting access to non-task-related features. This aligns with prior work linking dimensional increase to better generalization performance [e.g., 12]. We will clarify this connection and cite additional literature in the revised manuscript.\n\n**7. Negative $R^2$ values**\n\nWe will report the $R$-values.\n\n**8. Relation between ID and shattering dimensionality**\n\nWe appreciate the opportunity to clarify that while both ID and the shattering dimensionality [SD, 10] relate to complexity, they capture different aspects. ID (here estimated via MLE [13]) is a data-driven measure that relies on relative distances between data points and reflects the number of degrees of freedom in the data manifold, providing insight into the local geometric structure of the representations. In contrast, SD measures the ability of a model to separate data under all possible labelings and  “quantifies how many random classifications of a given set of points a linear readout can solve” [10]. While this can be computationally demanding, ID relies on the one-time computation of distances between close neighbors. We will clarify this distinction in the revised manuscript.\n\n**9. Effect of $k$ on the ID**\n\nThank you for pointing out that the $k$-value, i.e., the number of neighbors used in the ID computation is an important hyperparameter that could have an effect on the results. We therefore repeated the analysis with different $k$-values ($k$=5,10, 20) and found that all results hold when changing the number of neighbors. We will include these additional analyses in the supplement.\n\n**10. Importance of super-linearity**\n\nThank you for raising this important conceptual question. The observation of super-linear increases in both ED and ID dimensionality across the human ventral visual hierarchy initially served as a descriptive characterization of how representational complexity changes along the cortical processing stream. Super-linearity in this context might suggest a nonlinear growth in representational capacity or abstraction across stages of the hierarchy, to **potentially support a rich, abstract feature space for flexible behavior**. However, these interpretations remain speculative and are beyond the empirical scope of the current paper. While we originally included this observation to highlight the rapid increase in dimensionality in the brain and the profound difference to ANNs, our core finding is not the exact functional form of the dimensionality increase, but rather the qualitative divergence between brains and models. In light of several reviewer comments regarding also an overemphasis of this finding, **we decided to de-emphasize the importance of the precise shape of the increase** in the revised manuscript and we will adjust both the framing and language to focus on the divergence in representational geometry between brains and ANNs.\n\n**11. Different measures of cortical distance**\n\nWe agree that the functional form of dimensionality increase can be affected by the type of cortical measure and order of the regions and we appreciate the suggestion of using the rank instead of the geodesic distance. We implement this suggestion by i) converting the geodesic distance into ranks instead of exact measures, and ii) using a hierarchical rank along the visual processing hierarchy [14, 15]. While the geodesic rank reveals a more linear increase in ED, ID becomes highly variable with an unclear structure. Using the hierarchical rank, we again observe a continuous increase in ED and ID, further confirming our results. Additionally, we would like to point out that in addition to the cortical geodesic distance we also tested the Euclidean distance on the cortical surface as well as in volume space. Across all of these distance metrics, we found a super-linear dimensionality increase along the ventral visual hierarchy. We will add the new results in the revised version of the manuscript.\n\n**References**\n\n[1] Chen et al., 2011. Cortical surface-based searchlight decoding.\n\n[2] Github: Horizon2333, imagenet-autoencoder.\n\n[3] Kubilius et al., 2018. Cornet: Modeling the neural mechanisms of core object recognition.\n\n[4] Dosovitskiy et al., 2020. An image is worth 16x16 words: Transformers for image recognition at scale.\n\n[5] Liu et al., 2021. Swin transformer: Hierarchical vision transformer using shifted windows.\n\n[6] Radford et al., 2021. Learning transferable visual models from natural language supervision.\n\n[7] Mehrer et al., 2021. An ecologically motivated image dataset for deep learning yields better models of human vision.\n\n[8] Lin et al., 2022, Egocentric Video-Language Pretraining.\n\n[9] Jing et al., 2021. Understanding dimensional collapse in contrastive self-supervised learning.\n\n[10] Posani et al., 2025. Rarely categorical, always high-dimensional: how the neural code changes along the cortical hierarchy.\n\n[11] Chung et al., 2018. Classification and Geometry of General Perceptual Manifolds.\n\n[12] Elmoznino & Bonner, 2024. High-performing neural network models of visual cortex benefit from high latent dimensionality.\n\n[13] Levina & Bickel, 2004. Maximum likelihood estimation of intrinsic dimension.\n\n[14] Felleman & Van Essen, 1991. Distributed hierarchical processing in the primate cerebral cortex.\n\n[15] Rolls et al., 2024. A ventromedial visual cortical ‘Where’stream to the human hippocampus for spatial scenes revealed with magnetoencephalography."}, {"review_id": "hnHHEkJhFJ", "reviewer": "Reviewer_aasq", "summary": "This paper systematically analyzes and compares the linear and nonlinear activations of the human brain and ANN when viewing natural images. They point out that nonlinear dimensionality shows a collapse in later layers of ANNs. They found that this collapse would limit Semantic Generalization.", "questions": "1. Your work says “We find that in the... between brains and models” (Line 63-66). The layers of the human brain,  as hierarchical pathways, such as primary visual cortex -> intermediate visual cortex -> advanced visual cortex, while the layers of the ANN are the network layers of the model. Do these two layers mean the same thing? How do you prove that a certain layer of the model corresponds to a certain visual layer of the human brain?\n\n2. In recent years, several viewpoints have emerged on the interpretability of ANNs. Some people believe that artificial neural networks trained in physical space are similar to the networks of real brains [1]. Others have shown that \"models with higher performance on benchmark tasks achieve peak prediction accuracy in earlier layers. In contrast, models with lower performance show delayed representations and require deeper layers to achieve similar levels of brain prediction accuracy.\"[2]. This result means that the model is gradually becoming more and more like the brain as the level changes. When this similarity reaches its peak, it turns to generate exclusive processing, which exceeds the target range of current neural recordings, resulting in a decline in the encoding model. So, if you want to compare the levels of two \"brains\" in physical space, should the ANN level be truncated at the most brain-like layer (because it can best explain brain response activities)? This seems to be more objective because the deep layer will lose some interpretability due to the conversion of the target. This may get completely different results.\n\n3. You mentioned that \"ANNs exhibit a collapse in intrinsic dimensionality in deeper layers\". The explanation of intrinsic dimension in the article is nonlinear correlation. If the model has a complete basis, it can well linearly represent the nonlinear components in the model's perceived information, then it will naturally get a lower intrinsic dimension because it can be well linearly approximated. What do you think about this issue?\n\n4. You mentioned that in the brain, higher-dimensional representations are associated with stronger abstract generalization abilities. [3] seems to have made a similar conclusion. Is it necessary to discuss it?\n\nI would be willing to improve my score if the above questions can be answered well.\n\n**Reference**\n\n[1] Achterberg, J., Akarca, D., Strouse, D.J. et al. Spatially embedded recurrent neural networks reveal widespread links between structural and functional neuroscience findings. Nat Mach Intell 5, 1369–1381 (2023).\n\n[2] Mischler, G., Li, Y.A., Bickel, S. et al. Contextual feature extraction hierarchies converge in large language models and the brain. Nat Mach Intell 6, 1467–1477 (2024).\n\n[3] Wang, A.Y., Kay, K., Naselaris, T. et al. Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset. Nat Mach Intell 5, 1415–1426 (2023).", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. This paper reveals a functional mismatch in representational geometry between brains and ANNs.\n\n2. This paper demonstrates that global average pooled features from ANNs align best with fMRI-derived linear geometry, which is a significant discovery.\n\n**Weaknesses**\n\n1. The conclusions of this paper are similar to those of some other literature [1], which affects its originality.\n\n2. The explanation of the stratification of the cerebral cortex and the levels of ANN is not clear enough.\n\n[1] Wang, A.Y., Kay, K., Naselaris, T. et al. Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset. Nat Mach Intell 5, 1415–1426 (2023).", "was_revised": true, "final_justification": "The authors provided reasonable explanations for issues commonly raised by the reviewers, such as clarifying the relationship between brain layer stratification and ANNs, and the alignment between brain regions and model layers. Furthermore, they addressed my concerns by truncating the results at the most brain-like layers. The authors also provided reasonable responses, so I am willing to increase my score.", "rebuttal": "Thank you for your feedback and giving us the opportunity to clarify and improve our results. Please let us know if there is further clarification we can provide.\n\n**1. Similarity to previous studies**\n\nWe thank the reviewer for highlighting this relevant study, which demonstrates that the incorporation of language can improve neural predictivity in late-stage visual regions using voxel-wise encoding models. We fully agree that this work conceptually contains some similar ideas to our work. At the same time, our study provides a complementary and distinct contribution in several key aspects. While [1] primarily focus on improving encoding model performance through training objectives, our work shifts the focus toward the representational geometry of both artificial and biological systems. To the best of our knowledge, we are the first to report a continuous effective (linear, ED) and intrinsic (nonlinear, ID) dimensionality increase along the ventral visual hierarchy in humans using large-scale fMRI data, while also comparing these patterns to ANNs. Additionally, we examine how ED and ID dimensionality evolve along the hierarchy in ANNs, and identify a consistent collapse of ID in deeper ANN layers that differs from our observations in the brain. In contrast to [1], we show that additional language information cannot resolve this dimensionality mismatch. Further, we show that deep layers with reduced ID, also show a reduced generalization to abstract, behaviorally rated features (e.g., heaviness). Importantly, these features differ from the object categories used in [1] (e.g., person, vehicle, food) by their level of abstractness and continuous values that were behaviorally determined. This adds a behavioral relevance dimension and places stronger demands on the generalization capacity of the models. Together, while both studies analyze ANN–brain alignment, our work contributes a new conceptual framework, i.e., dimensionality alignment. We agree with the suggestion to more clearly contextualize our work relative to this prior study (also related to your last question) and will revise the manuscript accordingly.\n\n**2. Clarifying the stratification of the cortex and levels of the ANN**\n\nThank you for pointing out the lack of clarity in our explanation, a concern that was also raised by other reviewers. In our analysis, the cerebral cortex was segmented into visual hierarchy regions using the HCP atlas (see Supplement Section A1.3, [2]). We then defined cortical depth as the geodesic distance from the center of each region to the center of the V1 foveal representation, computed along the cortical surface. To ensure robustness, we additionally tested other distance measures, including Euclidean distances in both surface and volume space, and found similar results.\nIn the artificial neural networks we followed previous literature to compute the relative depth [3, 4]: First, we extracted activations from all ReLU layers (see lines 116-117; GeLU layers for transformer-based models), we then normalize the position of these layers in the processing hierarchy by the number of all processing steps (layers) in the model. We will now revise the manuscript to include a more detailed explanation. Additional information regarding newer models such as ViTs will be included in the updated Supplement.\n\n**3. Alignment between brain regions and model layers**\n\nThank you for raising this important point about the alignment between model depth and brain hierarchy. Similar concerns were also raised by other reviewers, and we have implemented three different analyses to directly address this point.\nFirst, we tested if the drop in dimensionality persists if we reduce our results to the layers that were identified by **BrainScore** [5] to best match V1, V2, V4, and IT. We found that both ED and ID increase up to the layers corresponding to V4, but ID notably drops in the IT layers, supporting our claim of a mismatch in representational geometry. However, as BrainScore relies on unit activations (without GAP) and is focused on macaque data, we additionally conducted two alignment analyses using the THINGS fMRI data.\nFirst, we trained an **encoding model to predict the vertex responses from the model’s activations** [1,5,6]. This analysis revealed that V1 is best predicted by early to intermediate layers, early processing regions (V2-V4) by intermediate layers (around peak ID), and higher ventral regions by late layers where ID has already declined, confirming that the drop in ID occurs in layers most similar to high-level brain areas,, like FFC.\nSecond, to remove noise from the data we projected both fMRI activity and ANN activations onto their respective first principal components and computed the correlation. Again, we found that early and intermediate model layers align best with primary and early visual areas, and later layers with higher areas, further validating our results.\nOverall, these results suggest that the observed divergence between brains and models cannot be explained by misaligned depth and supports our conclusion that current ANNs may lack key computational properties required to maintain high-dimensional, abstract representations observed in higher-order visual cortex. We will add these findings in the revised manuscript to contextualize the models’ ID drop with respect to model–brain alignment.\n\n**4. Truncating results at the most brain-like layers**\n\nThis is a valuable insight, and we thank the reviewer for highlighting recent perspectives on ANN–brain interpretability. We agree that aligning processing stages between ANNs and the brain is challenging and also depends strongly on the chosen metric [7]. Truncating results at the most brain-like layer does not qualitatively change our findings. We believe that our study adds a complementary alignment criterion: assessing whether models preserve the brain’s observed increase in representational geometry along the visual hierarchy, providing a perspective beyond voxelwise predictivity. Prior work [8] linked dimensionality reduction to improved generalization, but typically within-class generalization, not the more difficult generalization to abstract features tested here. Hence, while we agree that task-specific compression in models could contribute to the late-stage decline in dimensionality, we believe that this difference is meaningful and worth reporting. In summary, truncating ANNs at the most brain-like layer is a valid approach, but our findings show that even these layers differ structurally from biological systems. Highlighting these discrepancies is an important step toward building models that not only predict neural activity but better capture the brain’s internal organization. We will clarify this in the revised manuscript.\n\n**5. Lower nonlinear dimensionality due to a complete basis**\n\nWe appreciate the comment and respond below based on our current understanding of the concern. In case we have not interpreted it correctly, we welcome further elaboration.\n\nWe agree that the low intrinsic dimensionality observed in later layers could partly reflect a convergence toward a space with a complete basis, where nonlinear structures become more linearly approximable. However, given the task-specific nature of training, particularly in models like ResNet50 trained on ImageNet, we believe that the observed dimensionality collapse rather reflects a true reduction in representational diversity, potentially shaped by the model’s training objective. In the early- to mid-layers of the models, nonlinear dimensionality is behaving largely similarly to linear dimensionality, which suggests that in these layers representations capture both linear and nonlinear variations. However, in later layers the nonlinear dimensionality collapses while linear dimensionality remains high, implying that although the network maintains a set of linearly independent features, these features increasingly fail to span the richer, nonlinear structure that we observe in the human brain. This divergence may reflect a task-driven compression where only task-relevant, linearly decodable features are retained [8], limiting the model's capacity for flexible abstraction. In our work, we observe that the collapse of nonlinear dimensionality in deeper ANN layers coincides with a reduced ability to decode abstract, high-level features. While this does not establish a causal relationship, the parallel between dimensionality compression and reduced generalization suggests a potential constraint in the representational capacity of these layers. If our explanation does not yet fully capture and resolve your concerns, we would appreciate further elaboration on the issue to ensure that this point is thoroughly addressed in the revision.\n\n**References**\n\n[1] Wang et al., 2023. Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset.\n\n[2] Glasser et al., 2016. A multi-modal parcellation of human cerebral cortex.\n\n[3] Ansuini et al., 2019. Intrinsic dimension of data representations in deep neural networks.\n\n[4] Sorscher et al., 2022. Neural representational geometry underlies few-shot concept learning.\n\n[5] Schrimpf et al., 2018. Brain-score: Which artificial neural network for object recognition is most brain-like?.\n\n[6] Naselaris et al., 2011. Encoding and decoding in fMRI.\n\n[7] Soni et al., 2024. Conclusions about neural network to brain alignment are profoundly impacted by the similarity measure.\n\n[8] Recanatesi et al., 2019. Dimensionality compression and expansion in deep neural networks."}, {"review_id": "13P0hs7biS", "reviewer": "Reviewer_WfsF", "summary": "This study addresses the important question of how information is processed across hierarchical stages in both biological and artificial neural systems. The authors begin by clearly distinguishing between linear (effective) and non-linear (intrinsic) dimensionality. The work is focused on the vision modality. The authors compare different CNN-based networks with the different regions of the brain and compare their representational dimensionality. The results show that there is a striking mismatch between the ANNs and the human brain. In contrast to the brain where intrinsic dimensionality increases in a linear fashion, ANNs intrinsic dimensionality follows the canonical hunchback shape defined by an initial increase followed by a continuous decrease. \n\nThis is a very interesting and well-documented study aimed at understanding how the dimensionality of representations in the human visual cortex, specifically the ventral visual stream, evolves. Is it compression–expansion, continuous expansion similar to other species, or expansion–compression as observed in ANNs? Thus, the objective here is to investigate the commonalities and differences in representational geometry between biological and artificial systems.\n\nOverall, the authors have presented a well-curated study with a primary emphasis on biological systems (neuroscience), while artificial systems receive comparatively limited attention and serve mainly as a secondary point of comparison. I will thoroughly explain this in the Strengths and Weaknesses section.", "questions": "1. What image resolution was used when testing Human subjects and ANNs? Was there a difference between the two? If yes, then why?\n2. The authors define dimensionality later in the Methods section (Line 123), despite discussing it extensively earlier in the Introduction. Given that this is a computational neuroscience paper and is likely to attract readers from both machine learning and neuroscience backgrounds, it would benefit the reader if this definition were introduced earlier in the paper.\n3. The authors provide no details about the pre-trained models. They should provide information about their objective functions and other necessary information for readers who are not familiar with the ANNs training objectives.\n4. How did the authors calculate the relative depth of the ANNs as shown in the graphs?\n5. The authors provide no information about the human subjects or their age groups. If the authors believ this information was included in the manuscript or the Appendix section, please refer to those line numbers in the comment.", "limitations": "The authors have not included any limitations in this paper. It is highly recommended to add a Limitation section.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper conducts an extensive set of experiments on the biological side, specifically examining the human brain and its sub-regions in detail. The results are clean and well-presented for both biological and artificial systems. \n\n2. The paper has a single objective, and it achieves that objective through the experiments presented in the main text. However, I have some concerns regarding the experiments conducted for the ANNs, which I will address below in the 'Weakness' section.\n\nWeakness:\nDue to the following major weaknesses, I have deducted two points. If the authors can provide follow-up experiments addressing some of these concerns, I will increase the score.\n\n1. [MAJOR] Comparing ANNs with the human brain using pre-trained models may not be the most appropriate approach. While ANNs are typically trained on large datasets of static, isolated images (e.g., ImageNet), the human brain is exposed to continuous, smooth, and temporally structured visual input throughout life. This results in a significant mismatch in their respective visual diets (i.e., training data). As a result, it's unclear whether the differences reported in the study arise from this mismatch in training data or from deeper architectural or objective functions.\n\nA potential solution is to train ANNs on egocentric datasets (such as first-person video) or in simulated environments that better mimic the visual diets of humans. While such datasets may not achieve perfect equivalence, they would allow for a closer and more meaningful comparison between biological and artificial systems.\n\n2. [MAJOR] Does intrinsic dimensionality collapse in the deeper layers of all ANNs? The title and much of the language in the paper refer broadly to \"artificial neural networks (ANNs)\"; however, the experiments are conducted exclusively on convolutional neural networks (CNNs). While I have no objection to the exclusive use of CNNs for the current analyses, the framing may be misleading. As it stands, the paper appears to generalize its findings to all ANNs, including architectures such as Transformers, which have markedly different inductive biases and architectural features. Clarifying this distinction, or including additional experiments with non-CNN architectures, would strengthen the paper and ensure that its conclusions are correctly scoped.\n\nTransformers have a fundamentally different architecture compared to CNNs. Unlike CNNs, Transformers do not rely on pooling layers, hierarchical structures, or hardcoded spatial biases. Recent studies have shown that Transformers can mimic key features of the brain. For example, they show representations similar to those of hippocampal cells, demonstrate neuron–astrocyte-like interactions, and are capable of learning object recognition when trained through the eyes of newborn animals.\n\nGiven these findings, it is important to include Transformers in the analyses presented in this paper. Specifically, it would be valuable to test whether the results reported here can be replicated using a Vision Transformer (ViT) model. It is acceptable for the ViTs to be pre-trained, just as the CNNs used in the current experiments were. Furthermore, it is also acceptable and valid if the ViTs produce different results; such differences would still offer meaningful insights into architectural and representational differences between models.\nWithout testing other ANN types, the claims in this paper offer a weak generalization.\n\n3. [MINOR] Why is it that a randomly initialized ResNet-50 shows a similar ID pattern to that observed in Subject 1, while a trained ResNet-50 does not exhibit this pattern? I suggest repeating this experiment with different architecture sizes, such as ResNet-18, ResNet-34, and ResNet-101, just for the untrained case. It would be interesting to see whether the pattern observed in the untrained ResNet-50 is preserved across different architecture sizes.", "was_revised": true, "final_justification": "The authors have performed additional experiments and generalized their findings to a different class of models, transformers. The new results are strong and therefore strengthen the original claims.\n\nFurthermore, the authors have also addressed all the queries that were raised. Therefore, I am increasing my score.", "rebuttal": "Thank you for your feedback and interest in our work. Your suggestions helped us broaden our analyses and strengthen our results. Please let us know if there is further clarification we can provide.\n\n**1. Models trained with a more human-like visual diet**\n\nThank you for suggesting to include models trained with a more human-aligned visual diet. This was a highly valuable suggestion that substantially strengthened our study. In response, we extended our analyses with two additional experiments: i) models trained on a visual diet that better reflects human visual experience than ImageNet (Ecoset [1]), and ii) a model trained on ego-centric videos to capture object perception from a dynamic, first-person perspective (EgoVLP [2]). Interestingly, across both of these models, we observe effective (ED) and intrinsic (ID) dimensionality **both to increase across all layers**, without the late-stage ID compression found in ImageNet-trained models. Further, the Ecoset model only shows a weak drop in generalization performance to abstract features in the last layer. This suggests that training on more naturalistic visual statistics may encourage models to maintain richer, higher-dimensional representations throughout their processing hierarchy, supporting better generalization to abstract features and leading to geometric trends that more closely resemble those observed in the brain. The EgoVLP model, on the other hand, shows low, highly variable, and decreasing generalization to abstract features. It is important to note, however, that this model was trained on dynamic, egocentric videos that differ substantially from the static THINGS images used for evaluation. While EgoVLP may encode information in a high-dimensional subspace, the relevant feature separation might occur along dimensions that are not well captured by our test dataset. We are aware of datasets featuring human subjects watching naturalistic videos, as well as recent models specifically trained on such data. Incorporating these resources will allow for a more appropriate evaluation of egocentric and video-based models would be an interesting future direction. Overall, our new results highlight the importance of the visual diet in shaping representational geometry and suggest that aligning training data with human experience may be key to improving ANN–brain alignment.\n\n**2. Testing dimensionality collapse in non-CNN architectures**\n\nThank you for your comment regarding the generalizability of our findings beyond CNNs. We agree that framing the results as representative of all ANNs was too broad given the focus on CNNs. To broaden the scope of our findings, we extended our experiments to include ViTs [3], Swin transformers [4], and CLIP-ViT [5]. We show that all models exhibit a similar pattern of ID collapse in their deeper layers. Moreover, the ED of ViTs and Swin transformers also generally drops in later layers, while CLIP-ViT shows an ED pattern that is more similar to the brain and CNN architectures with a steady increase in ED along the hierarchy. These findings suggest that the observed divergence between brain and ANN representational geometry is not exclusive to CNNs, but also applies to ViTs, reflecting a more general limitation of current feedforward vision architectures and, as shown above, visual statistics that deviate from the human visual diet. Note that in response to other reviewer feedback, we extended our analyses to also include both **biologically inspired (CORNet [6]), unsupervised models (Autoencoders [7]), and other CNN-architectures (AlexNet, Inception, VGG16, ResNets)**. The unsupervised models showed an increase in ID throughout the encoder, but a marked drop in the decoder, while ED remained relatively stable across all layers. This pattern indicates a different type of mismatch to brain data compared to supervised models. Across all CORNet models, ED consistently increased along the hierarchy. For ID, CORNet-Z exhibited a continuous increase, while the recurrent models show the typical drop in late layers. However, CORNet-Z is very shallow, which might confound the results. All other CNN architectures exhibited the same dimensionality patterns as ResNet50. Overall, these extensions highlight that the dimensionality dynamics vary substantially across architectures and objectives, yet key mismatches to the brain persist under most conditions. We will incorporate these results in the revised manuscript and adjusted the title and text to more accurately reflect the scope of our analyses. We also appreciate the additional contextualization and studies mentioned by the reviewer and will include these in the manuscript.\n\n**3. Different untrained models**\n\nThank you for suggesting to test different model architectures to better understand the indeed surprising increase of ID in the untrained models. We observe a similar step-wise increase across all untrained ResNet models. While deeper architectures maintain a prolonged plateau in ID in the intermediate layers, all models consistently show a sharp increase in ID in the final two to three layers. Note however that across all untrained models, we observed markedly lower dimensionality compared to trained models, consistent with prior findings [8]. We will comment about this in the revised manuscript and will add the additional analyses to the supplement.\n\n**4. Image resolution**\n\nIn the fMRI experiment, images from the THINGS database were cropped to square format and varied in size (average: 996×996 pixels; minimum: 480×480 pixels; <1.8% smaller than 500 pixels). During presentation to human participants, images were shown on a gray background at a maximum visual angle of 10 degrees, with a 0.5-degree fixation crosshair overlaid [9]. For the ANN analyses, we followed the preprocessing procedures used during model training: images were typically rescaled to 224×224 pixels for input into the models. Thus, there is indeed a resolution difference between the human and ANN input. This reflects standard practice in computational neuroscience, where model inputs are adjusted to match the conditions of model training, ensuring valid and meaningful feature extraction. Importantly, since both humans and models were exposed to the same images (despite different resolutions), and our focus lies in comparing representational trends across stages rather than precise pixel-level responses, we do not expect this difference to qualitatively impact the main conclusions. Also, our main results regarding differences between models and the brain relate to the later processing stages that are less affected by image resolution and visual degree as early regions. Nonetheless, we will clarify and comment on this in the text.\n\n**5. Late introduction of dimensionality**\n\nThank you for the comment. We will introduce these concepts earlier in the text.\n\n**6. Details about pre-trained models**\n\nWe agree that more information about the pre-trained models will be helpful for readers not familiar with ANNs and their training objectives and will add this in the revised version. In short, the ResNet50 was trained on ImageNet to perform object classification. SimCLR was trained using a contrastive self-supervised learning objective on ImageNet, where the model learns to bring augmented views of the same image closer in the representation space while pushing apart views from different images. Finally, CLIP was trained using a contrastive image–text alignment objective on a large dataset of image–caption pairs, learning to associate images with their corresponding textual descriptions.\n\n**7. Relative depth calculation**\n\nThank you for pointing out the lack of detail regarding the computation of relative depth, a point that was also raised by other reviewers. First we extracted the activations of all ReLU layers (or GeLU in transformer based models). We then normalized the layer indices of these extracted layers by the total number of layers in the model. We will clarify this procedure in the Methods section of the revised manuscript.\n\n**8. Information about the human subjects**\n\nWe did not include more details about the human subjects in the appendix but will correct this in the revised supplement. Overall, the THINGS fMRI study included three participants (2 female, 1 male) with a mean age at the beginning of the study of 25.33 years. The behavioral study to generate the perceptual scores for the abstract features (e.g., heavy, moves, etc.) included 4,156 participants (2,418 female, 1712 male, and 26 other) with a mean age of 36.02±11.80 years. The BOLD5000 fMRI data comprises four individuals (3 female, 1 male) with a mean age of 25.5 years.\n\n**9. Missing limitations section**\n\nWe agree that including a dedicated limitations section will enhances the clarity and transparency of the paper, and will add one in the final version. While some limitations, e.g., the use of geodesic distance, were already addressed in the Discussion, we will make these points more explicit and consolidate them into a clearly labeled limitations section.\n\n**References**\n\n[1] Mehrer et al., 2021. An ecologically motivated image dataset for deep learning yields better models of human vision.\n\n[2] Lin et al., 2022, Egocentric Video-Language Pretraining.\n\n[3] Dosovitskiy et al., 2020. An image is worth 16x16 words: Transformers for image recognition at scale.\n\n[4] Liu et al., 2021. Swin transformer: Hierarchical vision transformer using shifted windows.\n\n[5] Radford et al., 2021. Learning transferable visual models from natural language supervision.\n\n[6] Kubilius et al., 2018. Cornet: Modeling the neural mechanisms of core object recognition.\n\n[7] Github: Horizon2333, imagenet-autoencoder.\n\n[8] Recanatesi et al., 2019. Dimensionality compression and expansion in deep neural networks.\n\n[9] Hebart et al., 2023. Things-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior."}]

Author rebuttal:

Meta-review: This paper investigates how the representational dimensionality evolves across the human visual cortex compared to artificial neural networks (ANNs) trained for vision tasks. Using fMRI responses and measures of effective (linear) and intrinsic (nonlinear) dimensionality, the authors show that dimensionality increases steadily along the human ventral pathway, supporting richer semantic abstraction in higher areas. In contrast, intrinsic dimensionality collapses in later layers of ANNs, limiting semantic generalization. The work shows that this pattern is independent on the specifics of the training objective (task) when trained on the same image database.

All reviewers agreed that the work is important in the field of neuroAI, helping to understand how ANNs are similar to biological brains and how they are not. During the rebuttal phase and encouraged by the reviewers comments, the authors performed several additional analyses whose results substantially improved the significance of the work. For example, the authors extended their analysis beyond the resnet50 based class of networks (e.g., vision transformers) and for different training datasets. These additional results showed that the original findings generalize to other network architectures but that the dimensionality collapse does not happen if the networks were trained on image sets that better resemble the natural visual diet of humans (Ecoset and EgoVLP). Including these results in the final version of the paper, as promised by the authors, will further strengthen the significance of the work, demonstrating the importance of the training diet in shaping the representational geometry of both artificial and biological neural networks.

Besides the question about generalization to transformer networks and the dependence on the training set, reviewers mostly had technical comments that were all well addressed during an extensive but very constructive rebuttal process. Tellingly, by the end all reviewers had increased their original score. The paper represents a quite valuable and timely contribution to the neuroAI field.
Decision: Accept (poster)