SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 4: Model stage depth and error mode geometry. (A) Each point represents the error mode geometry obtained from a specific model stage of a trained neural network, where the color of the value represents the depth of the model stage. The color is normalized for each model to the [0,1] range where the earlier and later model stages have lighter and darker colors, respectively.  $R_{em}$  and  $\sqrt{D_{em}}$  values are obtained from theoretical values, and contour lines correspond to theoretical values of the neural prediction error as in Fig. 2. (B,C) Predicted  $E_g(p)$  that would be obtained when using the eigenspectra  $(\lambda_i)$  from the first (Early) or last (Late) stage of each of the 32 analyzed models, paired with the alignment coefficients  $(W_i)$  terms from the Early or Late stage of each model. Full error mode geometry is given in (B) where each point corresponds to a single model with the chosen  $W_i$  and  $\lambda_i$  terms, and (C) shows box plots summarizing the  $E_g(p)$  from each comparison. In V1, using the  $\lambda_i$  from the early stage of the model resulted in better neural prediction error and a lower  $R_{em}$  but higher  $\sqrt{D_{em}}$ , regardless of the  $W_i$  terms used. In IT, using the alignment terms  $W_i$  from the late model stage resulted in a smaller neural prediction error and lower  $R_{em}$ , with little change in  $\sqrt{D_{em}}$ . (D) Full spectra for  $\lambda_i$ ,  $W_i$  and  $\widetilde{W_i}$  from Early and Late model stages of ResNet50 on the V1 and IT datasets.
we measured  $\lambda_{i}$  from one model and paired it with the  $W_{i}$  measured from the other model (Fig. 5C). Using  $W_{i}$  from the trained model led to much lower  $R_{em}$  than when using  $W_{i}$  from the random model. This decrease in  $R_{em}$  when using the  $W_{i}$  terms from the trained model was the main driver of the improvement in  $E_{g}(p)$  when we used the trained model to predict the IT neural data. In contrast, when using the eigenspectra of the random model, the  $D_{em}$  was lower than when using the eigenspectra of the trained model, at the cost of a slight increase in  $R_{em}$ . Note that the model eigenspectra clearly decayed at a significantly faster rate in the random vs. trained models (Fig. 5D), but it was the properties of the alignment terms  $W_{i}$  that drove the change in  $E_{g}(p)$  between trained and untrained models. Thus, the eigenvectors of the trained model were better aligned with the neural data compared to the random model, which resulted in low prediction error regardless of which eigenspectra was used.
3.3 Adversarial vs. Standard Training
Recent work has suggested that adversarially trained networks have representations that are more like those of biological systems [47, 48, 15]. We analyzed the neural prediction error and error mode geometry of adversarially trained ("Robust") neural network models [49, 50], and their nonadversarially trained counterparts. The error mode geometry for an  $\ell_2(\epsilon = 3)$  ResNet50 is given in Fig. 6A, and similar results are shown for an  $\ell_{\infty}(\epsilon = 4)$  ResNet50 in Fig. SI.5.8.
The effect of adversarial training on error mode geometry varied across model stages and cortical regions. In the V1 dataset, lower neural prediction error was observed for early stages of robust
Figure 5: Error mode geometry changes with model training. (A) Error mode for trained models (blue) and randomly initialized models (green) for each brain region. The same model stage of each network is connected with a line, and stage depth is shown with opacity. Contours are given by theoretical values of  $E_{g}(p)$ , as in previous plots. (B) Neural prediction error was lower for trained models compared to random models in all brain regions, but most notable in V2, V4, and IT. Median is given as a black line and box extends from the first quartile to third quartile of the data. Whiskers denote 1.5x the interquartile range. (C) Predicted  $E_{g}(p)$  that would be obtained on IT dataset when using eigenspectra from trained or random ResNet50 models, paired with the alignment  $W_{i}$  terms from each of the trained or random ResNet50 models. Size of point denotes model stage depth. Using the  $W_{i}$  terms from the trained model resulted in a lower  $E_{g}(p)$  due to a lower  $R_{em}$ , while using the  $\lambda_{i}$  from the trained models increased  $D_{em}$  and did not appreciably change  $E_{g}(p)$ . (D) Model eigenvalues, alignments, and error modes for the final analyzed model stage of the ResNet50 architecture on IT dataset for both trained and random models. Even though the eigenspectra for the random model was notably different than the eigenspectra for the trained model, we observed in (C) that the driver of the decreased  $E_{g}(p)$  for the trained model was a difference in the  $W_{i}$  terms.
models (Fig. 6B), in line with previous work [48]. This corresponded to a decrease in  $R_{em}$  with little change in  $D_{em}$ , suggesting that the decrease in neural prediction error was shared relatively equally across all error modes. In contrast, at late model stages,  $D_{em}$  was smaller for the robust models compared to the standard models, but there was no difference in neural prediction error between the robust and standard networks. In regions V2, V4, and IT, we observed that there was little difference in neural prediction error between the standard and robust models, but that error mode geometry in robust models was associated with a lower  $D_{em}$  and a higher  $R_{em}$ . This suggests that the difference between robust and standard models in these regions was due to a small number of error modes having higher  $\widetilde{W_i}$  in the robust models. This did not lead to better neural prediction error for regions V2, V4, and IT, but nevertheless, the error mode geometry revealed that the data was predicted in different ways.
To better understand the differences between robust and standard networks in V1, we ran a series of experiments testing the effects of adversarial training on model eigenspectra and alignment with neural data. For each model stage of the ResNet50 architecture, we paired the eigenspectra  $(\lambda_{i})$  of the standard or robust network with the alignment coefficients  $(W_{i})$  of the standard or robust network (Fig. 6C). We then calculated the error mode geometry and neural prediction error for all four possible pairings. This approach allowed us to examine how adversarial training-related changes in the spectrum and alignment coefficients affected the error mode geometry for each model stage.
We found that the alignment coefficients of the robust models consistently yielded lower V1 prediction error (Fig. 6D). In early model stages, using the  $W_{i}$  from the robust models led to a lower neural
Figure 6: Error mode geometry and adversarial robustness. (A) Error mode geometry for standard ResNet50 (Blue) and adversarily-trained (Robust) ResNet50 (Green) models for each brain region. The same stage of each network is connected with an arrow, and stage depth is shown with opacity. Contours are given by theoretical values of  $E_{g}(p)$  as in previous plots. (B) For standard and robust ResNet50 models,  $E_{g}(p)$  for V1 data as a function of stage depth. (C) Predicted  $E_{g}(p)$  that would be obtained on V1 dataset for  $\lambda_{i}$  of standard or robust ResNet50 networks paired with  $W_{i}$  of standard or robust ResNet50 networks. Each point was from a single stage of the model, and plots are broken up by the early stages (first 8/16 analyzed stages) and the late stages (last 8/16 analyzed stages) of the model for visualization purposes. (D) Summarized neural prediction error for the experiment in (C). For all stages, the robust  $W_{i}$  led to lower error, but in the late stages, there was an additional effect from the choice of  $\lambda_{i}$ .
prediction error via reduced  $R_{em}$ , and the choice of  $\lambda_i$  had a negligible effect on the prediction error for these stages. In the later model stages, the  $W_i$  from the robust models also achieved a lower  $E_g$ , but for these stages, the choice of  $\lambda_i$  had a non-negligible impact on the resulting prediction error. Overall, these results suggest that adversarial training leads to an improved alignment between the eigenvectors of the model and early visual cortex activity.
Many metrics have been proposed to compare model representations to those observed in biological systems [51, 52, 22, 23, 11]. However, it is typically unclear which aspects of the representational geometry lead to a particular similarity score. In this work, we applied the theory of spectral bias and task-model alignment for kernel regression [33] to the generalization error for encoding models of neural activity. This provides a theoretical link between a model's ability to predict neural responses and the eigenspectra and eigenvectors of model representations. We introduced geometric measures  $R_{em}$  and  $D_{em}$  of the error modes to summarize how the spectral properties relate to neural prediction error, and showed that models with similar errors may have quite different geometrical properties. We applied this theory to investigate the roles of stage depth, learned representations, and adversarial training in neural prediction error throughout the ventral visual stream.
Dimensionality of neural and model representations
Recent work has investigated spectral properties of neural activity in biological [53, 54] and artificial [31, 28] systems. One empirical study suggested that high-dimensional model representations have
lower neural prediction error, where effective dimensionality was defined as the participation ratio of the eigenvalues alone [31]. However, we empirically found that this effective dimensionality of representations was not well correlated with prediction error for our datasets (Sec. SI.5.5). The lack of a consistent relationship between effective dimensionality and neural prediction error was expected based on our theory, as the neural prediction error depends both on the eigenvalues and eigenvectors of a model. These quantities co-vary from model to model, and thus model comparisons should include information from both. In this work, we demonstrated how our theory can be used to examine whether differences in observed prediction error are primarily due to the eigenvalues or alignment coefficients (which are dependent only on the eigenvectors), and found that in many instances, the alignment coefficients were the main driver of lower prediction error.
Dependence of Neural Prediction Error on Training Set Size
Our spectral theory of neural prediction characterizes the relationship between the training set size, the generalization error, and the spectral properties of the model and brain responses. This highlights that model ranking based on neural prediction error may be sensitive to the train/test split ratio. In Eq. 3, we see that, for small  $p$ , the error modes corresponding to small  $\lambda_{i}$  generally remain close to their initial value  $(\widetilde{W}_{i}(p) \approx \widetilde{W}_{i}(0))$ , and hence their contribution to neural prediction error does not decay (Eq. 4). On the other hand, at large sample size  $p$ , these small  $\lambda_{i}$  error modes change from their initial values and therefore may change the ranking of the models. These observations regarding the role of model eigenspectra and dataset size may be used in the future to design more comprehensive benchmarks for evaluating models.
Although our empirical analysis follows what we expect from theory and shows a diversity of error mode geometries, the datasets that we used (particularly for V1 and V2) have a limited number of samples. In some cases, such as the large  $p$  limit, theoretical results for the neural prediction error may not match empirical results. Additionally, we only investigated one dataset for V2, V4, and IT, and only two datasets for V1. Future work may reveal different trends when using different datasets. Moreover, this theory assumes that the brain responses are deterministic functions of the stimuli. As shown in [33], this assumption may be relaxed, and future work can examine the role of neuronal noise in these contexts.
The results in this paper demonstrate that our theory can characterize the complex interplay between dataset size, representational geometry, and neural prediction error. It provides the basis for numerous future directions relating spectral properties of the model and data to measures of brain-model similarity. Insights from our theory may suggest ways that we can build improved computational models of neural activity.
While we focused on DNN models pre-trained on image datasets, a closely related line of work has investigated the properties of DNN models directly trained to predict neural responses [46, 55, 56]. Future work could compare the error mode geometries from end-to-end networks to those obtained from networks pre-trained on image data. Such an analysis could elucidate how these two vastly different training schemes are both able to predict neural data well. Additionally, our experiments focused on the case of the same small regularization coefficient for all regression mappings, but future work could explore the case of optimal regularization. The spectral theory can also be applied to other types of brain-model comparisons, such as Partial Least Squares or Canonical Correlation Analysis.
Although we were able to differentiate many error mode geometries based on  $R_{em}$  and  $\sqrt{D_{em}}$ , we initially expected to see many more values of  $(R_{em}, \sqrt{D_{em}})$  for each fixed  $E_g(p)$ . The investigated vision models seem to be constrained to a particular region in the  $(R_{em}, \sqrt{D_{em}})$ -space (Fig. 3). Future work may shed light on whether this is due to implicit biases in model architectures or due to a fundamental coupling of  $(R_{em}$  and  $\sqrt{D_{em}})$  due to constraints on the  $E_i(p)$  term.
There are many architectural differences between the tested DNNs and biological systems. Future work may investigate how biologically plausible constraints [9] (e.g., stochasticity [48, 57], divisive normalization [58], power-law spectra [59]) and diverse training objectives [44, 13]) lead to improved neural prediction error through changes in error mode geometry.
We would like to thank Nga Yu Lo, Eero Simoncelli, N Apurva Ratan Murty, Josh McDermott and Greta Tuckute for helpful discussions, and Jim DiCarlo for comments on an early version of this work. We would also like to thank the reviewers for their helpful comments, which strengthened the paper. This work was funded by the Center for Computational Neuroscience at the Flatiron Institute of the Simons Foundation and the Klingenstein-Simons Award to S.C. All experiments were performed on the Flatiron Institute high-performance computing cluster.
Broader Impacts
Our work provides a novel approach towards comparing the fidelity of neural encoding model predictions, and may provide insight that leads to improved models of neural activity. Such models are applicable for neuroengineering purposes, such as retinal prosthetics or cochlear implants. However, training the deep learning models tested in the paper consumes a large amount of energy, contributing to carbon emissions. Although we used pre-trained neural networks here, future work may need to train networks with biases specifically for low neural prediction error.
SI.1 Spectral Theory of Generalization Error in Regression
The spectral theory of generalization error for regression, as developed in [32, 33], relates the spectral properties of the data and target to the generalization error in an analytical way. Here, we summarize the key results from these works and describe how we use the results in our context.
For a generic regression task, we begin by considering a model  $\mathbf{x}(\mathbf{s}) : \mathbb{R}^D \to \mathbb{R}^M$ , which maps  $D$ -dimensional stimuli to a set of  $M$ -dimensional features  $\mathbf{x}$ . Additionally, we assume that the neural responses are generated by a ground truth function  $\mathbf{y}(\mathbf{s}) : \mathbb{R}^D \to \mathbb{R}^N$ , mapping the stimuli to the activations of a set of  $N$  biological features  $\mathbf{y}$  (e.g., firing rates). Finally, we assume that the stimuli are drawn from a generic distribution  $\mathbf{s} \sim \nu(\mathbf{s})$ .
In regression, the goal is to estimate the ground truth  $\mathbf{y}(\mathbf{s})$  from a training set of size  $p$  and test it on all possible stimuli as drawn from  $\nu(\mathbf{s})$ . Given a training dataset with  $p$ -samples  $\mathcal{D}_{\mathrm{tr}} = \{\mathbf{s}^{\mu}, \mathbf{y}^{\mu}\}_{\mu=1}^{p}$  where  $\mathbf{s}^{\mu} \sim \nu(\mathbf{s})$  and  $\mathbf{y}^{\mu} = \mathbf{y}(\mathbf{s}^{\mu})$ , ridge regression returns an estimator  $\hat{\mathbf{y}}(\mathbf{s})$  as:
$$
\hat {\mathbf {y}} (\mathbf {s}; \mathcal {D} _ {\mathrm {t r}}) = \mathbf {x} (\mathbf {s}) ^ {\top} \hat {\boldsymbol {\beta}} (\mathcal {D} _ {\mathrm {t r}})
$$
$$
\hat {\beta} \left(\mathcal {D} _ {\mathrm {t r}}\right) = \underset {\beta \in \mathbb {R} ^ {M \times N}} {\arg \min } \sum_ {\mu = 1} ^ {p} \left\| \mathbf {y} ^ {\mu} - \mathbf {x} \left(\mathbf {s} ^ {\mu}\right) ^ {\top} \beta \right\| ^ {2} + \alpha_ {\mathrm {r e g}} \| \beta \| _ {F} ^ {2}. \tag {SI.1.1}
$$
where  $\alpha_{\mathrm{reg}}$  is the ridge parameter introduced to regularize the solution. Here, the regression weights and estimator depend on the particular choice of the training set  $\mathcal{D}_{\mathrm{tr}}$  of size  $p$ . We define the normalized generalization error as:
$$
E _ {g} \left(\mathcal {D} _ {\mathrm {t r}}\right) = \frac {1}{\int d \mathbf {s} \nu (\mathbf {s}) \left\| \mathbf {y} (\mathbf {s}) \right\| ^ {2}} \int d \mathbf {s} \nu (\mathbf {s}) \left\| \mathbf {y} (\mathbf {s}) - \hat {\mathbf {y}} (\mathbf {s}; \mathcal {D} _ {\mathrm {t r}}) \right\| ^ {2}, \tag {SI.1.2}
$$
which also depends on the training set  $\mathcal{D}_{\mathrm{tr}}$ . Explicit solution of the objective in Eq. SI.1 gives:
$$
\hat {\mathbf {y}} (\mathbf {s}; \mathcal {D} _ {\mathrm {t r}}) = \mathbf {x} (\mathbf {s}) ^ {\top} \mathbf {X} _ {1: \mathbf {p}} ^ {\top} \left(\mathbf {X} _ {1: \mathbf {p}} \mathbf {X} _ {1: \mathbf {p}} ^ {\top} + \alpha_ {\mathrm {r e g}} \mathbf {I}\right) ^ {- 1} \mathbf {Y} _ {1: \mathbf {p}}, \tag {SI.1.3}
$$
where  $\mathbf{X}_{1:\mathbf{p}}\in \mathbb{R}^{p\times M}$  is the matrix of the model responses to the stimuli in  $\mathcal{D}_{tr}$ ,  $\mathbf{Y}_{1:\mathbf{p}}\in \mathbb{R}^{p\times N}$  is the matrix of the neural responses in  $\mathcal{D}_{\mathrm{tr}}$ , and  $\mathbf{I}$  denotes the  $p\times p$  identity matrix. Defining the kernel function  $K(\mathbf{s},\mathbf{s}^{\prime}) = \mathbf{x}(\mathbf{s})^{\top}\mathbf{x}(\mathbf{s}^{\prime})$ , the predictor can be written as:
$$
\hat {\mathbf {y}} (\mathbf {s}) = \mathbf {k} (\mathbf {s}) ^ {\top} \left(\mathbf {K} + \alpha_ {\mathrm {r e g}} \mathbf {I}\right) ^ {- 1} \mathbf {Y} _ {\mathbf {1}: \mathbf {p}}, \tag {SI.1.4}
$$
where  $\mathbf{K}$  is  $p\times p$  matrix with elements  $\mathbf{K}_{\mu \nu}\equiv K(\mathbf{s}^{\mu},\mathbf{s}^{\nu})$  and  $\mathbf{k}(\mathbf{s})$  is a  $p$  -dimensional vector with elements  $\mathbf{k}(\mathbf{s})_{\mu}\equiv K(\mathbf{s},\mathbf{s}^{\mu})$  . Hence, the generalization error depends on the kernel  $K(\mathbf{s},\mathbf{s}^{\prime})$  , a particular draw of a training set  $\mathcal{D}_{\mathrm{tr}}$  of size  $p$  , and the ridge parameter  $\alpha_{\mathrm{reg}}$
Note that every time a new training set is drawn from the stimuli distribution  $\nu(\mathbf{s})$ , the generalization error may change. We are interested in the dataset averaged generalization error which only depends on the size of the training set  $p$  rather than the particular instances of each  $D_{\mathrm{tr}}$ . This is defined as:
$$
E _ {g} (p) = \left\langle E _ {g} \left(\mathcal {D} _ {\mathrm {t r}}\right) \right\rangle_ {\mathcal {D} _ {\mathrm {t r}} \sim \nu (\mathbf {s})} \tag {SI.1.5}
$$
where the average is taken over all possible choices for  $\mathcal{D}_{\mathrm{tr}}$  of size  $p$ . Now, this quantity only depends on the training set size  $p$ , input distribution  $\nu(\mathbf{s})$ , model kernel  $K(\mathbf{s},\mathbf{s}^{\prime})$  and the ground truth function  $\mathbf{y}(\mathbf{s})$ . This quantity can also be thought of as the cross-validated generalization error, where the generalization errors for each regressor corresponding to a particular instance of  $\mathcal{D}_{\mathrm{tr}}$  are averaged.
The theory of generalization error in [32, 33] finds that this dataset averaged  $E_{g}(p)$  depends on the spectral properties of the kernel. Mercer's theorem [60] suggests that the kernel can be decomposed into orthogonal modes as  $K(\mathbf{s},\mathbf{s}^{\prime}) = \sum_{i}\lambda_{i}v_{i}(\mathbf{s})v_{i}(\mathbf{s}^{\prime})$  where the eigenvalues  $\lambda_{i}$  and orthonormal eigenfunctions  $v_{i}(\mathbf{s})$  are obtained by solving the equation
$$
\int d \mathbf {s} ^ {\prime} \nu (\mathbf {s} ^ {\prime}) K (\mathbf {s}, \mathbf {s} ^ {\prime}) v _ {i} (\mathbf {s} ^ {\prime}) = \lambda_ {i} v _ {i} (\mathbf {s}), \quad \int d \mathbf {s} \nu (\mathbf {s}) v _ {i} (\mathbf {s}) v _ {j} (\mathbf {s}) = \delta_ {i j}. \qquad \mathrm {(S I . 1 . 6)}
$$
Here we emphasize that the resulting eigenvalues and eigenfunctions strongly depend on the distribution of the stimuli  $\nu(\mathbf{s})$ . For example, for the same model kernel, the spectrum might look completely different depending on the stimuli being textures or natural scenes.
The dependence of the neural response function  $\mathbf{y}(\mathbf{s})$  to the generalization error comes from the mode decomposition of  $\mathbf{y}(\mathbf{s})$  on to the eigenfunction  $v_{i}(\mathbf{s})$ :
$$
\mathbf {y} (\mathbf {s}) = \sum_ {i} \mathbf {w} _ {i} v _ {i} (\mathbf {s}), \quad \mathbf {w} _ {i} \equiv \int d \mathbf {s} \nu (\mathbf {s}) \mathbf {y} (\mathbf {s}) v _ {i} (\mathbf {s}), \tag {SI.1.7}
$$
where  $\mathbf{w}_i$  is the projection of the neural response function  $\mathbf{y}(\mathbf{s})$  onto the  $i^{th}$  eigenfunction  $v_{i}(\mathbf{s})$ .
As described in [33, 32, 61, 62], using techniques from statistical mechanics, the generalization error is given by the self-consistent equations
$$
E _ {g} (p) = \frac {\kappa^ {2}}{1 - \gamma} \sum_ {i} \frac {W _ {i}}{(p \lambda_ {i} + \kappa) ^ {2}}, \tag {SI.1.8}
$$
$$
\kappa = \alpha_ {\mathrm {r e g}} + \kappa \sum_ {i} \frac {\lambda_ {i}}{p \lambda_ {i} + \kappa}, \quad \gamma = \sum_ {i} \frac {p \lambda_ {i} ^ {2}}{(p \lambda_ {i} + \kappa) ^ {2}},
$$
where  $\kappa$  must be solved self-consistently, and we defined the alignment coefficients  $W_{i}$
$$
W _ {i} \equiv \frac {\left\| \mathbf {w} _ {i} \right\| ^ {2}}{\int d \mathbf {s} \nu (\mathbf {s}) \left\| \mathbf {y} (\mathbf {s}) \right\| ^ {2}} = \frac {\left\| \mathbf {w} _ {i} \right\| ^ {2}}{\sum_ {j} \left\| \mathbf {w} _ {j} \right\| ^ {2}}, \tag {SI.1.9}
$$
which give the ratio of the variance of neural responses along mode  $i$  to its total variance. In most cases, it is not possible to find an analytical solution to  $\kappa$ , but it is straightforward to solve it numerically.
In this paper, since our data are finite, we take the distribution  $\nu(\mathbf{s})$  to be the empirical distribution on the full dataset  $\mathcal{D}_{\mathrm{full}} = \{\mathbf{s}^{\mu}, \mathbf{y}^{\mu}\}_{\mu=1}^{P}$  of  $P$  data points:
$$
\nu (\mathbf {s}) = \frac {1}{P} \sum_ {\mu = 1} ^ {P} \delta \left(\mathbf {s} - \mathbf {s} ^ {\mu}\right), \tag {SI.1.10}
$$
which is an approximation to the true stimuli distribution where  $P \to \infty$ .
In this case, the functions  $\mathbf{x}(\mathbf{s})$  and  $\mathbf{y}(\mathbf{s})$  are fully described by the data matrices,  $\mathbf{X}$  and  $\mathbf{Y}$ , and the generalization error in Eq. SI.1.2 becomes the neural prediction error,