SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
E _ {g} \left(\mathcal {D} _ {\mathrm {t r}}\right) = | | \hat {\mathbf {Y}} \left(\mathcal {D} _ {\mathrm {t r}}\right) - \mathbf {Y} | | _ {F} ^ {2} / | | \mathbf {Y} | | _ {F} ^ {2}.
$$
Since the full distribution of stimuli are characterized by the empirical distribution  $\nu(\mathbf{s})$ , training sets are generated by randomly sampling  $p$  samples from  $\mathcal{D}_{\mathrm{full}}$ . Furthermore, the eigenvalue problem in Eq. SI.1.6 reduces to a PCA problem for the empirical kernel  $\mathbf{K} = \mathbf{X}\mathbf{X}^{\top}$
$$
\frac {1}{P} \mathbf {K v} _ {i} = \lambda_ {i} \mathbf {v} _ {i}, \tag {SI.1.11}
$$
and the projections of neural responses  $\mathbf{Y}$  on the eigenvectors  $\mathbf{v}_i$  reduces to
$$
\mathbf {w} _ {i} = \mathbf {Y} ^ {\top} \mathbf {v} _ {i}. \tag {SI.1.12}
$$
We note that this method may misestimate the neural prediction error, since the true eigenvalues and eigenvectors of the kernel  $K(\mathbf{s},\mathbf{s}^{\prime})$  are approximated by the finite empirical kernel matrix  $\mathbf{K}$  for  $P$  samples [63].
With the spectral data  $\{\lambda_i\}$  and  $\{\mathbf{w}_i\}$ , we obtain the dataset averaged neural prediction error  $E_{g}(p)$  (Eq. 3) as:
$$
E _ {g} (p) = \sum_ {i} W _ {i} E _ {i} (p), \quad E _ {i} (p) = \frac {\kappa^ {2}}{1 - \gamma} \frac {1}{(p \lambda_ {i} + \kappa) ^ {2}}, \quad W _ {i} = \| \mathbf {w} _ {i} \| ^ {2} / \| \mathbf {Y} \| _ {F} ^ {2}. \tag {SI.1.13}
$$
Here, we normalized the weights so that  $\sum_{i}W_{i} = 1$  and  $E_{g}(p = 0) = 1$ . The functions  $E_{i}(p)$  depend only on the eigenvalues and training set size  $p$ . Since each  $W_{i}$  is the normalized variance of  $\mathbf{Y}$  along the eigenvector  $\mathbf{v}_i$ ,  $W_{i}E_{i}(p)$  gives the contribution to the generalization error along mode  $i$  when the training set has size  $p$ . In our work, we defined these as error modes  $\widetilde{W_i} (p) = W_iE_i(p)$ .
To understand the behavior of the functions  $E_{i}(p)$ , we look at their limits. When  $p = 0$ ,  $E_{i}(0) = 1$  for all modes and no variance is explained. As  $p \to \infty$ , we either learn the variance  $W_{i}$  entirely ( $E_{i}(p \to \infty) \to 0$ ) if the corresponding  $\lambda_{i} > 0$ , or we do not learn it at all if  $\lambda_{i} = 0$  ( $E_{i}(p \to \infty) \to 1$ ). It can be further shown that, for all  $p > 0$ ,  $E_{i}(p) < E_{j}(p)$  if  $\lambda_{i} > \lambda_{j}$  [33], which implies that the percent reduction in  $\widetilde{W}_{i}(p)$  is greater for modes with larger eigenvalues.
Several remarks are in order:
- The ordered eigenvalues  $\{\lambda_i\}$  characterize the spectral bias of the model: for a fixed training set size  $p$ , the reduction in  $E_{i}(p)$  is larger for modes associated with large eigenvalues  $\lambda_{i}$  than for modes associated with small  $\lambda_{i}$ . Therefore, a sharply decaying eigenvalue spectrum implies that there is a bias toward certain modes being learned at higher fidelity than others, while a flat eigenvalue spectrum implies that learning will be equally distributed across all modes.
-  $W_{i}$  characterizes how much of the variance of the neural responses  $\mathbf{Y}$  lie in the  $i^{th}$  mode. These alignment coefficients  $\{W_{i}\}$  characterize task-model alignment; because of spectral bias, if most of the neural response variance is associated with eigenvectors with high  $\lambda_{i}$  terms, the neural prediction error will decrease faster compared to when most neural response variance is on eigenvectors with low  $\lambda_{i}$ . For example, if  $W_{i}$  are largest for the modes for which  $\lambda_{i}$  are smallest, we would expect a large generalization error when the training set size is small.
- We consider both  $\{\lambda_i\}$  and  $\{W_i\}$  as spectral properties of the model for the measured data (e.g. see Fig. 1C), and generalization error is determined by the interplay between these two spectral properties, together with the training set size  $p$ . For a fixed set of neural responses  $\mathcal{D} = \{\mathbf{s}^{\mu}, \mathbf{y}^{\mu}\}_{\mu=1}^{P}$ , different DNN models have different  $\{\lambda_i\}$  and  $\{W_i\}$  since for each model both eigenvalues (spectral bias) and the eigenvectors (task-model alignment) of the model activations change in a complicated way. Since we must specify both of these aspects of the model, comparison of model activations to neural responses should require at least two measures to be complete. Drawing from this argument, we may also conclude that reducing model/data comparisons to commonly used single scalar measures might be misleading as they may miss some of the factors contributing to the predictivity the others capture [64].
- In this work, we considered two geometrized measures  $R_{em}$  and  $\sqrt{D_{em}}$  that directly relate to the neural prediction error. The two measures jointly summarize whether the distribution of the error modes,  $\widetilde{W}_i$ , are relatively spread out (higher  $D_{em}$ , lower  $R_{em}$ ) or tightly concentrated (lower  $D_{em}$ , higher  $R_{em}$ ) at a fixed generalization error. We found that these terms were a convenient way to summarize spectral properties of the generalization error, but in principle, other measures could be defined.
SI.2 Experimental Details
Code for analyzing the empirical and theoretical neural prediction error can be found at https://github.com/chung-neuroai-lab/SNAP.
We conducted our experiments with various vision models trained with different learning paradigms and training schemes. In total, we used 32 models and picked several model stage activations from each. Except for vision transformers (ViT), chosen model stages were intermediate convolutional model stage activations after passed through a ReLU non-linearity. For ViT, we choose intermediate encoder model stages. For untrained models, models were randomly initialized for each iteration
for each experiment. All experiments were conducted on single Nvidia H100 or A100 GPUs with 80GB RAM using PyTorch 2 [65] and JAX 0.4 [66] libraries. Additional details on model stage selection and model architectures can be found in the associated code. In the remainder of this section, we report the models used in this study and explain step-by-step how we obtain the model stage activations to perform regression.
SI.2.1 Supervised Models
We investigated supervised vision models trained on the ImageNet 1000-way categorization task [67] with publicly available architectures and checkpoints obtained from the PyTorch library [65]. We included Alexnet [68], several ResNets [69], WideResNets [70], DenseNets [71], ConvNeXts [72], EfficientNets [73], MobileNet [74] and various vision transforms (ViT) [75].
We also investigated ANN models that were "biologically inspired" with a CorNet architecture [76]. For these models, the pre-trained weights are pulled using the library https://github.com/dicarlolab/CORnet.
In Table SI.2.1, we list all standard supervised models used in this study, together with their Top-1 ImageNet accuracies.
Supervised	ViT	Biologically Inspired
alexnet - Acc: 54.92	vit_b_32 - Acc: 75.16	cornet_s - Acc: 72.51
resnet18 - Acc: 68.98	vit_l_16 - Acc: 85.16	cornet_r - Acc: 53.92
resnet50 - Acc: 80.63	vit_l_32 - Acc: 76.35	cornet_z - Acc: 45.67
resnet101 - Acc: 81.71	vit_h_14 - Acc: 85.74	cornet_rt - Acc: 53.72
resnet152 - Acc: 82.25
wide_resnet50_2 - Acc: 78.34
wide_resnet101_2 - Acc: 78.77
densenet121 - Acc: 73.71
densenet161 - Acc: 76.70
densenet169 - Acc: 75.31
densenet201 - Acc: 76.20
convnext_small - Acc: 83.69
convnext_base - Acc: 83.92
convnext_large - Acc: 84.45
efficientnet_b0 - Acc: 76.87
efficientnet_b4 - Acc: 73.89
mobilenet_v2 - Acc: 71.13
Table 1: List of supervised models studied in this work together with their Top-1 ImageNet accuracies on the validation set.
SI.2.2 Self-Supervised and Adversarially Trained Models
We also considered models trained either with self-supervised training (SSL) [77, 78, 79] or adversarial training (referred to as "robust" models) [80, 81]. All analyzed models were ResNet50 or ResNet101 architectures.
For SSL models, we considered Barlowtwins [77], SimCLR [78] and MoCo [79] models whose pre-trained weights were pulled from https://github.com/facebookresearch/barlowtwins, https://github.com/facebookresearch/vissl and https://github.com/facebookresearch/moco-v3, respectively.
For the robust models, we obtained the pre-trained weights from https://github.com/MadryLab/robustness [50].
In Table SI.2.2, we list all SSL and robust models used in this study, together with their Top-1 ImageNet accuracies.
SSL	Robust
barlowtwins - Acc: 71.68	robust_resnet50_l2_3 - Acc: 56.25
simCLR_resnet50w1 - Acc: 65.71	robust_resnet50LINF_4 - Acc: 61.26
simCLR_resnet101 - Acc: 69.37	robust_resnet50LINF_8 - Acc: 46.33
moco_resnet50 - Acc: 74.15
Table 2: List of self-supervised (SSL) and adversarially-trained (robust) networks studied in this work, together with their Top-1 ImageNet accuracies on the validation set.
SI.2.3 Obtaining Model Stage Activations
We obtained activations from each model stage on the stimuli from neural datasets. We employ the image preprocessing steps that were used during model training. For all models, this preprocessing is resizing image stimuli to (224, 224) pixels and normalizing them with the mean and variance of the ImageNet dataset, which are  $\mu = (0.485, 0.456, 0.406)$  and  $\sigma = (0.229, 0.224, 0.225)$ , respectively. All activations were flattened before regression.
SI.2.4 Ridge Regression Experiment
After obtaining the model stage activations, we perform ridge regression by computing the predictions as:
$$
\mathbf {Y} ^ {*} (p) = \frac {1}{p} \mathbf {X} \mathbf {X} _ {\mathbf {1}: \mathbf {p}} ^ {\top} \left(\frac {1}{p} \mathbf {X} _ {\mathbf {1}: \mathbf {p}} \mathbf {X} _ {\mathbf {1}: \mathbf {p}} ^ {\top} + \alpha_ {\mathrm {r e g}} \mathbf {I} _ {p}\right) ^ {- 1} \mathbf {Y} _ {\mathbf {1}: \mathbf {p}}, \tag {SI.2.1}
$$
where  $\mathbf{I}_p$  denotes the  $p\times p$  identity matrix and  $\alpha_{\mathrm{reg}}$  denotes the ridge parameter. In this notation,  $\mathbf{Y}^{*}(p)$  is the  $P\times N$  matrix of the predictions for the entire neural response set that is inferred using only a subset of  $p$  samples as the training set. Then,  $\mathbf{X}_{1:\mathbf{p}}$  denotes the  $p\times M$  matrix of the training set that is the model activations for randomly sampled  $p$  stimuli and  $\mathbf{Y}_{1:\mathbf{p}}$  denotes the  $p\times N$  matrix of the corresponding neural activations.
Note that the least-squares regression is ill-defined when the number of data points are more than the number of regression variables (i.e.  $p > M$ ) since the matrix  $\mathbf{X}_{1:p}\mathbf{X}_{1:p}^{\top} + \alpha_{\mathrm{reg}}\mathbf{I}$  is singular for  $\alpha_{\mathrm{reg}} = 0$ . In order to account for this issue, one can use the push-through identity to obtain:
$$
\mathbf {Y} ^ {*} (p) = \frac {1}{p} \mathbf {X} \left(\frac {1}{p} \mathbf {X} _ {\mathbf {1}: \mathbf {p}} ^ {\top} \mathbf {X} _ {\mathbf {1}: \mathbf {p}} + \alpha_ {\mathrm {r e g}} \mathbf {I} _ {M}\right) ^ {- 1} \mathbf {X} _ {\mathbf {1}: \mathbf {p}} ^ {\top} \mathbf {Y} _ {\mathbf {1}: \mathbf {p}}, \tag {SI.2.2}
$$
where now  $\mathbf{I}_M$  is the  $M\times M$  identity matrix. The form in Eq. SI.2.2 is the conventional form of ridge regression, while the equation Eq. SI.2.1 is the form obtained via the so-called kernel trick [82, 36]. However, we only use the form in Eq. SI.2.1, since the model activations often are high-dimensional  $(\sim 1M)$  and hence  $M\gg P$ , where  $P\sim 10^{3}$  or less.
We repeat the same computation over 5 random choices of a training set and report neural prediction error Eq. SI.1.11 averaged over these trials. Furthermore, in Sec. SI.4, we show the agreement between empirically calculated and theoretically predicted errors.
In our experiments, we choose  $\alpha_{\mathrm{reg}} = 10^{-14}$  for numerical stability, which roughly corresponds to double-precision. Note that this choice of ridge parameter is to get close to least-squares regression, in which case  $\alpha_{\mathrm{reg}}$  is identically zero. We considered a very small ridge parameter, since even for ridgeless regression, strong spectral biases of overparameterized models avoid overfitting and effectively behave as if there is a non-zero ridge parameter [83, 32, 40, 84, 33, 85].
Here we chose a single regularization parameter for all experiments, however the choice of regularization parameters and the relation to the error modes is critical for fully characterizing the responses of encoding models. Future work could address this by choosing ridge parameters separately for different models and model stages, or allowing flexibility for different regularizations for individual unit responses (rather than one regularization for each neural region).
SI.2.5 Computing Theoretical Error Mode Geometry
After obtaining the model stage activations, we compute the eigenvalues and eigenvectors of each activation using the formula in Eq. SI.1.6. In terms of empirical Gram matrices, this corresponds to
performing Principal Component Analysis (PCA) on the Gram matrix. We project the neural response data onto the obtained eigenvectors and calculate  $W_{i}$ . Then, theoretical values of  $\sqrt{D_{em}}$  and  $R_{em}$  are calculated using the formula given in Eq. 5. For each  $p$ , the value of  $\kappa$  is solved self-consistently using the JAX auto-differentiation library [66] and SciPy optimizers [86].
SI.3 Relation to Brain-Score Neural Prediction Metrics
In this work, we considered ridge regression to compare model activations to neural responses. While there are many other choices for model comparisons such as canonical correlation analysis (CCA), partial least squares regression (PLS) and kernel-target alignment (KTA) [22, 23, 51], we chose to focus on ridge regression as it is theoretically well studied in terms of spectral properties [32, 33].
To demonstrate that the neural prediction error we obtained from ridge regression is comparable to other widely-used metrics, we directly compared our predicted values to those obtained from PLS using the Brain-Score [11] framework. Brain-Score uses PLS regression and reports the noise-corrected Pearson  $r^2$  score on test data to rank models. Note that PLS regression finds a new basis that maximizes the correlations between two sets. Then the regression is done on new variables by performing dimensionality reduction on both sets using the common basis. If one uses all the components in this new basis, then PLS is equivalent to least squares regression.
To calculate the PLS Pearson  $r^2$  value, we first extracted model activations as described in SI.2.3. We then fit PLS regression using the standard 25 latent components [11] and calculated the Pearson correlations between PLS-based predictions and actual neural responses on held-out data. We took the median correlation across all neurons and divided it by the median noise ceiling, which was calculated using split-half reliability across trials featuring identical stimuli. Train and test data points were randomly assigned, and the final Pearson  $r^2$  values were calculated by averaging over 10 random splits. This procedure is described in more detail in [11] and corresponds to the default analysis for public Brain-Score neural prediction benchmarks in the following regions and datasets:
V1: movshon.FreemanZiemba2013public.V1-pls
V2: movshon.FreemanZiemba2013public.V2-pls
V4:DICarlo.MajajHong2015public.V4-pls
IT:DICarlo.MajajHong2015public.IT-pls
Figure SI.3.1: Brain-Score PLS regression  $r^2$  values vs. ridge regression empirical  $E_g(p)$  values. Each point corresponds to the predictions obtained for a single model and stage on the corresponding region data. Pearson  $r^2$  values and Spearman  $\rho$  values are given for the correlation between the Brain-Score value and the neural prediction error. In all regions, there is a general trend where models that do well on one metric also do well on the other metric. A train test split of  $60\% / 40\%$  is used for our ridge regression procedure and the  $90\% / 10\%$  split for Brain-Score PLS.
In Fig. SI.3.1, we compare the Brain-Score Pearson  $r^2$  from each model stage to the empirical ridge-regression  $E_g(p)$ . The Brain-Score metric and neural prediction error from ridge regression are moderately correlated, despite the fact that we are (1) comparing values between ridge regression and PLS regression and (2) comparing a Pearson  $r^2$  metric to  $E_g(p)$  values, which are not directly comparable (Pearson  $r^2$  measures the angle between predictions and true labels, while neural prediction error measures the distance between the two).
To directly address the difference in metrics, in Fig. SI.3.2, we compare the Brain-Score PLS regression Pearson  $r^2$  on the test data to ridge regression Pearson  $r^2$  on the test data (using the same regression weights as were used to obtain the  $E_g(p)$  values above. When using the same metrics on the two different regression methods, there is a strong correlation for all regions. Combined, these results show that by using the ridge regression formulation we generally preserve the ordering of the commonly used Brain-Score metric.
Figure SI.3.2: Brain-Score PLS regression Pearson  $r^2$  values vs. ridge regression Pearson  $r^2$  values on test data. Each point corresponds to the predictions obtained for a single model and stage on the corresponding region data. Pearson  $r^2$  values and Spearman  $\rho$  values are given for the correlation between the Brain-Score value and the Pearson correlation obtained from the test set in the ridge regression. Note that our ridge regression Pearson  $r^2$  values are not noise corrected, but if we were to do so, it would correspond to dividing by a scalar value for each region (the noise ceiling obtained from the population). The same models and predictions are shown in Fig. SI.3.1, including the train test split of  $60\% / 40\%$  for our ridge regression procedure and the  $90\% / 10\%$  split for Brain-Score PLS.
SI.4 Empirical vs. Theoretical Neural Prediction Error
Complimentary to the discussion around Fig. 3, we confirmed that the theoretical predictions (see Sec. SI.1) obtained from the spectral properties of data match with the empirical regression discussed in Sec. SI.2 for various values of  $p$ . In Fig. SI.4.1, we plot the learning curves for selected model stages of a pre-trained ResNet18 architecture for all brain regions. We see that the theory matches the empirical values for all regions.
Note that particular choices of train/test splits may alter the ranking of which encoding models have the lowest neural prediction error (for instance, see layer4.1.relu changing from the lowest neural prediction error at small  $p$  to the highest error at large  $p$  in Fig SI.4.1 IT). Additionally, as we mentioned in the main text, increasing the train/test split yielded worse agreement between empirical and theoretical neural prediction errors. The main reason for this disagreement is that the theoretical values depend highly on the estimation of the true eigenvalue distribution of the empirical Gram matrix of data. However, as discussed in Sec. SI.1, the tail of the true eigenvalues are often underestimated by the empirical eigenvalues and that creates a discrepancy between observed  $E_{g}(p)$  and theoretically predicted one [63]. This discrepancy can be clearly seen in Fig. SI.4.2, where we plot the empirical neural prediction errors against the theoretically predicted ones. As the amount of training data increases (corresponding to a decrease in amount of test data), the correlation between the two quantities decreases monotonically.
Figure SI.4.1: Empirical vs. theoretical learning curves. In this figure, we show the learning curves for regression from selected model stages in a trained ResNet18 architecture to all brain regions. The colored dots correspond to the mean empirical error over 5 trials, and the colored dashed lines correspond to the theoretical learning curves obtained by the formula for  $E_{g}(p)$ . The vertical black line corresponds to the 60 train / 40 test split used for the main analysis in this paper.
Increasing Train/Test Split
Figure SI.4.2: Empirical and theoretical predictions worsen with train/test split. We show the empirical values of ridge regression  $E_{g}(p)$  compared to the theoretical values. As the amount of data used for training increases (corresponding to a decrease in test data), the correlation between empirical and theoretical values decreases.
SI.5 Additional Experimental Results
SI.5.1 Geometry grouped by architecture and training types
For visualization purposes, we show the error mode geometries grouped by the architecture (Fig. SI.5.1) and for a subset of models all with ResNet50 backbones but with different training types (Fig. SI.5.2).
Figure SI.5.1: Error mode geometry for supervised networks grouped by architecture type. All models included were trained in a supervised manner on ImageNet (standard training without any adversarial perturbations) and were obtained from publicly available checkpoints in the PyTorch Library, with the exception of CORNets which were obtained from https://github.com/dicarlolab/CORNet. Model groups were ResNet architectures (resnet18, resnet50, resnet101, resnet152), Wide-ResNet architectures (wide_resnet50_2, wide_resnet101_2), ConvNext architectures (convnext_base, convnext_small, convnext_large), DenseNet architectures (densenet121, densenet161, densenet169, densenet201), EfficientNet architectures (efficientnet_b0, efficientnet_b4), MobileNet architectures (mobilenet_v2), CORNet architectures (cornet_r, cornet_rt, cornet_s, cornet_z), and ViT architectures (vit_b_32, vit_h_14, vit_l_16, vit_l_32).
Figure SI.5.2: Error mode geometry for supervised and self-supervised ResNet50 architectures. Supervised ResNet50 architecture (trained on ImageNet without any adversarial perturbations) was compared to ResNet50 architectures trained with self-supervised methods SimCLR [78], MoCo [79], and BarlowTwins [77] There are no clear differences in geometries between models with different training mechanisms. This is in line with previous work that found similarities in representations between supervised and self-supervised networks [87, 88, 89].
SI.5.2 Geometry across model stages
In the main text, we focused on data from regions V1 and IT when analyzing the error mode geometry's dependence on model stage depth. Analysis of regions V2 and V4 is included in Fig. SI.5.3. Additionally, to show the error mode geometries from different model stages in Fig. 4A and Fig. SI.5.3A, we zoomed in on the parts of the space with the most data points and maintained the same axes limits for regions that shared the same stimulus set. In Fig. SI.5.4, we present all model stages, including outliers, colored by model stage depth.
B
Figure SI.5.3: Relation of error mode geometry with model stage depth in V2 and V4. (A) Each point represents error mode geometry for a specific model stage of trained neural network models, where the color of the value represents the depth of the model stage. For each model, the color code is normalized in the [0, 1] range where the earlier and later model stages have lighter and darker colors, respectively.  $R_{em}$  and  $\sqrt{D_{em}}$  values are obtained from theoretical values, and contour lines correspond to theoretical values of the neural prediction error as in Fig. 2. (B,C) Predicted  $E_{g}(p)$  that would be obtained when using the eigenspectra from the first (Early) or last (Late) stage of each model, paired with the alignment  $W_{i}$  terms from the Early or Late stage of each model. Full error mode geometry is given in (B), while summarized bar plots of the  $E_{g}(p)$  from each comparison is given in (C). (D) Full spectra for  $\lambda_{i}$ ,  $W_{i}$  and  $\widetilde{W_{i}}$  for Early and Late model stages of V2 and V4 datasets.
Figure SI.5.4: Error mode geometry for trained neural networks, colored by model stage depth. Same plots as Fig. 4A and Fig. SI.5.3A, but including the outliers.