SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We further elaborate on the spectral properties of different model stages and the corresponding error mode geometry by using the  $\ell_{\infty}$  trained ResNet50 [50] model as an example. We plot the eigenvalues of the model activations from each model stage for all tested visual regions(Fig. SI.5.5). For V1 and V2, we saw that earlier model stages had better predictions, while for V4 and IT, later model stages performed better.
Figure SI.5.5: Spectral properties of  $\ell_{\infty}$  trained ResNet50 model on all regions. For each region (V1, V2, V4, IT), the left panel shows the eigenvalues of the model stages. The legend shows the best and worst model stage in terms of neural prediction error, and the colors correspond to the colors associated with that model stage (see Fig. 4). The right panel shows the error mode spectrum for each model stage, with the inset showing the ordered spectrum.
The spectral properties of each model stage depend highly on the input distribution. We notice that, under the input distribution of V1/V2 data, the eigenvalues of earlier model stages decayed more slowly than later model stages, while under the input distribution of V4/IT data this pattern is reversed where later model stages decayed more slowly. These differences are not due to the recorded neurons, as the eigenspectra is only a function of the stimulus set, pointing out the importance of considering the stimuli used for the neural predictions. Future work could consider choosing stimulus sets to maximize the difference between model eigenspectra. This would likely lead to greater differences between models in their predictions of neural activity patterns.
SI.5.3 Geometry for Trained vs. Untrained Models
In the main text analysis of trained vs. untrained models, we zoomed in on the parts of the space with the most data points and maintained the same axes limits for regions that shared the same stimulus set (Fig. 5)A. In Fig. SI.5.6, we show all the points of the trained and untrained networks, including the outliers. We also include an analysis of the error mode geometry for trained vs. untrained networks when predicting an additional V1 dataset [46] Fig. SI.5.7.
Figure SI.5.6: Error mode geometry changes with model training, full plots showing outliers. Same plot as Fig. 5A, but including the outliers
Figure SI.5.7: Error mode geometry for V1 dataset from [46]. Random and Trained error mode geometry for (A) previously analyzed V1 dataset, (B) V1 natural image dataset from [46] matched to stimulus set size of (A), and (C) using full V1 natural image dataset from [46]. In all cases, there was a small decrease in  $E_{g}(p)$  when using the trained neural network activations to predict the neural data (but this difference was less notable than in some of the other regions). Increasing the dataset size from 135 to 1250 stimuli did not change the overall trends.
SI.5.4 Geometry for Robust vs. Standard Models
In this section, we consider the model stage-wise changes in error mode geometry for an  $\ell_{\infty}$  adversarially trained network, as opposed to the  $\ell_{2}$  trained network presented in Fig. 6A. In Fig. SI.5.8, we find that the error mode geometry is very similar for this network, suggesting a consistent effect of adversarial training on the error mode geometry.
Figure SI.5.8: Error mode geometry for  $\ell_{\infty}$  adversarially trained model. Same plot as Fig. 6A, but shown for an  $\ell_{\infty}$  trained ResNet50.
For visualization purposes, in Fig. SI.5.9, we plot the eigenspectra and error mode weights for a standard and adversarially trained model.
SI.5.5 Effective Dimensionality and Neural Prediction Error
Previous work has suggested that the effective dimensionality of a model's responses to visual stimuli correlates strongly with its neural predictivity [31]. Here, we did not find a relationship between neural prediction error and the dimensionality of the model activations. We measured the effective dimensionality  $(ED)$  of the model stage responses for the stimuli in each neural experiment, extracted as described in SI.2.3, with the participation ratio of the eigenvalues of  $\mathbf{G}_{\mathbf{X}}$ :
$$
E D \equiv \frac {\left(\sum_ {i} \lambda_ {i}\right) ^ {2}}{\sum_ {i} \lambda_ {i} ^ {2}}. \tag {SI.5.1}
$$
We tested the correlation between  $ED$  and both Brain-Score performance and neural prediction error. In Fig. SI.5.10, we show that neural prediction error does not correlate well with model dimensionality across any of the datasets considered here. This is to be expected based on the theory presented - although the spectral properties of the model activations to the stimulus set is one of the factors
Figure SI.5.9: Spectral plots for standard and adversarially robust ResNet50. Eigenvalue spectra and error mode weights are given for all model stages of a standard ResNet50 (blue) and  $\ell_2(\epsilon = 3)$  Robust ResNet50 (green) for all brain regions.
leading to the observed values of  $\tilde{W_i}$ , the direction of the model eigenvectors is also important for determining the error mode geometry. These properties are captured by our proposed geometric quantities  $R_{em}$  and  $D_{em}$ , which are directly related to the neural prediction error. Additionally, in the main text, we investigated the neural prediction error in cases where we modified the eigenvalues or alignment coefficients (which are dependent on the model eigenvectors).
Figure SI.5.10: Effective dimensionality from model activations vs.  $E_{g}(p)$  values for each neural region. When the effective dimensionality and generalization error are measured using the full space of model activations, we do not see a strong correlation with either Pearson correlation ( $r^2$ ) or Spearman correlation ( $\rho$ ) between  $ED$  and  $E_{g}(p)$ .
We note that the authors of [31] applied either an average-pooling or principle-components-based dimensionality reduction method to model activations before carrying out their regression analyses. To ensure that we were able to reproduce the results of [31], we performed similar pooling on our model activations, which is complete average-pooling on activations from each convolutional filter. When average-pooling to our model activations, we found that setting  $\alpha_{\mathrm{reg}} = 10^{-14}$  as above yielded several outliers with unusually high  $E_g$  values. We therefore set  $\alpha_{\mathrm{reg}}$  to the value that yielded the lowest possible  $E_g$  for each set of model activations. When pooling model activations, we found a relatively strong relationship between prediction error and model dimensionality in IT similar to that reported in [31] (Fig. SI.5.11). We further replicated the results in [31] by directly comparing the effective dimensionalities obtained from the spatially averaged activations to the Brain-Score noise-corrected PLS regression values (Fig. SI.5.12). Taken together, these results indicate that pooling discards important variability in the model responses, and that when we use the full model activations the model dimensionality does not correlate well with neural prediction error.
Figure SI.5.11: Effective dimensionality from spatially averaged unit activations to stimulus set vs.  $E_{g}(p)$  values for each neural region. When the effective dimensionality and regression are both performed on the spatially averaged unit activations, we see a stronger correlation between the effective dimensionality and the neural responses, particularly in IT and V4. However, the neural prediction errors are generally higher when using these pooled responses (note the different y axes compared with Fig. SI.5.10), demonstrating that this type of pooling discards much of the important variability in the model responses
Figure SI.5.12: Effective dimensionality from spatially averaged unit activations vs. Brain-Score values for each region. When the effective dimensionality is calculated on the spatially averaged unit responses, rather than the full space of activations, we see a correlation that mirrors that reported in [31] for IT responses. In other brain regions, the relationship is less monotonic, appearing as the "joint regime" discussed in [31]. Note that the Brain-Score metric does not rely on taking the spatial average across units, but does perform dimensionality reduction as implemented with PLS.