SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Drew Linsley\*1,2, Ivan F. Rodriguez\*1, Thomas Fel3, Michael Arcaro4, Saloni Sharma5, Margaret Livingstone5, Thomas Serre1,2,3 drew_linsley, ivan_felipe_rodriguez@brown.edu
One of the most impactful findings in computational neuroscience over the past decade is that the object recognition accuracy of deep neural networks (DNNs) correlates with their ability to predict neural responses to natural images in the inferotemporal (IT) cortex [1,2]. This discovery supported the long-held theory that object recognition is a core objective of the visual cortex, and suggested that more accurate DNNs would serve as better models of IT neuron responses to images [3-5]. Since then, deep learning has undergone a revolution of scale: billion parameter-scale DNNs trained on billions of images are rivaling or outperforming humans at visual tasks including object recognition. Have today's DNNs become more accurate at predicting IT neuron responses to images as they have grown more accurate at object recognition?
Across three independent experiments, we find this is not the case: DNNs have become progressively worse models of IT as their accuracy has increased on ImageNet. To understand why DNNs experience this trade-off and evaluate if they are still an appropriate paradigm for modeling the visual system, we turn to recordings of IT that capture spatially resolved maps of neuronal activity elicited by natural images [6]. These neuronal activity maps reveal that DNNs trained on ImageNet learn to rely on different visual features than those encoded by IT and that this problem worsens as their accuracy increases. We successfully resolved this issue with the neural harmonizer, a plug-and-play training routine for DNNs that aligns their learned representations with humans [7]. Our results suggest that harmonized DNNs break the trade-off between ImageNet accuracy and neural prediction accuracy that assails current DNNs and offer a path to more accurate models of biological vision. Our work indicates that the standard approach for modeling IT with task-optimized DNNs needs revision, and other biological constraints, including human psychophysics data, are needed to accurately reverse-engineer the visual cortex.
The release of AlexNet [8] was significant not only for shifting the paradigm of computer vision into the era of deep learning, it also heralded a new approach for "systems identification" and
$^{1}$ Department of Cognitive, Linguistic, & Psychological Sciences, Brown University, Providence, RI
$^{2}$ Carney Institute for Brain Science, Brown University, Providence, RI
<sup>3</sup>Artificial and Natural Intelligence Toulouse Institute, Toulouse, France
$^{4}$ Department of Psychology, University of Pennsylvania, Philadelphia, PA
$^{5}$ Harvard Medical School, Cambridge, MA
approximating the transformations used by neurons in the visual cortex to build robust and invariant object representations. Over the past decade, deep neural networks (DNNs) like AlexNet, which are trained for object recognition on ImageNet [9], have been found to contain units that significantly better fit neural activity in inferior temporal (IT) cortex in non-human primates compared to classic hand-tuned models from computational neuroscience [1]. It was later found that DNN predictions of neural data improved as these models grew more accurate at object recognition and began to rival humans on the task [10, 11]. This surprising similarity between such task-optimized DNNs and brains supported the extant theory that object recognition is a core principle shaping the organization of the visual cortex [12], and raised the question of how important the many biological details amassed by visual neuroscience actually are for predicting neural responses in visual cortex. In the years since those findings, deep learning has undergone a revolution of scale, and current DNNs which rival or exceed human accuracy in vision and language are significantly larger and trained with orders of magnitude more data than ever before [13]. Does the object recognition accuracy of a DNN still correlate with its ability to predict IT responses to natural objects?
To answer this question, we turned to Brain-Score [3], the standard approach for benchmarking the accuracy of models at predicting neural activity in visual cortex of non-human primates. In brief, the Brain-Score evaluation method involves linearly mapping model unit responses to neural activity and then evaluating model unit predictions on held-out images. With this approach, we found a consistent trend across three different IT datasets hosted on the official Brain-Score website (Brain-score.org), which reflect neural responses to gray-scale versions of realistic rendered objects and natural images [14-16]. As DNNs have improved on ImageNet [9] over recent years, they have become progressively less accurate at predicting IT neuron responses to images (Fig. 1) *
In this study, we investigated two potential explanations for why task-optimized DNNs are turning into poor models of IT. (i) The internet data diets of DNNs and the routines used to train them to high accuracy on ImageNet leads them to learn the wrong visual features for explaining primate vision and IT responses to objects [7, 17-19]. (ii) Newer DNNs are becoming less brain-like in their architectures, and this problem has been magnified as DNNs have grown larger and ultimately deeper than the visual cortex [20].
Figure 1: Deep neural networks (DNNs) are becoming progressively worse models of inferior temporal (IT) cortex as they grow more accurate on ImageNet. Experimental data shown here is taken from Brain-score.org, and each experiment utilized different stimuli. The 104 dots in each panel depict the ImageNet accuracy and neural prediction accuracy of DNNs, and the grey-shaded region denotes the pareto-front governing the trade-off between these variables. EffNet=EfficientNet.
Contributions. To understand if the trade-off between ImageNet accuracy and IT predictions that we observed (Fig. 1) is due to the training routines and data diets of DNNs or their architectures, we turned to a new set of experimental recordings of IT neuronal responses to high-resolution color images [6]. The experimental images were significantly closer to the statistical distribution of images in ImageNet (Fig. S1.), unlike prior studies of IT [14]. The recordings also provided a coarse estimate of which image features drove neuronal activity (Fig. 2), which helped characterize DNN errors in explaining neural responses and explain why DNNs are becoming worse models of IT. We adopted the Brain-Score evaluation method to measure the prediction accuracy of 135 DNNs on recordings from medial (ML) and posterior (PL) lateral IT in two different Monkeys. To summarize our findings:
*In [11] it was noted that while there was an overall correlation of 0.92 between DNN accuracy on ImageNet and predicting IT responses, the correlation weakened for state-of-the-art models.
- We observed the same trade-off we found on Brain-Score.org data (Fig. 1) in each of our recordings: DNNs are becoming less accurate at predicting IT responses as they improve on ImageNet (Fig. 3).
- DNNs trained on ImageNet learn different features than those encoded by IT (Fig. 4), and this mismatch is not helped by training on more internet data, using newer DNN architectures like Transformers, relying on self-supervision, or optimizing for adversarial robustness.
- We successfully broke this trade-off by training DNNs with the neural harmonizer [7] and aligning the representations they learn for object recognition with humans.
- We further demonstrate that harmonized DNNs (hDNNs) are not only significantly better at explaining IT responses than any other DNN available, they also generate interpretable hypotheses on the features driving IT neuron responses.
(d)
Figure 2: IT recordings that reveal spatial maps of neuronal responses to complex natural images offer unprecedented insights into their feature selectivity [6]. (a) Neurons in posterior (PL) and/or medial (ML) lateral IT in two animals were localized using functional magnetic resonance imaging (fMRI), and neural responses to images were recorded using chronically implanted 32-channel multi-electrode arrays. (b) The monkeys were rapidly shown each image multiple times for 200ms each time (Monkey 1:  $n = 256$ , Monkey 2:  $n = 49$ ). Images were positioned differently each time to measure neural responses to every part of the image. (c) This procedure yielded spatially resolved maps of neural activity for an image, revealing the relative importance of different features for the recorded neurons. (d) DNNs were fit to these recordings following the Brain-Score evaluation method [11], in which partial least squares decoders were used to find the units in a DNN that provided the best match for neuronal responses to images.
Neural recordings. We leveraged recordings of IT neuronal responses from two monkeys [6], which were designed to reveal feature preferences of putative face-selective neurons. The recorded neurons also exhibited selectivity for non-face object features [6]. Recordings were made using chronically implanted 32-channel multi-electrode arrays within the fMRI-defined middle lateral (ML) and posterior lateral (PL) face patches of one monkey (Monkey 1), and ML of another monkey (Monkey 2, Fig. 2a). The activating regions in these areas were mapped  $(1 - 3^{\circ})$ , then each image was shown to the animals after they were cued to fixate at a specific position in space (Fig. 2b). The same images were shown multiple times while the monkeys fixated at a red dot in the center of the screen, which made it possible to derive spatial activity maps of neuronal responses (Fig. 2c). Fixation positions fell in a  $16 \times 16$  grid for Monkey 1 and  $7 \times 7$  grid for Monkey 2, and the average neuronal activity at each position was taken to generate spatial activity maps. A total of 14 images were shown to Monkey 1, and a different set of 14 images were shown to Monkey 2.
The recordings for Monkey 1 resulted in responses from 32 neurons in ML and 31 neurons in PL. For Monkey 2, we obtained responses from 32 neurons (see Appendix B for more details). Following [6], we binned neuronal responses every  $40\mathrm{ms}$  of the recording, from  $50\mathrm{ms}$  to  $250\mathrm{ms}$ . Within each bin, we calculated the noise ceiling for every neuron, which represents the maximum correlation achievable between any two neurons within that time interval. Noise ceilings for each recording were similar to those reported for standard IT Brain-Score datasets (Appendix B). In the main text, we present modeling results from the time bins that exhibited the highest average noise ceiling for each Monkey and recording site. Additional results from other time bins, which are consistent with these findings, can be found in Appendix E.
DNNs. We investigated the neural fits of 135 different DNNs representing a variety of approaches used in computer vision today: 62 convolutional neural networks trained on ImageNet [21-34,34-47] (CNNs), 23 DNNs trained on other datasets in addition to ImageNet (which we refer to as "DNN extra data") [32,36,48], 25 vision transformers [49-54] (ViTs), 10 DNNs trained with self-supervision [55, 56], and 15 DNNs trained to be robust to noise or adversarial examples [57, 58]. Each model was implemented in PyTorch with the TIMM toolbox (https://github.com/huggingface/pytorch-image-models) and pre-trained weights. Inference was executed on one NVIDIA TITAN X GPU. Additional model details, including the licenses used for each, are detailed in Appendix C
Neural Harmonizer. It was recently found that DNN representations are becoming less aligned with human perception as they evolve and improve on ImageNet [7, 59, 60]. A partial solution to this problem is the neural harmonizer, a training routine that can be combined with any DNN to align its representations with humans without sacrificing performance. Here, we test the hypothesis that aligning DNNs with human visual representations can similarly significantly improve their accuracy in predicting neural responses to images.
Training DNNs with the neural harmonizer on ImageNet involves an additional loss to standard cross-entropy for object recognition. This extra loss forces a model's gradients to appear as similar as possible to feature importance maps derived from human behavioral experiments (see [7] for details). To implement this loss, let  $\mathcal{P}_i(.)$  be a function that a multi-scale Gaussian pyramid of a human feature importance map  $\phi$  to  $N$ , with  $i \in \{1, \dots, N\}$ . During training, we seek to minimize  $\sum_{i}^{N}||\mathcal{P}_{i}(\pmb{g}(\pmb{f}_{\theta},\pmb{x})) - \mathcal{P}_{i}(\pmb{\phi})||^{2}$  in order to align feature importance maps between humans and DNNs at every scale of the pyramid. Before they are compared, feature importance maps from humans and DNNs are normalized and rectified using  $\pmb{z}(.)$ , a function that transforms a feature importance map  $\phi$  from either source to have 0 mean and unit standard deviation. This procedure yields the complete neural harmonization loss:
$$
\begin{array}{l} \mathcal {L} _ {\text {H a r m o n i z a t i o n}} = \lambda_ {1} \sum_ {i} ^ {N} \left| \left| \left(\boldsymbol {z} \circ \mathcal {P} _ {i} \circ \boldsymbol {g} \left(\boldsymbol {f} _ {\theta}, \boldsymbol {x}\right)\right) ^ {+} - \left(\boldsymbol {z} \circ \mathcal {P} _ {i} (\phi)\right) ^ {+} \right| \right| _ {2} (1) \\ + \mathcal {L} _ {C C E} (\boldsymbol {f} _ {\theta}, \boldsymbol {x}, \boldsymbol {y}) + \lambda_ {2} \sum_ {i} \theta_ {i} ^ {2} (2) \\ \end{array}
$$
Following the original neural harmonizer implementation [7] and training recipe, we trained six DNNs on ImageNet and human feature importance maps from the ClickMe game [61]: VGG16 [25], LeViT [62], ResNetV2-50 [63], EfficientNet_b0 [22], ConvNext [32], and MaxViT [64]. Each model was trained with Tensorflow 2.0 on 8 V4 TPU cores with all of the images in the ImageNet training set, and ClickMe human feature importance maps for the 200,000 images which were annotated. Images and feature importance maps were augmented with mixup [65], random left-right flips, and random crops during training. Only the object recognition loss was computed for images without human feature importance maps. Model weights were optimized with stochastic gradient descent and momentum, batches of 512 images, label smoothing [66], a learning rate of 0.3, and a learning rate schedule that began with a 5-epoch warm-up period followed by a cosine decay over 90 epochs at steps 30, 50 and 80.
Neural fitting. We evaluated the neural fit of each model by computing their fit separately for each IT recording, using the Brain-Score evaluation method [11]. This method involved fitting image-evoked activities from a layer of a deep neural network (DNN) to the corresponding neural responses using the partial least squares regression algorithm from Scikit-Learn.
To implement the Brain-Score evaluation method on the spatially-resolved recordings we investigate here, we first split each image shown to each animal into an equal number of patches as there were fixation locations in an image. Each image patch captured the receptive field of recorded neurons, and in total, there were 4,046 image patches for ML and 4,046 image patches for PL in Monkey 1, and 1,134 image patches for ML in Monkey 2. We measured DNN neuronal fits as the Spearman correlations between model predictions and true neural responses for each patch of an image held out of training divided by each neuron's noise ceiling. We then stored the median Spearman correlation over neurons and repeated the training/testing procedure to get the mean correlation across all images viewed by a monkey. Separate fitting procedures were performed for every layer of activities in a model, and we report a DNN's Brain-Score as its best possible fit across layers.
Figure 3: DNNs trained on ImageNet face a trade-off between achieving high object recognition accuracy and predicting responses to natural images in IT. We measured the accuracy of 135 DNNs at predicting image-evoked responses from neurons in posterior lateral (PL) and medial-lateral (ML) areas of IT [6] by computing the neural predictivity of each model with the Brain-Score evaluation method [11]. DNN neural predictivity is progressively worsening as models improve on ImageNet. This problem can be partially fixed by training DNNs with the neural harmonizer, which aligns their learned object representations with humans. Error bars denote  $95\%$  bootstrapped confidence intervals.
Task Optimization is insufficient for reverse-engineering IT. Ever since it was found that DNNs optimized for object recognition produce accurate predictions of IT neural responses to images, it was suggested that prediction accuracy would continue improve alongside DNN performance on ImageNet [10-12]. Is this the case with today's DNNs that rival or exceed the performance of human object recognition?
To answer this question, we leverage recordings of neuronal responses to high-resolution natural images in medial (ML) and posterior lateral (PL) IT (see Methods and Fig. 2). These images fall within the same statistical distribution as ImageNet images (Fig. S1), ensuring that our findings are not influenced by distributional shifts that are a well-known problem faced by computer vision models [67]. Image-evoked neural responses were spatially mapped, enabling insights into the visual features driving the responses of IT neurons and DNNs. Moreover, while these regions were localized according to their selectivity to face stimuli, they were also noted to respond to non-face stimuli [6].
Across 135 different DNNs, representing the variety of approaches used today in computer vision, we found that DNNs pretrained on ImageNet have become progressively less accurate at predicting ML and PL responses in two separate Monkeys (Fig. 3). For instance, ConvNext tiny, which achieved  $74.3\%$  accuracy on ImageNet, is as accurate in predicting neural responses to images as the ResNetv2-152x4, which reached  $84.9\%$  accuracy on ImageNet. Moreover, training routines that have been suggested to be more biologically plausible or yield representations that are closer to biological vision, such as training with self-supervision [68] or for adversarial robustness [69], made no difference. All DNNs trained on internet data faced a pareto-front that bounded their accuracy in predicting IT responses as a function of their ImageNet accuracy.
DNNs need biologically-aligned training routines There are at least two potential reasons why ImageNet-trained DNNs face a trade-off between object recognition and neural prediction accuracy: (i) DNN data diets and training routines work well for ImageNet but lead them to learn features
that are misaligned with biological vision [7, 17-19] or  $(ii)$  their architectures are a poor match to visual cortex [20]. To test this first possibility, we turned to the neural harmonizer. Given prior work demonstrating that aligning DNNs with human perception using the neural harmonizer significantly improved their ability to predict human behavior, we reasoned that harmonization might similarly improve DNN explanations of neural data by forcing them to rely on human-like visual features. Indeed, we found that harmonized DNNs (hDNNs) were significantly more accurate at predicting image-evoked responses in ML ( $T(5) = 5.30$ ,  $p < 0.01$ ) and PL ( $T(5) = 6.11$ ,  $p < 0.001$ ) neurons of Monkey 1 and ML ( $T(5) = 6.89$ ,  $p < 0.001$ ) neurons of Monkey 2 (reported as the average of  $T$  score from independent samples  $t$ -tests comparing the predictivity of each hDNN to the DNN with the highest neural-predictivity). The success of hDNNs indicates that the architectural mismatch between DNNs and the visual cortex is far less of a problem for modeling the visual cortex than the training routines and data diets that are being used today to achieve high accuracy on ImageNet. Behavior - even from humans - is an important constraint on DNNs that is needed to build more accurate models of primate visual cortex.
Figure 4: DNNs optimized for object recognition on ImageNet rely on different features than those encoded by neurons in primate inferior temporal (IT) cortex.. The activity of PL IT neurons is plotted next to the predicted activity of a model representing each class of DNNs: harmonized DNNs (hDNNs), visual transformers, self-supervised DNNs, convolutional neural networks, DNNs trained on more data than ImageNet, and adversarially robust DNNs.
Spatially-mapped neural responses reveal a feature mismatch between IT and DNNs The IT recordings used in the Brain-Score benchmark all utilized the same paradigm, in which hundreds or thousands of images were shown to multiple animals during passive viewing [14]. In contrast, the recordings we rely on involve many fewer unique images, but neural responses for each image are spatially mapped. This spatial mapping reveals what types of features are driving neural responses and makes it easier to understand the successes and failures of DNNs in explaining these data [6].
Although the neurons in ML and PL were located based on their selectivity to faces, the spatial maps of their responses revealed a much more complex response profile. IT neurons in both animals were strongly driven by faces, non-face objects, and contextual cues associated with faces [6]. In contrast, the best-fitting units of DNNs with state-of-the-art accuracy on ImageNet, like the ResNetv2-152x4 (Fig. 4), responded strongly to background features. This problem was shared by DNNs trained
on internet data but with routines that have been considered more biologically plausible, like self-supervised learning (e.g., SSL ResNet-18), or routines that yield perceptual robustness that is closer to humans, like adversarial training (e.g., Robust ResNetv2-50). hDNNs like the harmonized MaxViT, on the other hand, were, in general, reliably predictive of what parts of images elicited the most activity from IT neurons.
Harmonized ResNet 50
ResNet 50
Figure 5: Predictions of the visual features drive neuronal responses in PL of Monkey 1 from a harmonized ResNet-50 and a standard ResNet-50. Image patches for each model depict what the important features are across all images shown to the animal. The relative importance of each feature for predicting the recordings is color-coded in the bar chart on the right.
hDNNs generate testable predictions for visual neuroscience The surprising effectiveness of DNNs pre-trained on object recognition for predicting IT responses to images was a significant finding in computational neuroscience, which raised the tantalizing possibility that these models could form the basis of neuroprosthetics, and reduce the need for animal experiments. However, a persistent problem with DNNs since their inception is their lack of interpretability, meaning that using DNNs for systems identification effectively swaps one black box (the visual system) with another (a DNN) without getting the field any closer to understanding how vision works. Recent strides in the field of explainable artificial intelligence (XAI) have begun to alleviate this problem. For instance, it is now possible to reliably locate and characterize the features in datasets, like ImageNet, that drive patterns of model behavior using concept recursive activation factorization (CRAFT [70]). When paired with accurate models of IT and recordings that reveal locations in images that elicit neuronal responses, CRAFT can generate testable predictions of what those features are.