Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex
Authors: Drew Linsley
Ivan F Rodriguez Rodriguez
Thomas FEL
Michael Arcaro
Saloni Sharma
Margaret Livingstone
Thomas Serre
Venue: NeurIPS 2023 poster (2023)
Keywords: neural system identification
behavioral alignment
neural object recognition
TL;DR: Deep neural networks are becoming worse models of neurons in primate inferotemporal cortex as they have improved on ImageNet.

Abstract: One of the most impactful findings in computational neuroscience over the past decade is that the object recognition accuracy of deep neural networks (DNNs) correlates with their ability to predict neural responses to natural images in the inferotemporal (IT) cortex. This discovery supported the long-held theory that object recognition is a core objective of the visual cortex, and suggested that more accurate DNNs would serve as better models of IT neuron responses to images. Since then, deep learning has undergone a revolution of scale: billion parameter-scale DNNs trained on billions of images are rivaling or outperforming humans at visual tasks including object recognition. Have today's DNNs become more accurate at predicting IT neuron responses to images as they have grown more accurate at object recognition?

Surprisingly, across three independent experiments, we find that this is not the case. DNNs have become progressively worse models of IT as their accuracy has increased on ImageNet. To understand why DNNs experience this trade-off and evaluate if they are still an appropriate paradigm for modeling the visual system, we turn to recordings of IT that capture spatially resolved maps of neuronal activity elicited by natural images. These neuronal activity maps reveal that DNNs trained on ImageNet learn to rely on different visual features than those encoded by IT and that this problem worsens as their accuracy increases. We successfully resolved this issue with the neural harmonizer, a plug-and-play training routine for DNNs that aligns their learned representations with humans. Our results suggest that harmonized DNNs break the trade-off between ImageNet accuracy and neural prediction accuracy that assails current DNNs and offer a path to more accurate models of biological vision. Our work indicates that the standard approach for modeling IT with task-optimized DNNs needs revision, and other biological constraints, including human psychophysics data, are needed to accurately reverse-engineer the visual cortex.

Performance-optimizze deep neural networks are evolving into worse models of inferotemporal visual cortex
Drew Linsley*1.2 Iván F. Rodriguez"), Thomas Fel', Michael Arcaro",
Saloni Sharma², Margaret Livingstone", Thomas S.,3 drew frew_insseey ivan_ fel aeeiiierrrrrrrr rodri cedrrigez@bbonnc.o edu
Abstract
One of the most impactful findings in computational neuroscience over the past decade is that the object recognition accuracy of deep neural networks (DNNs) correlates with their ability to predict neural responses to natural images the inferotemporal (IT) cortex 2 This discovery supported the long-held theory that object recognition a core objective of the visual cortex, and suggested that more accurate DNNs would serve better models of IT neuron responses images 5 Since then, deep learning has undergone a revolution scale: billion parameter-scale DNNS trained billions of images rivaling outperforming humans at visual tasks including object recognition. Have today's 's DNNs become more accurate at predicting IT neuron responses images they have grown more accurate at object recognition?
Across three independent experiments, we find this is not the case: DNNs have become progressively worse models ImageNet. To understand why DNNs experience this trade-off and evaluate if they are still an appropriate paradigm for modeling the visual system, we turn recordings of IT that capture spatially resolved maps of neuronal activity elicited by natural images (6]. These neuronal activity maps reveal that DNNs trained ImageNet learn to rely different visual features than those encoded by and that this problem worsens as their accuracy increases. We successfully resolved this issue with the neural harmonizer, plug and paa- training routine for DNNs that aligns their learned representations with humans (7). Our results suggest that harmonized DNNs break the trade-off between ImageNet accuracy and neural prediction accuracy that assails current DNNs and offer a path to more accurate models of biological vision. Our work indicates that the standard approach for modeling with task-optimized DNNs needs revision, and other biological constraints, including human psychophysics data, needed to accurately reverse- engineer the visual cortex.
as their accuracy increased on
Introduction
The release of AlexNet 8 was significant not only for shifting the paradigm of computer vision into the era of deep learning, it also heralded a new approach for "systems identification" and
"These authors contributed equally.
Depparment of Cognitive, Linguistic, & Psychological Sciences, Brown University, Providence, RI
"Camnyy Institute for Brain Science, Brown University, Providence, RI
3 Artificial and Natural Intelligence Toulouse Institute, Toulouse, France "Department of Psychology, University of Pennsylvania, Philadelphia, PA Harvard Medical School, Cambridge, MA
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

approximating the transformations used by neurons in the visual cortex to build robust and invariant object representations. Over the past decade. deep neural networks (DNNs) like AlexNet, which are trained for object recognition on ImageNet [9]. have been found to contain units that significantly better fit neural activity inferior temporal (TT) cortex non-human primates compared to classic hand-tuned models from computational neuroscience (T. It was later found that DNN predictions of neural data improved as these models grew more accurate at object recognition and began to rival humans the task I... This surprising similarity between such task-optimized DNNs and brains supported the extant theory that object recognition a core principle shaping the organization of the visual cortex [12]. . and raised the question of how important the many biological details amassed by visual neuroscience actually are for predicting neural responses in visual cortex. In the years since those findings, deep learning has undergone revolution of scale, and current DNNs which rival or exceed human accuracy in vision and language are significantly larger and trained with orders of magnitude more data than ever before [13]. Does the object recognition accuracy of DNN still correlate with its ability to predict IT responses to natural objects?
To answer this question, we turned to Brain-Score [3), the standard approach for benchmarking the accuracy of models at predicting neural activity in visual cortex of non-human primates. In brief, the Brain-Score evaluation method involves linearly mapping model unit responses to neural activity and then evaluating model unit predictions on held-out images. With this approach, we found consistent trend across three different IT datasets hosted the official Brain-Score website (Brain- score. score.org, org). which reflect neural responses to gray-scale versions of realistic rendered objects and natural images [4 16]. . As DNNs have improved ImageNet [9] over recent years.
they have become progressively less accurate at predieting IT neuron responses to images (Fig. D
In this study, we investigated two potential explanations for why task-optimized DNNs are turning into poor models of IT. (i) The internet data diets of DNNs and the routines used to train them to high accuracy ImageNet leads them to learn the wrong visual features for explaining primate vision and IT responses to objects 1777-19)
(ii) Newer DNNs are becoming less brain-like their architectures, and this problem has been magnified as DNNs have grown larger and ultimately deeper than the visual cortex 20
0.7
Majaj et al., 2015
Sanghavi et al., 2020a
Sanghavi et al., 2020b
CNN
Neural predictivtt
0.0
Resfict Piccaa
Mobiiche
Reckeveno
Robust
Transtormar
0.5
Self supe-supe-
Modi alle:
Reedial
Incopter
0.4
RegNe:
Moelot
vel
0.3
Modhss
30%
40%
50%
60%
70%
80%
90%
30%
40%
50%
60%
70%
90%
90%
30%
40%
50%
60%
70%
80%
90%
ImageNet Accuracy
Figure 1: Deep neural networks (DNNs) are becoming progressively worse models of inferior temporal (TT) cortex as they grow more accurate on ImageNet. Experimental data shown here is taken from Brain- score. org, and each experiment utilized different stimuli. The 104 dots in each panel depict the ImageNet accuracy and neural prediction accuracy of DNNs, and the grey-shaded region denotes the pareto-front governing the trade-off between these variables. EffiintEfficieeNNee..
Contributions. To understand if the trade-off between ImageNet accuracy and IT predictions that we observed (Fig. [ is due to the training routines and data diets of DNNs or their architectures, we turned to a new set of experimental recordings of IT neuronal responses to high-resolution color images [6]. The experimental images were significantly closer to the statistical distribution of images in ImageNet (Fig. STD, unlike prior studies of IT [1]. The recordings also provided a coarse estimate of which image features drove neuronal activity (Fig. 2), which helped characterize DNN errors in explaining neural responses and explain why DNNs are becoming worse models of IT. We adopted the Brain-Score evaluation method to measure the prediction accuracy of 135 DNNs on recordings from medial (ML) and posterior (PI.) lateral IT in two different Monkeys. To summarize our findings:
In it was noted that while there was an overall correlation of 0.92 between DNN accuracy on ImageNet and predicting IT responses, the correlation weakened for state-of thethh-rrtttttte models.

---

We observed the same trade-off we found on Brain- org data (Fig. I in each of our recordings: DNNs are becoming less accurate predicting IT responses as they improve on ImageNet (Fig. 3)
DNNs trained on ImageNet learn different features than those encoded by IT (Fig. 4. and this mismatch is not helped by training on more internet data, using Transformers, relying on self-supervision,, or optimizing for adversarial robustness.
DNN architectures like
We successfully broke this trade-off by training DNNs with the neural harmonizer and aligning the representations they learn for object recognition with humans.
We further demonstrate that harmonized DNNs (hDNNs) are not only significantly better at explaining ITT responses than any other DNN available, they also generate interpretable hypotheses on the features driving I'T neuron responses.
(a)
(b)
Population receptive field
Fixation position
ML
Paradigm
Fixations per image
Actiiiy
Pre-trained Deep Neural
(d)
Inferotemporal medial lobe (ML) population activity maps
Portial least squares decoders to PL and ML
Figure IT recordings that reveal spatial maps of neuronal responses to complex natural images offer unprecedented insights into their feature selectivity (6). (a) Neurons in posterior (PL) and/or medial (ML) lateral IT two animals were localized using functional magnetic resonance imaging (fMRI) and neural responses to images were recorded using chronically implanted 32-channel multi-electrode arrays. (b) The monkeys were rapidly shown each image multiple times for 200ms each time (Monkey 1: n = 256, Monkey 2: n = 49). Images were positioned differently each time measure neural responses every part of the image. (c) This procedure yielded spatially resolved maps of neural activity for an image, revealing the relative importance of different features for the recorded neurons.
(d) DNNs were fit these recordings following the Brain-Score evaluation method [IT], in which partial least squares decoders were used to find the units in DNN that provided the best match for neuronal responses to images.
Methods
Neural recordings. We leveraged recordings of IT neuronal responses from two monkeys (6]. which were designed to reveal feature preferences of putative face-selective neurons. The recorded neurons also exhibited selectivity for non-face object features (6). Recordings were made using chronically implanted 32-channel multi-electrode arrays within the IMRI-defined middle lateral (ML) and posterior lateral (PL) face patches of one monkey (Monkey 1), and ML of another monkey (Monkey 2, Fig. Pa). The activating regions in these areas were mapped (1-39), then each image was shown the animals after they were cued to fixate a specific position space (Fig. (2)) The same images were shown multiple times while the monkeys fixated at red dot in the center of screen, which made possible to derive spatial activity maps neuronal responses (Fig P Fixation positions fell in 16 x 16 grid for Monkey and 7 x 7 grid for Monkey 2, and the average neuronal activity at each position was taken to generate spatial activity maps. A total of 14 images were shown to Monkey and different set of 14 images were shown to Monkey 2.

---

The recordings for Monkey resulted in responses from 32 neurons in ML and 31 neurons in PL. For Monkey 2. we obtained responses from 32 neurons (see Appendix [ for more details). Following [6]. we binned neuronal responses every 40ms of the recording, from 50ms to 250ms. Within each bin, we calculated the noise ceiling for every neuron, which represents the maximum correlation achievable between any two neurons within that time interval. Noise ceilings for each recording were similar to those reported for standard IT Brain-Score datasets (Appendix (B). In the main text, we present modeling results from the time bins that exhibited the highest average noise ceiling for each Monkey and recording site. Additional results from other time bins, which are consistent with these findings, can be found in Appendix E
DNNs. We investigated the neural fits of 135 different DNNs representing variety of approaches used in computer vision today: 62 convolutional neural networks trained on ImageNet 2443433444
(CNNs), 23 DNNs trained on other datasets in addition to ImageNet (which we refer to as " "DNN extra
data") 32 36 48) 25 vision transformers [49 54) (ViTs), 10 DNNs trained with self-supervisio
(55
56]. and is DNNs trained to be robust to noise or adversarial examples (57 58). Each model
was implemented in PyToorch with the TIMM toolbox btttps://g thub. com/ hugginggacc
pytorch- wat--iimeemmmeeee models) and pre-trained weights. Inference was executed on one NVIDIA TITAN
X GPU. Additional model details, including the licenses used for each, are detailed in Appendix C
Neural Harmonizer. It was recently found that DNN representations with human perception as they evolve and improve on ImageNet (7[59 60]. A partial solution to this problem is the neural harmonizer, a training routine that can be combined with any DNN to align its representations with humans without sacrificing performance. Here, we test the hypothesis that aligning DNNs with human visual representations can similarly significantly improve their accuracy in predicting neural responses to images.
becoming less aligned
Training DNNs with the neural harmonizer on ImageNet involves an additional loss to standard cross-entropy for object recognition. This extra loss forces a model's ' gradients to appear as similar as possible to feature importance maps derived from human behavioral experiments (see [7] for details). To implement this loss, let (.)() be a function that a multi-scale Gaussian pyramid of a human feature importance map @ to N, with i {1, N). During training, seek to minimize ΣN [P:(g(fo, æ))- - ( in order to align feature importance maps between humans and DNNs at every scale of the pyramid. Before they are compared, feature importance maps from humans and DNNs are normalized and rectified using z.). a function that transforms a feature importance map o from either source to have 0 mean and unit standard deviation. This procedure yields the complete neural harmonization loss:
N =A1 Σ P) -20
Hammonization
+ Locr(foo v, + A2 Σ of
(2)
Following the original neural harmonizer implementation 7 and training recipe, we trained six
DNNs on ImageNet and human feature importance maps from the ClickMe game [61]: VGG16 [25], LeViT (62), ResNet V2-N (63), wfffciinnttett__e........ b_ [22]. ConvNext 332. and MaxViT (64] Each model was trained with Tensorflow 2.0 on 8 V4 TPU cores with all of the images the ImageNet training set, and ClickMe human feature importance maps for the 200,000 images which were annotated. Images and feature importance maps were augmented with mixup (65]. random left-right flips, and random crops during training. Only the object recognition loss was computed for images without human feature importance maps. Model weights were optimized with stochastic gradient descent and momentum, batches of 512 images, label smoothing (66). learning rate of 0.3, and learning rate schedule that began with a 5-epoch warm-up period followed by a cosine decay over 90 epochs steps 30, 50 and 80.
Neural fitting. We evaluated the neural fit of each model by computing their fit separately for each IT recording, using the Brain-Score evaluation method (1T). This method involved fitting image-evoked activities from a layer of deep neural network (DNN) to the corresponding neural responses using the partial least squares regression algorithm from Scikit-Learn.

---

To implement the SeaieeScore evaluation method on the ppatiall--resovver recordings we investigate here, we first split each image shown each animal into an equal number of patches as there were fixation locations in an image. Each image patch captured the receptive field of recorded neurons, and in total, there were 4, 046 image patches for MI. and 4, 046 image patches for PI. in Monkey 1, and 1, 134 image patches for MI in Monkey 2. We measured DNN neuronal fits as the Spearman correlations between model predictions and true neural responses for each patch of an image held out of training divided by each neuron's s noise ceiling. We then stored the median Spearman correlation over neurons and repeated the training/testing procedure to get the mean correlation across all images viewed by monkey. Separate fitting procedures were performed for every layer of activities in model, and we report a DNN's Brain-Score as its best possible fit across layers.
Monkey PL
Monkey ML
Monkey
ML
DEN Tranalarees
0.6
Rotuuct
HIDD
Comtiee
Noural predecsity
0.5
Ettar Comfhett
MeviIT
VIGG
LeViT
EINE
eites DomyNoet
MacViT
Nounal lurricuue
DAN
VSG
LeViTe
MaXiTT
9.4
0.3
70% 75%
80%
85% 70%
75%
110%
85%
70%
75%
10%
115%
ImageNett Accuracy
Figure 3: DNNs trained on ImageNet face trade-off' between achieving high object recognition accuracy and predicting responses to natural images in IT. We measured the accuracy of 135 DNNs at predicting image-evoked responses from neurons in posterior lateral (PL. and medial- lateral (ML) areas of IT [6] by computing the neural predictivy of each model with the Brain- Score evaluation method (TT]. DNN neural predictivity is progressively worsening as models improve on ImageNet. This problem can be partially fixed by training DNNs with the neural harmonizer, which aligns their learned object representations with humans. Error bars denote 95% bootstrapped confidence intervals.
3 Results
Task Optimization is insufficient for reverse-engineeringg IT. Ever since it was found that DNNs optimized for object recognition produce accurate predictions of IT neural responses images, it was suggested that prediction accuracy would continue improve alongside DNN performance on ImageNet 12]. Is this the case with today's DNNs that rival exceed the performance of human object recognition?
To answer this question, we leverage recordings of neuronal responses to high-resolution natural images in medial (ML.) and posterior lateral (PI.) IT (see Methods and Fig. 2). These images fall within the same statistical distribution as ImageNet images (Fig. (ST), ensuring that our findings are not influenced by distributional shifts that are a well-known problem faced by computer vision models (67). Image-evoked neural responses were spatially mapped, enabling insights into the visual features driving the responses of IT neurons and DNNs. Moreover, while these regions were localized according to their selectivity to face stimuli, they were also noted to respond to non-face stimuli (6).
Across 135 different DNNs, representing the variety of approaches used today in computer vision, we found that DNNs pretrained on ImageNet have become progressively less accurate at predicting ML and PL responses in two separate Monkeys (Fig. 3 For instance, ConvNext tiny, which achieved 74.3% accuracy on ImageNet, is as accurate in predicting neural responses to images as the ResNetv2- 152xx, which reached 84.9% accuracy on ImageNet. Moreover, training routines that have been suggested to be more biologically plausible yield representations that are closer to biological vision, such as training with self-supervision [68] or for adversarial robustness [69]. made no difference. All DNNs trained internet data faced a pareto-front that bounded their accuracy in predicting IT responses as a function of their ImageNet accuracy.
DNNs need biologically-aligned training routines There are at least two potential reasons why ImageNetttrained DNNs face trade-off between object recognition and neural prediction accuracy: (i) DNN data diets and training routines work well for ImageNet but lead them to learn features
5

---

that are misaligned with biological vision 7 19 or (if) their architectures are poor match to visual cortex (20). To test this first possibility, we turned the neural harmonizer. Given prior work demonstrating that aligning DNNs with human perception using the neural harmonizer significantly improved their ability to predict human behavior, we reasoned that harmonization might similarly improve DNN explanations of neural data by forcing them to rely on human-like visual features. Indeed, we found that harmonized DNNs (hDNNs) were significantly more accurate at predicting image-evoked responses in ML (5) 5.30, p < 0.01) and PL (T) 6.11, p < 0.001) neurons of Monkey and ML (T(5) = 6.89, p < 0.001) neurons of Monkey 2 (reported as the average of T score from independent samples t-tests comparing the predietivity of each hDNN to the DNN with the highest neural- preddictttdcctty........ The success of hDNNs indicates that the architectural mismatch between DNNs and the visual cortex is far less of problem for modeling the visual cortex than the training routines and data diets that are being used today to achieve high accuracy on ImageNet. Behavior even from humans- is important constraint on DNNs that is needed to build more accurate models of primate visual cortex.
Harmonized MaxViT 1a 0.54
SSL ResNet18 p² 0.31
ConvNext tiny 1 0.39
ResNet v2 152x4 p 0.38
Robust ResNet v2 r= 0.27
Neural activity
MaxVIT P= 0.41
Images
0%
Activity 100%
Figure 4: DNNs optimized for object recognition on ImageNet rely on different featurre than those encoded by neurons in primate inferior temporal (IT) cortex.. The activity of PL IT neurons is plotted next to the predicted activity of a model representing each class of DNNs: harmonized DNNs (hDNNs), visual transformers, self-supervised DNNs, convolutional neural networks, DNNs trained on more data than ImageNet, and adversarially robust DNNs.
Spapeelly-mapped neural responses reveal feature mismatch between IT and DNNs The IT recordings used in the thousands of images were shown to multiple animals during passive viewing (14). In contrast, the recordings we rely on involve many fewer unique images, but neural responses for each image are spatially mapped. This spatial mapping reveals what types of features are driving neural responses and makes it easier to understand the successes and failures of DNNs in explaining these data (6).
benchmark all utilized the same paradigm, in which hundreds or
Although the neurons in ML and PL were located based on their selectivity to faces, the spatial maps of their responses revealed a much more complex response profile. IT neurons in both animals were strongly driven by faces, non-face objects, and contextual cues associated with faces (6). In contrast, the fest-fitting units of DNNs with state-of-the-ar accuracy on ImageNet, like the ResNetv2-152xd (Fig. 4) responded strongly to background features. This problem was shared by DNNs trained

---

on internet data but with routines that have been considered more biologically plausible, like self- supervised learning (e.g., SSL ResNet-18), or routines that yield perceptual robustness that is closer to humans, like adversarial training (e.g., Robust ResNetv2-50). hINNNs like the harmonized zed MaxViT, on the other hand, were, general, reliably predictive of what parts of images elicited the most activity from IT neurons.
Harmonized ResNet 50
0.3
Relative importance for explaining neural data
0.2-
0.1
0.0
3
Image features
ResNet 50
0.6
Relative importance for explaining neural data
data
0.4
0.2
0.0
3
Image features
Figure 5: Predictions of the visual features drive neuronal responses in PL of Monkey from harmonized ResNet-50 and standard ResNet-50. Image patches for each model depict what the important features are across all images shown to the animal. The relative importance of each feature for predicting the recordings is color-coded in the bar chart on the right.
hDNNs generate testable predictions for visual neuroscience The surprising effectiveness of DNNs pre-trained on object recognition for predicting IT responses to images was a significant finding in computational neuroscience, which raised the tantalizing possibility that these models could form the basis of neuroprosthetics, and reduce the need for animal experiments. However, persistent problem with DNNs since their inception is their lack of interpretability. meaning that using DNNs for systems identification effectively swaps one black box (the visual system) with another (a DNN) without getting the field any closer to understanding how vision works. Recent strides the field of explainable artificial intelligence (XAI) have begun to alleviate this problem. For instance, it now possible to reliably locate and characterize the features in datasets, like ImageNet, that drive patterns of model behavior using concept recursive activation factorization (CRAFT (70). When paired with accurate models of IT and recordings that reveal locations in images that elicit neuronal responses, CRAFT can generate testable predictions of what those features are.
We used CRAFT extract features from harmonized and regular ResNet-50s trained on ImageNet that explain their predictions of neural activity in PL of Monkey 1 (Fig. 5). To do this, we decomposed a DNNs predicted activities for image patches into the most important and distinct features using non-negative matrix factorization, then scaned over every single image patch to find those which most strongly explained each discovered feature. While the harmonizern ResNet-50 predicted that parts of faces, arms, and heads explained the majority of variance in IT, the less accurate and unharmonized

---

ResNet-50 predicted that the background, oriented-edges, and feature combining head, body, arm, and hand parts were most important for IT. Thus, hDNNs not only break the trade-off between ImageNet and neural prediction accuracy of ImageNet-trained DNNs, they can also generate testable hypotheses on the relative importance of different features for visual system neurons.
Related work
The scaling laws of deep learning The immense gains of DNNs in computer vision [3] and natural language processing [71] over recent years share a common theme: they have relied on unprecedented computational resources train models with billions of parameters on internet-scale datasets. These benefits of scale have been studied over the past several years across number of domains and often yield predictable improvements on standard internet benchmarks (7I, Surprisingly, scale has also caused models to exhibit human-like behavior in psychophysics experiments [13]57] 73 (7). In other words, many aspects of primate behavior are captured progressively better by DNNs as they scale-up and improve in accuracy on benchmarks like ImageNet.
Scale sufficient for modeling internet benchmarks, but not primate intelligence In parallel, there's a growing body of research suggesting that large-scale DNNs are becoming less aligned with human perception in multiple ways. For instance, the representations and visual decision- making behavior of DNNs are inconsistent with humans, and this problem has worsened as they have improved on ImageNet [7]. DNNs are also becoming less capable at predicting perceptual similarity ratings of humans, and they remain vulnerable [75] to so-called "adversarial attacks." Our work adds to this body of research, indicating that current scaling laws are incompatible with modeling primate behavior and poorly suited for explaining the neural computations that shape it.
Aligning DNNs with primate vision There have been a number of methods proposed for aligning DNNs with primate vision beyond the neural harmonizer that we leverage here (7). It was found that co-training DNNs for object recogmition and minimizing representational dissimilarity improved the adversarial robustness of a recurrent DNN (76). Others have shown that similar forms of representational alignment improve few-shot learning and predictions of human semantic judgments in DNNs 78). The successes of these behavioral and representational alignment methods highlight the real limitations of current DNN training routines and data diets for generating artificially intelligent systems that act like biological ones.
Biological learning routines There have been many efforts to align the object functions, learning rules, and data diets of DNNs more closely with biological systems. It was found that DNNs trained with suffesuperviiion on ImageNet instead of supervised recognition achieve similar accuracy in predicting VL, V4, and IT responses to images [68]. DNNs trained with self-supervised learning on head- mou-ted camera video from infants instead of ImageNet were approximately as accurate at predicting neural data as DNNs trained on Imagenet but needed far less data to do this [68). Others have found that adversarial training of DNNs yields models that are more accurate at predicting neural responses to images [79] and have similar tolerance to adversarial attacks as neurons in IT [69]
5 Discussion
revised approach for reverse engineering visual cortex Biological data is expensive to gather and often noisy. This makes the prospect of accurately modeling the responses of IT to complex stimuli especially daunting. DNNs optimized for object recognition represented a potential solution for this problem: pretraining on internet datasets alleviated training issues associated with small-scale neural data, and the architectures and training routines of accurate DNNs could offer insights into the circuits that underlie visual perception well as the developmental principles that shape those circuits. Our findings suggest that while DNNs still hold great potential for predicting image-evoked responses from IT neurons, new training routines and data diets are necessary for continued improvement.
The neural harmonizer is a partial solution to the problems that DNNs face in modeling primate IT. The success of hDNNs in breaking the pareto-front faced by 135 different DNNs indicates that significant a aspects of primate perception and the neural circuits that shape it cannot divined from internet data alone. We believe that the neural harmonizer and learning constraints provided by
8

---

seage-scale human behavior data is only short-term solution to this problem, and that if DNNs were able to learn about the visual world more like primates do, they would be even better at predicting neural data. In support of this goal, we release our code and data at https: /tt////erreeeeee lab. g thub io/neural, harmonizer/
Limitations. One limitation of our work is that we use the responses of neurons with face-selectivity to replicate and understand the trade-off between ImageNet accuracy and neural predictivity faced by DNNs. As faces (and even humans) are not a category in ImageNet, it is possible that this dataset [6] could bias our results in ways that are difficult to predic However, we find the same ImageNet accuracy and neural predictivity trade-off on this dataset (Fig- 3) as we did on the three recordings of object selective neurons hosted on the Brain-Score website (Fig. D. indicating that DNNs face similar issues in predicting each set of recordings. Moreover, the neural harmonizer was successful in breaking this trade-off, even though it involves ImageNet training using human feature importance maps for objects. Finally, it was noted in the original paper where our recordings came from that the neurons, while localized based on their sele-selectivity, responded to non-face stimuli as well [6]. In summary, our work reliably demonstrates that new paradigms models of IT, and spatially- -esoove resolved recordings such as those used in this work support this goal.
needed to advance DNNs as
Another limitation of our work is that while we found that hDNNs are significantly more predictive of IT neuron responses than any other DNN, they still explain only 50-60% the variance in neuronal activity. One straightforward way of doing better is by expanding the dataset of human feature importance maps we used for harmonization from annotations for approximately 200,000 images to the entire 1.2M ImageNet dataset.
Broader impacts. By building better models of primate IT, we are taking significant steps toward the reducing the reliance of visual neuroscience on animal models for experimentation, supporting the development of neuroprosthetic devices that resolve visual dysfunctions, and providing vision scientists with a richer understanding of how IT works. Our findings also highlight main limitation of the scaling laws that are guiding progress throughout artificial intelligence today: scale is not sufficient for explaining biological intelligence.
Acknowledgments and Disclosure of Funding
This work was supported by ONR (N00014-19-1-2000 NSF 1912290 and MARR-19548) I), DARPA (D19AC00015), NIH/NINDS (R21 NS 1 12743)), and the ANR-3IA Artificial and Natural Intelligence Toulouse Institute LANR-19-P13A-00000
Additional support provided by the Carney Institute for Brain Science and the Center for Computation and Visualization (CCV). We acknowledge the Cloud TPU hardware resources that Google made available via the TensorFlow Research Cloud (TFRC) program well computing hardware supported by NIH Office of the Director grant S100D025181.
References
[1] Cadicu, C.F., Hong. H., Yamins, D.L.K., Pinto, N., Ardila, D., Solomon, E.A., Majaj, N.J., DiCarlo, J.J.: Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS Comput. Biol. 10(12) (December 2014) e1003963
[2] Yamins, D.I.K., Hong, H., Cadicu, C.F., Solomon, F.A., Seihert, D., DiCarlo, JJ.: Performance-optmizzed hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sci. U. S. 111(23) (June 2014) 8619-8624
[3] Schrimpf, M., Kubilius, Lee, M.J. Ratan Murty, N.A., Ajemian, R., DiCarlo, JJJ:: Integrative benchmarking to advance neurally mechanistic models of human intelligence. Neuron 108(3) (November
2020) 413-423 423
[4] DiCarlo, J.J., Cox, D.D.: Untangling invariant object recognition. Trends Cogn. Sci. 11(8) (August 2007) 333-341
[5] DiCarlo, J.J., Zoccolan, D., Rust, N.C.: How does the brain solve visual object recognition? Neuron 73(3) (February 2012) 415-434
Note, however, that there are many animal and human faces in Imagenet images, giving DNNs ample opportunity to learn about them.

---

[6] Arcaro, M.J., Ponce. C., Livingstone, M.: The neurons that mistook a hat for face. Elife 9 (June 2020)
[7] Fel*, T.., Rodriguez*, L.F,, Linsley*, D., Serre, T.: Harmonizing the object recognition strategies of deep neural networks with humans. Adv. Neural Inf. Process. Syst. (2022)
[8] Krizhevsky, A., Sutskever. I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, Q.Q eds.: Advances in Neural Information
Processing Systems 25, Curran Associates, Inc. (2012) 1097-110S
[9] Deng, J., Dong. W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. (June 2009) 248 248-255
[10] Yamins, D.L.K., Hong. H., Cadieu, C.F., Solomon, E.A., Seibert, D., DiCarlo, JJJ.: Performance-optimee hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sci. U. S. A. 111(23) (June 2014) 8619-8624
[11] Schrimpf, M., Kubilius, J., Hong, H., Majaj, N.J., Rajalingham, R., Issa, E.B., Kar, K., Bashivan, P.,
Prescott- Roy, J., Geiger, F. Schmidt, K., Yamins, D.L.K., DiCarlo, J.J.: Brain-Score: Which artificial neural network for object recognition is most Brain-Like? (January 2020)
[12] Yamins, D.L.K., DiCarlo, JJJ Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. 19(3) (March 2016) 356-365
[13] Dehghani, M., Djolonga, J. Mustafa, B., Padlewski, P., Heek, J., Gilmer, J., Steiner, A., Caron, M., Geirhos, R.. Alabdulmohsin, I., Jenatton, R., Beyer, L., Tschannen, M., Arnab, A., Wang, X., Riquelme, C.. Minderer. M., Puigcerver, J.. Evci, U., Kumar, M., van Steenkiste, S., Elsayed, G.F. Mahendran, A.. Yu, F., Oliver, A., Huot, F., Bastings, J., Collier, M.P.. Gritsenko, A., Birodkar, V.. Vasconcelos, C., Tay, Y..
Mensink, T., Kolesnikov, A., Pavetić, F., Tran, D., Kipf, T., Lučié, M., Zhai, X., Keysers, D., Harmsen, Houlsby, N.: Scaling vision transformers to 22 billion parameters. (February 2023)
[14] Majaj, N.J., Hong, H., Solomon, E.A., DiCarlo, J.J.: Simple learned weighted sums of inferior temporal neuronal firing rates accurately predict human core object recognition performance. J. Neurosci. 35(39) (September 2015) 13402-13418
[15] Sanghavi, S., Apurva Ratan Murty, N., DiCarlo, JJJ.: SunghaviMurty2020 (November 2022)
[16] Sanghavi, S., Jozwik, K.M., DiCarlo, JJJ.: Sanghavilozwik22220 (November 2022)
[17] Linsley, D., Shiebler, D.,, Eberhardt, S., Serre, T.: Learning what and where attend. (2019)
[18] Linsley, D., Eherhardt, S., Sharma, T., Gupta, P., Serre, T.: What are visual features underlying human versus machine vision? In: 2017 IEEE International Conference on Computer Vision Workshops
(ICCVW). (October 2017) 2706-2714
[19] Ullman, S., Assif, L., Fetaya, E., Harari, D.: Atoms of recognition in human and computer vision. Proc. Natl. Acad. Sci. U. S. A. 113(10) (March 2016) 2744-2749
[20] Eberhardt, S., Cader, J.G., Serre, T.: How deep is the feature analysis underlying rapid visual categorization?
In Lee, D.D., Sugiyama, M., Luxbbrrg U.V., Guyon, I., Garnett, R., eds.: Advances in Neural Information Processing Systems 29, Curran Associates, Inc. (2016) 1100-1108
[21] Chen, X., Yan, B., Zhu, J.. Wang, D., Yang. X., Lu, H.: Transformer tracking. (March 2021)
[22] Tan, M., Le, Q.V.: EfficientNet: Rethinking model scaling for convolutional neural networks. (May 2019)
[23]
Radosavovic, I., Kosaraju, R.P., Girshick, R., He, K., Dollár, P:: Designing network design spaces. (March
2020)
[24] Howard, A., Sandler, M.. Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., Le, Q.V., Adam, H.: Searching for MobileNetV3. (May 2019)
[25] Simonyan, K., Zisserman, A.: Very deep convolutional networks for Large-Scale image recognition. (September 2014)
[26] Huang. L.. Zhao, X., Huang, K.: GOT-1Ok: large High-Diversity benchmark for generic object tracking in the wild. (October 2018)
[27] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. (December 2015) [28] Zhang. H., Wu, C., Zhang, Z., Zhu, Y., Lin, H., Zhang, Z., Sun, Y., He, T., Mueller, J., Manmatha, R., Li,
M., Smola, A.: ResNeSt: Split- Attention networks. (April 2020)
[29] Gao. S.H. Cheng. M.M., Zhao, K., Zhang, X.Y. Yang. M.H., Torr, P.: Res2Net: A new Multi-Scale backbone architecture. IEEE Trans. Pattern Anal. Mach. Intell. 43(2) (February 2021) 652-662
[30] Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., Houlsby, N.: Big transfer (BiT):
General visual representation learning. (December 2019)
[31] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: Inverted residuals and linear bottlenecks. (January 2018)
10

---

[32] Liu. Z.. Mao. H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie. S.: A ConvNet for the 2020s. (January 2022)
[33] Szegedy, C., loffe, S., Vanhoucke, V., Alemi, A.: nve Inception-Reshe and the impact of residual connections on learning. (February 2016)
[34] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. (December 20155)
[35]
Chollet, F.: Xception: Deep learning with depthwise separable convolutions. (October 2016)
[36]
Radford, A., Kim, J.W., Hallaey, Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P..
Clark, J., Krueger, G., Sutskever, L: Learning transferable visual models from natural language supervision. (February 2021)
[37] Xie, C., Tan, M., Gong, B., Wang, J. Yuille, A., Lc, Q.V.: Adversarial examples improve image recognition. (November 2019)
[38] Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. (November 2016)
[39] Brendel, W. Bethge, M.: Approximating CNNs with caaeooooloal-Feateeee models works surprisingly well on ImageNet. (March 2019)
[40] Mchta, D., Sotnychenko, Mueller, Xu, W., Elgharib, M., Fua, Scidel, H.P. Rhodin, Pons-Moll, G., Thcobalt, C.: XNect: real-time multi-person 3D motion capture with a single RGB camera. ACM Trans. Graph. 39(4) (July 2020) 82:1-82:17
[41] Chen, Y., Li, J., Xiao, H., Jin, X., Yan, S., Feng, J.: Dual path networks. (July 2017)
[42] Wang, C.Y., Liao, H. Y.M., Yeh, LH.. Wu, Y.H., Chen, P.Y., Hsieh, J.W.: CSPNet: new backbone that can enhance learning capability CNN. (November 2019)
[43] Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., Le, Q.V.: MnasNet: Platforn-Aware neural architecture search for mobile. (July 2018)
[44] Wu, B.. Dai, X., Zhang, P., Wang. Y., Sun, F,, Wu, Y.. Tian, Y., Vajda, P., Jia, Y.. Keutzer, K.: Fb- net: Hardware-aware efficient convnet design via differentiable neural architecture search. CoRR abs/1812.03443 (2018)
[45] Han, K., Wang, Y. Tian, Q. Guo, J., Xu, C., Xu, C.: Ghostnet: More features from cheap operations. CoRR abs/1911.11907 (2019)
[46] Yu, F., Wang, D., Darrell, T.: Deep layer aggregation. CoRR abs/1707.06484 (2017)
[47] Wang, Q.. Wu, B., Zhu, P., Li, P.. Zuo, W.. Hu, Q.: Eca-net: Efficient channel attention for deep convo- lutional neural networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition
(CVPR), Los Alamitos, CA, USA, IEEE Computer Society (jun 2020) 11531-11539
[48] Xie, Q., Luong, M.T., Hovy, E., Le, Q.V.: Self-training with noisy student improves ImageNet classification. (November 2019)
[49] d' Ascolii S., Touvron, H., Leavitt, M., Morcos, A., Biroli, G., Sagun, L.: ConViT: Improving vision transformers with soft convolutional inductive biases. (March 2021)
[50] Touvron, H., Cord, M., Douze. M.. Massa, F., Sablayrolles, A., Jégou, H.: Training data-efficient image
transformers & distillation through attention. (December 2020)
[51] Tolstikhin, I., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung. J., Steiner, A.,
Keysers, D., Uszkoreit, J., Lucic, M., Dosovitskiy, A.: MLP-Mixer. An all-MLP architecture for vision.
(May 2021)
[52] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. (October 2020)
[53] Steiner, A., Kolesnikov. A., Zhai, X., Wightman. R., Uszkoreit, J., Beyer, L.: How to train your VIT? data, augmentation, and regularization in vision transformers. (June 2021)
[54] Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: Proceedings of the International Conference Computer Vision (ICCV). (2021)
[55] Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. (February 2020)
[56] Zeki Yalniz, I., Jégou, H., Chen. K., Paluri, M., Mahajan, D.: Billion- scale semi semi-supervised learning for image classification. (May 2019)

---

[57] Geirhos, R., Rubisch, P.. Michaelis. C., Bethge, M.. Wichmann. F.A., Brendel, W.: ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. (November 2018)
[58] Salman, H., Ilyas, A., Engstrom, L., Kapoor, A., Madry, A:: Do adversarially robust ImageNet models transfer better? (July 2020)
[59] Kumar, M., Houlsby, N., Kalchbrenner, N., Cubuk, E.D.: Do better ImageNet classifiers assess perceptual similarity better? (September 2022)
[60] Bowers, J.S., Malhotra, G., Dujmovié, M., Montero, M.L., Tsvetkow, C., Biscione, V., Pucbla, G., Adolfi,
F. Hummel, J.E., Heaton, R.F., Evans, B.D., Mitchell, J., Blything, R.: Deep problems with neural network models of human vision. Behav. Brain Sci. (December 2022) 1-74
[61] Linsley, D., Shichler, D., Fberhardt, S., Serre, T:: Learning what and where attend with humans in the loop. In: International Conference on learning Representations. (2019)
[62] Graham, B., El-Nouby, A., Touvron, H., Stock, P., Joulin, A., Jégou, H., Douze, M.: Levit: vision transformer in convnet's clothing for faster inference (2021)
[63] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Computer Vision and Pattern Recognition. (2016)
[64] Tu. Z., Talebi. H., Zhang, H., Yang. F. Milanfar, P., Bovik, A., Li, Y:: Maxvit: Multi-axis vision transformer (2022)
[65] Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. (October 2017)
[66] Müller, R., Kornblith, S., Hinton, G.: When does label smoothing help? (June 2019)
[67] Linsley. D., Kim, J. Berson, D., Serre, T.: Robust neural circuit reconstruction from serial electron
microscopy with convolutional recurrent networks. (November 2018)
[68] Zhuang, C., Yan, S., Naycbi, A., Schrimpf, M., Frank, M.C., DiCarlo, J.J., Yamins, D.L.K.: Unsupervised
neural network models of the ventral visual stream. Proc. Nat. Acad. Sci. U. A. 118(3) (January 2021)
[69] Guo, C., Lee. M., Leclerc. G., Dapello, J. Rao, Y.. Madry, A., Dicarlo. J.: Adversarially trained neural representations are already robust biological neural representations. In Chaudhuri, K. Jegelka, S.. Song, I.,, Szepesvari, C., Niu, G., Sabato, S., eds.: Proceedings of the 39th International Conference on Machine Learning. Volume 162 of Proceedings of Machine Learning Research., PMIR (2022) 8072-8081
[70] Fel, T., Picard, A., Bethune, L., Boissin, T., Vigouroux, D., Colin, J., Cadène, R., Serre, T.: CRAFT: Concept recursive activation ForToiizttion for explainability. (November 2022)
[71] OpenAl: GPT-4 technical report. (March 2023)
[72] Kaplan, J. McCandlish, S., Henigham, T., Brown, T.B., Chess, B., Child, R., Gray, S., Radford, A., Wu,
Amodei, D.: Scaling laws for neural language models. (January 2020)
[73] Geirhos, R., Medina Temme, C.R., Rauber, J., Schütt, H.H., Bethge, M., Wichmann, F.A.: Generalisation in humans and deep neural networks. (August 2018)
[74] Geirhos, R., Narayanappa, K., Mitzkus, B., Thieringer, T., Bethge, M., Wichmann, F.A., Brendel, W.: Partial success in closing the gap between human and machine vision. (June 2021)
[75] Li, Y., Cheng, M., Hsieh, C.J., Lee, T.C.M.: A review adversarial attack and defense for classification methods. (November 2021)
[76] Dapello, J., Kar, K., Schrimpf, M., Geary, R.B., Ferguson. M., Cox. D.D., DiCarlo, JJJ.: Aligning model
and macaque inferior temporal cortex representations improves Model-to-Huun Humma behavioral alignment and adversarial robustness. (February 2023)
[77] Muttenthaler, L.. Dippel. J., Linhardt, L., Vandermeulen, R.A., Kornblith, S.: Human alignment of neural network representations. (November 2022)
[78] Sucholutsky, I., Griffiths, T.L.: Alignment with human representations supports robust few-shot learning. (January 2023)
[79] Berrios, W., Deza, A.: Joint rotational invariance and adversarial training of dual-stream transformer yields state of the art Brain-Score for area V4. (March 2022)
[80] Zamir, A., Sax, A., Shen, W., Guibas, L., Malik, Savarese, S.: Taskonomy: Disentangling task transfer learning. (April 2018)
[81] Mehrer, J., Spoerer, C.J., Jones, E.C., Kriegeskorte, N., Kietzmann, T.C.: An ecologically motivated image dataset for deep learning yields better models of human vision. Proc. Natl. Acad. Sei. U. S. A. 118(8) (February 2021)
12

---

A Datasets
Distribution shifts One possible explanation for the trade-off observed between DNN accuracy on ImageNet vs. predictions of neural data is that there was a distributional shift between the images used in the evaluations. To test this possibility, images from ImageNet and all images from the standard Brain-Score benchmark dataset for IT [14) and the images we used in our experiments [6]. For ImageNet, we used all images in the validation set that also had human annotations from ClickMe.
extracted activity vectors for large sample of
We measured distribution shifts by measuring the accuracy of linear classifier in discriminating between ImageNet and either the standard Brain-Score benchmark dataset for [14] or the images used in our experiments [6]. We began by taking equal-sized random samples of ImageNet image encodings and encodings from either the Brain-Score dataset (28 of 191 images) or our dataset (all 28 images). We then trained linear SVM using scikit-learn and performed leave-one-out cross-validation on either set of samples. Finally, we repeated this procedure 1,000 times, using random sample of ImageNet encodings each time, and measured the average accuracy for discriminating between ImageNet and either neural data image set. We found that images used in from
the standard Brain-Score benchmark for IT (14) are significantly further ImageNet than the images that we used in the main text (6) (7 (999) 98. 12, p < 0.001, Fig. S1).
Projection of stimuli features
Dicarlo Imagenet
Arcaro
Principal Component 2 (2%)
2
1-
-1
-2
-1
3 4
5
6
Principal Component 1 (2%)
Figure S1: Stimuli from [14), but not (6), are out-of-distribution from ImageNet. Penultimate- layer activities from a ResNetv2-50 trained on ImageNet were extracted for 1916 images from the validation set of ImageNet, 191 images from [14), and 28 images from 66 A principle components analysis decomposed those activities into a 2-Dimensional projection, which demonstrated a dis- tribution shift for stimuli from [14), but not [6]. Indeed, while stimuli from [14] could be reliably
discriminated from ImageNet (98.7 acc, T(999) = 98.12, p < 0.001), images from 6 were not
discriminable from ImageNet (56.5 acc, T(999) 98. 18,12,
13

---

0.70
0.65
0.60
Noise to Signal Ratio
0.55
0.50
0.45
0.40
0.35
George CIT
George PIT
0.30
Red CIT
50
100
150
200
250
Time (ms) after stimulus onset
Figure S2: Noise ceilings at every 40ms bin of activity from Oms 250ms. We binned every 40ms of neural responses following the original procedure for processing this data (6)
B Neural data analysis
We measured the maximum explainable variance, noise ceiling, in neural data recordings in every 40ms bin of activity from Oms 250ms. We binned every 40ms by following the original procedure for processing this data (6) Because images were repeatedly shown the same population receptive field, we computed each neuron's 's noise ceiling for image as the highest correlation between and another neuron for that image. We then focused our analyses in the main text on the time bin where each neurons in each animal/brain area achieved the highest average noise ceiling (Fig.
C Models
DNN Model Zoo We comprehensively evaluated the ability of a myriad of DNNs from the TIMM toolbox. These DNNs, available under the Apache 2.0 license, are intended for non-commercial research purposes. The complete list of DNNs we evaluated can be found in Table ST
D Neural Harmonizer Training
We relied on the original neural harmonizer recipe train and harmonize 14 DNNs for object recognition on the ImageNet [9] dataset. For each model, we searched for settings of the regularization terms A₁ and 'g, which control the relative importance of losses for object recognition and human feature alignment during training, that maximized accuracy on the ImageNet validation set. In total, we trained VGG16, ResNet550_v2. ViT ViT_b16, b16, EfficientNet. bo. ConvNext Tiny, and MaxViT Tiny (Table. S2.. We did not attempt to apply the neural harmonizer to models which relied on pretraining with datasets other than ImageNet because the ClickMe feature importance dataset we used only contained annotations subset of images in ImageNet. Thus, we expect that
14

---

Architecture
Model
Versions
VGG
4
ResNet
24
EfficientNet
8
ConvNext
10
MobileNet
6
Inception
7
DenseNet
4
RegNet
3
Xception
4
CNN
MixNet
4
DarkNet
2
NFNet
2
TinyNet
4
L.NNet
2
DLA
2
MnasNet
2
Resnext101
6
GhostNet
2
General ViT
leVTT
4
ViT
Max VIT
7
DeiT
4
Hybrid
CoAtNet
5
Table S1: Our DNN model zoo, taken from the TIMM library.
Model
Accuracy
Human Alignment
Note
VGG
69.3
61.5
==
ResNet 50
77.17
45.0
x=2
EfficientNet BO
77.51
52.3
=
ViT B16
75.7
72.6
5=5
ConvNext Tiny
75.9
73.2
==1
Max VIT Tiny
78.6
45.3
x=1
Table S2: DNNs trained with the neural harmonizer.
more work is needed for expanding ClickMe to larger-than-lmageNe scale data before these DNNs can be harmonized.
E Extended results
Cross validation One limitation of the evaluation approach used in the original Brain-Score work (TT) is that model selection is done on the same dataset as testing (i.e., there no partitioning of the data into separate training, validation, and test sets). Thus, it is possible that the results describe in the main text are byproduct of "double dipping." To test for this possibility, we repeated our experiments using separate training, validation, and test splits of the data. For each round of cross validation in this procedure, we held out one image for testing. We also held out half of the training data for selecting the model layer that best predicted neural responses (validation). Our results from this approach strongly correlated with the ones reported in our original submission and our overall conclusions remain the same (Fig. S3 Monkey ML p 0.98, p < 0.001: Monkey 2 ML p = 0.97. < 0.001); and Monkey 2 PL = 0.98, 0.001).
15

---

Monkey ML
Monkey 2 ML
0.6
ResNet50
Self supervised
Rabust
0.6
ResNe150
ConvNext
Conviext
CNN extra data Transformer extra data
EfficientNot
Neural predictivity
Neural predictivity
0.5
LeVIT
MaxViT
0.5
MaxVIT
VGG
EfficiontNet
VGG
LeViT
0.4
0.4
0.3
0.3
0.2
0.2
70%
75%
80%
85%
70%
75%
80%
85%
Figure S3: DNN neural prediction accuracy after evaluating on test set held out of training. Unlike the original approach introduced Brain-Score (TT], here we train, validate, and test models separate partitions of the dataset. DNNs trained and evaluated in this way follow the trends described in the main text.
Task
ML
PL
Autoencoding
0.005
-0.009
Class object
0.091
0.075
Curvature
0.019
-0.009
Denoising
-0,014
0.013
Edge occlusion
0.029
-0,0002
Edge texture
0.025
0.014
Egomotion
0.002
-0.018
Fixated pose
0.118
0.101
Jigsaw
0.099
0.097
Keypoints 2D
0.024
0.015
Keypoints
-0.057
-0.006
Nonfixated pose
0.164
0.086
Normal
0.065
0.056
Reshading
0.019
0,001
Room layout
0.131
0.087
Segment semantic
-0.011
0,029
Segment unsupervised 25D
0.041
0.008
Segment unsupervised 2D
0.052
0.043
Vanishing point
0.046
0.076
Table S3: Neural predictions for ML and PL in Monkey 1 using Taskonomy-trained DNNs.
Alternative objective functions and datasets To better understand how ImageNet per se affected DNNs' abilities to predict neural responses, evaluated
of them trained the Taskonomy dataset and task set [80] and also the Ecoset (81) naturalistic object categorization dataset and task. each task in the Taskonomy, and DNNs trained for classification on ecoset, and applied the standard Srain fitting procedure that we describe in our original submission Methods derive prediction accuracy scores for each model. These models were far less effective at explaining neural activity than any of the ImageNet trained models we provided in the main text (Tables [S3 and S4d To summarize, our human behavior aligned harmonized DNNs are more accurate at predicting neural responses than other DNN we tested, regardless of the dataset or task that was used to train them.
We took 19 DNNs pretrained
Model
Best time bin
Score
Alexnet
90
0.046
ResNet50
130
0.111
VGG16
130
0.086
Inception
210
0.109
Table S4: Neural predictions for ML in Monkey using Ecoset-trained DNNs.
16

---

Model
Neural
ImageNet
VGG baseline
0.523
67.23
VGG harmonized
0.524
67.369
ResNet50v2 baseline
0.501
76
ResNet50v2 harmonized
0.502
77.17
ConvNext baseline
0.519
74.3
ConvNext harmonized
0.524
75.8
Max VIT baseline
0.541
78.6
Max ViT harmonized
0.543
78.7
EfficientNetBO baseline
0.506
75,4
EfficientNetB0 harmonized
0.508
77.51
ViT baseline
0.472
74.9
VeV harmonized
0.473
75.4
VIT baseline
0.493
73.2
ViT harmonized
0.499
75.7
Table S5: Performance of select DNNs on inferotemporal (TT) recordings from Brain- score
org [14. For each DNN class, models with higher accuracy at predicting IT responses and ImageNet recognition are bolded.
Performance Brain-Score As point of comparison, tested a subset of our DNN on stan- dard inferotemporal (IT) recordings benchmark on BiBain-sccore org In each case, harmonization marginally improved performance over baseline (Table
Neural dynamics We also present modeling results on other time bins (with lower noise ceilings) than those considered in the main text (Figs. (S4 S6). For ML IT in monkey 1 and monkey we present 90ms-130mm 130ms, 900ms-17mms and 210ms-250ms. For PL. IT of Monkey 1, we present 130ms-170ms, 170ms-210ms, and 2100s-250ms, windows. In each case, found similar trade-off in these data as we did in those highlighted in the main text, as well as significant improvement for harmonized vs. unharmonized DNNs. This similarity extends the qualitative predictions made by each model on ML IT neurons (recall that we show the image-evoked activities of PL IT neurons in the main text, S
17

---

90-130
130-170 ms
210-250
0.7
Monkey 1(ML)
0.6
0.7
Monkey 2 (MU
0.6
0.5
0.4
0.3
0.2
ImageNet Accuracy
Transtormer
ConvNet
Robust
Bigger datasct
Solf supervised
Harmonized
Figure S4: Results using different time hins than in the main text for the same region (ML) for both monkeys. The pattern of results for each time-bin consistent with the main text, despite these data having lower noise ceilings.
130-170
170-210
210-250
0.7
Monkey 1(PL)
0.5
0.3
0.2
70
75
85
70
75
85
70
85
ImageNet Accuracy
Transformer
ConvNet
Robust
Bigger dataset
Self-supervised
Hormonized
Figure S5: Results using different time bins than in the main text for the IT PL in Monkey 1.. The pattern of results for each time-bin are consistent with the main text, despite these data having lower noise ceilings.
18

---

Harmonized MaxVIT /- 0.54
ConvNext tiny 0.39
Neural activity
MaxViT e- 0.41
ResNet18 1a 0.31
152x4 0.38
ResNot v2 p- 0.27
Images
0%
100%
Activity
Figure S6: DNNs optimized for object recognition ImageNet rely on different features than those encoded by neurons in primate inferior temporal (IT) cortex. The activity of ML IT neurons of Money 1 is plotted next the predicted activity of model representing each class of DNNs: harmonized DNNs (hDNNs), visual transformers, self-supervised DNNs, convolutional neural networks, DNNs trained on more data than ImageNet, and adversarially robust DNNs.
19


Reviews:
[{"review_id": "DhNQQTRDbY", "reviewer": "Reviewer_nyjr", "summary": "This paper investigates the general finding that modern deep neural network architectures have worse predictions of primate IT cortex, even though they perform better at object recognition. The authors investigate a set of IT recordings that incorporate spatially resolved population maps, and show that the best fitting units of modern DNNs have different spatial activity than the neuron they are predicting. The authors investigate a set of models that are trained with a “neural harmonizer” that helps align the model responses with human behavior and find that this training significantly improves the neural predictivity for many different architectures and also results in more similar spatial activation maps.  \n\nEdit after rebuttal period: I read the author’s rebuttal and resulting discussion about my concerns. Specifically, it was good to see the preliminary results showing that the harmonized models do not impair the predictivity for the public BrainScore IT neural dataset. These results better contextualize the claims by the authors. I would hope that the full analysis can be completed on the rest of the datasets but even just adding this preliminary data contextualizes their claims, and given this I updated my score.", "questions": "a) Can results from the brain-score benchmarks (Figure 1) be included for the neural harmonizer models that are showed in Figure 3? These datapoints would further support the claims, especially given that the neural data analyzed is primarily from face-patches. If the trend does not hold in those datasets how does this change the conclusions? \n\nb) Is cross validation performed by holding out an entire image or by holding out patches? This potentially matters for interpretation, as some of the image statistics may be very similar for neighboring patches of the image. \n\nc) I’m a little confused by the wording in lines 54-57. Is (i) about the training data and (ii) about the architecture? If so, could this be made explicit? \n\nd) In line 149 it is stated that separate fitting procedures were performed for every layer of activities and the best layer fit is reported – was this best layer determined using a separate “validation” set of data and the reported neural predictivity score is from “test” data? If not, there are some “double dipping” concerns for this type of analysis.", "limitations": "The authors include sections on limitations and broader impacts.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The authors analyze neural predictivity with a dataset that has not been extensively studied before, and analyze a large number of candidate neural networks varying in architecture and training procedure. \n* The authors investigate models trained on behavioral data in addition to ImageNet. Although these models were presented last year at NeurIPS, to my knowledge, their evaluations on neural data had not been reported, and the increase in predictivity with the behavioral data as a regularizer is impressive. I think makes the contribution novel enough even if the neural harmonizer methodology was presented before, as very few models have explicitly attempted to match precise human behavioral results as a way of increasing neural alignment. \n* Even though it is generally known that with modern models improved ImageNet performance does not lead to better brain predictions (further discussed in Weaknesses below), the full investigation and quantification of this in this paper is potentially beneficial to the field as a way to formally state that this trend is no longer the case.   \n* The analysis of spatially-mapped neural responses is novel, and the breakdown of which parts of the image are most important for explaining the neural data is interesting.", "weaknesses": "1) The authors present the lack of a correlation between neural predictivity and brain responses for modern models as a new finding, however this is a generally known phenomena mentioned in various papers as motivation for developing better metrics of similarity. Additionally, one can see this is the case by simply looking at the brain-score website. The authors include a footnote that in [11] this was mentioned, but it might be more appropriate to reframe the beginning of the abstract and the introduction so that this is less of a “finding by the authors” and more of an “observation by many in the field”. \n\n2) The authors state that “task optimization” is insufficient for reverse-engineering IT, however the only “task” that they consider is ImageNet. There are many other “tasks” that could be considered to train neural network models that are different from ImageNet (for instance including auxiliary tasks) and may lead to better predictions, so this seems like too strong of a claim. \n\n3) Similarly the claim that DNNs *need* biologically-aligned training routines seems too strong. This is one way of getting to improved predictivity, but it may not be the only way (and in fact, the predictions are still well below the noise ceiling).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The authors present the lack of a correlation between neural predictivity and brain responses for modern models as a new finding, however this is a generally known phenomena mentioned in various papers as motivation for developing better metrics of similarity.\n\nThank you for this comment. We addressed this point in our main rebuttal, but to quickly reiterate: we have taken your comment to heart and will be more precise about the novelty of our contribution, and clarify which findings were scraped and potentially an observation by many vs. truly novel results from our dataset and modeling efforts. \n\n> The authors state that “task optimization” is insufficient for reverse-engineering IT, however the only “task” that they consider is ImageNet. \n\nThis is a very important point, thank you! As discussed in the rebuttal, we expanded our analyses to include DNNs trained on other tasks and datasets (Taskonomy and Ecoset). None of these models are as accurate as ImageNet-trained or Harmonized models in predicting image-evoked responses in our recordings. With that said, we agree that there are many other potential tasks and datasets out there, and we will soften our claims as a result. In our revision, we will state that our harmonized DNNs are more accurate in predicting neural responses than any of the 135 ImageNet-trained, 19 Taskonomy-trained, or 4 Ecoset-trained DNNs that we tested.\n\n> Similarly the claim that DNNs need biologically-aligned training routines seems too strong. \n\nWe completely agree that biologically-aligned training routines may not be sufficient to reliably predict neural responses. However, our findings indicate that alignment with human behavior is the current best approach to achieving this goal. We will soften our claims to make it clear that harmonization and biologically-aligned training routines are not the only way (and maybe not the best way) to build better models of neural function.\n\n> Was [the] best layer determined using a separate “validation” set of data and the reported neural predictivity score is from “test” data?\n\nThank you for this question, which we addressed in our main rebuttal, and copy below. Please let us know if you have any further questions or concerns.\n\nOur neural data fitting procedure followed the original Brain-Score [1] procedure precisely, “The final neural predictivity score for the target brain region is computed as the mean across all train-test splits.” In other words, we (and the standard Brain-Score) did not use a held-out validation set to select the most predictive layer for each model. We agree with the reviewer, however, that this approach is potentially flawed. To test for this possibility we redid our analyses with train/val/test dataset partitions and selected a layer for each model as whichever one achieved the best performance on the validation set. For this procedure, we once again held one image out for testing, but in this version of the analysis, we also held out half of the training data for layer selection (validation). Our results from this approach strongly correlated with the ones reported in our original submission, and our overall conclusions remain the same (Monkey 1 ML Rebuttal Fig. A, $\\rho = 0.98$, $p < 0.001$, Monkey 2 ML Rebuttal Fig. B, $\\rho = 0.97$, $p < 0.001$, Monkey 1 PL, $\\rho = 0.98$, $p < 0.001$). In our revision, we will report results from these complete cross-validation analyses instead of the standard Brain-Score analyses.\n\n> Is cross validation performed by holding out an entire image or by holding out patches?\n\nWe always tested on held-out images. In our cross-validation procedure described above, our validation set consisted of held-out patches. As this is just for validation, we do not believe it confounds interpretation.\n\n> I’m a little confused by the wording in lines 54-57.\n\nIn those lines, (i) is about training data, objective functions, optimizers, learning algorithms (e.g., backpropagation or not?), and other choices that shape training that are unrelated to model architecture. (ii) Is strictly about model architecture, which can be induced with brain-like constraints, like normalizations and recurrence. We will rewrite these lines for clarity.\n\n> Can results from the brain-score benchmarks (Figure 1) be included for the neural harmonizer models that are shown in Figure 3? These data points would further support the claims, especially given that the neural data analyzed is primarily from face-patches. If the trend does not hold in those datasets how does this change the conclusions?\n\nAs discussed in the Contributions section of our submission, we focused on IT recordings from [1] because, “[These] experimental images were significantly closer to the statistical distribution of images in ImageNet (Fig. S1.), unlike [IT recordings used in Brain-Score][2].” The publicly available IT data on Brain-Score.org is of responses to images that are out-of-distribution of ImageNet (Fig. 1a and Fig. S1; Fig. 1b and Fig. 1c feature scraped results of DNN scores on private image datasets). Because of the well-known sensitivity of DNNs to distributional shifts [3], we believe it will be difficult to interpret the results of harmonized DNNs on these data. For this reason, we prefer to focus on our dataset from [1] in this manuscript. With that said, we included a link to our codebase on line 276 to make our data and models available to the wider community for additional analyses.\n\n[1] Arcaro, M.J., Ponce, C., Livingstone, M.: The neurons that mistook a hat for a face. Elife (June 2020)\n\n[2] Majaj, N.J., Hong, H., Solomon, E.A., DiCarlo, J.J.: Simple learned weighted sums of inferior temporal neuronal firing rates accurately predict human core object recognition performance. J. Neurosci. 35(39) (September 2015) 13402–13418\n\n[3] Geirhos, R., Medina Temme, C.R., Rauber, J., Schütt, H.H., Bethge, M., Wichmann, F.A.: Generalisation in humans and deep neural networks. \n"}, {"review_id": "qkYkwm1Ag8", "reviewer": "Reviewer_X5jQ", "summary": "This work summarizes the trend in DNN models of biological vision that networks that perform better on imagenet no longer necessarily provide better fits to neural data. It also shows neural-harmonized models do provide better fits to a data of mostly face-selective neurons. ", "questions": "line 58 has a typo\n\nThe abstract says \"Our results suggest that harmonized DNNs break the trade-off between ImageNet accuracy\nand neural prediction accuracy that assails current DNNs and offer a path to more accurate models of biological vision. \" I'm not sure what tradeoff is being referred to here. Is it that higher accuracy leads to worse neural prediction? That tradeoff is not broken by using the harmonizer. The harmonized models have better predictivity but not better Imagenet performance. \n\nFigure 4 shows feature relevance maps for example images, but without seeing this quantified across a large number of images, not much can be taken from this. The authors also make reference to models paying too much attention to background, etc, but again this is not measured in any objective way and seems to just be based on looking at these examples. Can the authors quantify these claims and show they hold across large populations of images? \n\nIn fig 5, why do many of the features have so little relative importance? Are these meant to be the top features? Also, why in the lower plot are there 4 colored bars but only three colored boxes? The axes are also different for these plots. Is there any significance to the fact that the relative importance for features from the harmonized model is half that of the resnet model? What else explains the responses and do those other features look like those from the unharmonized model? In total, I don't feel I can take any strong results away from this plot as-is.   ", "limitations": "The authors address limiations", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Neural harmonizing makes a substantial impact on neural fit.\n\nUnderstanding what makes DNNs stop fitting neural data is an important problem\n\nThe use of a more naturalistic color images is well-motivated", "weaknesses": "The framing of the paper (particularly the title) suggests that this paper is making a novel claim about DNN performance, when in fact this claim is based on a re-plot of BrainScore data and has been made before (in ref 11 and here: https://www.biorxiv.org/content/10.1101/688390v1). While it is good to have a replication of this finding, the substantial novel contribution of this paper is to show that neural harmonizing increases the match to neural data, so the title and framing should reflect that. \n\nThe measure of relevant features is too anecdotal as presented currently to support the claims (see below)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The framing of the paper (particularly the title) suggests that this paper is making a novel claim about DNN performance, when in fact this claim is based on a re-plot of BrainScore data and has been made before (in ref 11 and here: https://www.biorxiv.org/content/10.1101/688390v1). While it is good to have a replication of this finding, the substantial novel contribution of this paper is to show that neural harmonizing increases the match to neural data, so the title and framing should reflect that.\n\nThank you for the comment. We have addressed this in the main rebuttal. We are copying the response below for your convenience. Please let us know if you have any further questions, concerns, or points for clarification about this.\n\nThe correlation between the accuracy of DNNs on object recognition tasks like ImageNet and their ability to predict IT responses has been used as evidence for long-standing theories in vision science, such as core object recognition [1] (our title is an homage to this paper). As we try to emphasize in our submission, this correlation has not only weakened in recent years [2] for nonhuman primate electrophysiology, it has begun to progressively **worsen** as DNNs improve on ImageNet — especially over the last four years as DNNs have begun to dramatically increase in scale. We will clarify that it is this worsening-with-accuracy neural alignment of DNNs that we believe is truly novel (and alarming!). We will adopt the reviewers’ suggestions for clarifying that the results scraped from the Brain-Score website are potentially an “observation by many” (even if it is not yet a published finding) vs. the data we introduce and model is a “finding by the authors”.\n\nThe reviewers note that similar issues with performance optimization as we describe have been discussed in the human fMRI literature [3]. We will include a section on human fMRI in our revision as well as an elaboration on the important differences regarding inferences about neural computations based on electrophysiology (as we do) vs. fMRI (which at best offers a very slow and indirect readout of neural populations [4]).\n\n> I'm not sure what tradeoff is being referred to here... That tradeoff is not broken by using the harmonizer. The harmonized models have better predictivity but not better Imagenet performance.\n\nWe apologize for the lack of clarity. As non-harmonized models have improved in ImageNet accuracy, they have become progressively worse at predicting neural responses. In contrast, we see a mostly significant linear trend between the ImageNet accuracy and neural prediction accuracy of harmonized DNNs (Monkey 1, PL: $\\rho = 0.37$, $p < 0.01$, Monkey 2, PL: $\\rho = 0.23$, $p < 0.05$, Monkey 1, ML: $\\rho = 0.15$, $n.s.$). Moreover, nearly all harmonized DNNs outperform their nonharmonized baselines on ImageNet accuracy. We will clarify this in the manuscript.  \n\n> Figure 4 shows feature relevance maps for example images...\n\nWe apologize for the confusion. The results in Figure 3 of our submission depict the average correlation between spatially-resolved predictions of neural activities across images and the ground truth responses. In Figure 4 we plot maps of model predictions and true neural responses for several images. These are not saliency/gradient/feature attribution maps commonly used in explainable AI, but actual predictions and ground truth of the neural activity evoked by different regions of images. Our subjective interpretations of these spatial maps are that non-harmonized DNN responses are driven more by background features than harmonized models. We will clarify.\n\n> Questions about Figure 5.\n\nWe used CRAFT [5] to identify the primary features relied on by harmonized/unharmonized ResNet50s to predict IT image-evoked responses. To do this, CRAFT first computes non-negative matrix factorization to find features, and then total Sobol indices to measure the relative importance of each feature. Total Sobol indices provide a score representing the proportion of variance attributed to each individual concept and its interactions. These indices do not always sum to 1 because of how feature interactions are accounted for by total Sobol indices. For example, the importance of Feature 1 includes an interaction between Feature 1 and Feature 2, which is also captured within Feature 2. This leads to redundancy, which explains why the total Sobol indices can range between 0 and 1 without always summing to 1. We will revise our explanation of this procedure for clarity.\n\nWe also used CRAFT to find image patches that depict features of various importance. The bar plot colors in Fig. 4 denote relative importance features described by image patches on the left. (Note that we inadvertently highlighted 4 bars for unharmonized models; this should be 3.) CRAFT thus offers a principled and qualitative explanation of what features drive models' neural activity predictions.\n\n> line 58 has a typo\n\nWe fixed this. Thanks!\n\n\n[1] Yamins, D.L.K., Hong, H., Cadieu, C.F., Solomon, E.A., Seibert, D., DiCarlo, J.J.: Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sci. U. S. A. 111(23) 307 (June 2014) 8619–8624\n\n[2] Schrimpf, M., Kubilius, J., Hong, H., Majaj, N.J., Rajalingham, R., Issa, E.B., Kar, K., Bashivan, P., 327 Prescott-Roy, J., Geiger, F., Schmidt, K., Yamins, D.L.K., DiCarlo, J.J.: Brain-Score: Which artificial neural network for object recognition is most Brain-Like? (January 2020)\n\n[3] Jozwik, K., Schrimpf, M., Kanwisher, N., Dicarlo, James. 2019. To find better neural network models of human vision, find better neural network models of primate vision. BioRxiv.\n\n[4] Heeger, D., Ress, D. 2002. What does fMRI tell us about neuronal activity? Nature reviews neuroscience.\n\n[5] Fel, T., Picard, A., Bethune, L., Boissin, T., Vigouroux, D., Colin, J., Cadène, R., Serre, T. 2023. CRAFT:\nConcept recursive activation FacTorization for explainability. CVPR.\n"}, {"review_id": "BtEgMbWRSv", "reviewer": "Reviewer_GM1r", "summary": "This work address an important question in the construction of computational models of object recognition: the increase in performance of recent DNN models is not anymore accompanied (like in the past) by an increase in their ability to predict neural responses. This is a very relevant problem for the advancement of theoretical  neuroscience and its medical applications.\n\nThe Authors make extensive analysis of this detachment, using the Brain-Score metrics, and investigate two possible sources of this phenomenon, based respectively on the data and the architectures employed.\nCrucially, they employed a dataset with realistic stimuli and with spatial information that can be connected with the neural activity.\n\nFinally, the Authors introduce a technique called \"neural harmonizer\" that allows to partially align human and machine responses, and is suitable to generate interpretable hypothesis about the features that drive neural responses.", "questions": "Is it possible that, as a speculative question, the mismatch between the different features learned by DNNs and IT can be partially accounted for by the usage of backpropagation?", "limitations": "The limitations are critically discussed in the dedicated section, and the broader impacts are also addressed.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The paper is exceptionally well written and clear. It addresses a very relevant problem in computational neuroscience and its applications. It provides a convincing evidence that the neural harmonizer technique is effective in aligning human and DNNs responses, thus providing a base for important applications in prosthetics, reducing the need for animal experimentation. The originality comes from the combined action of using a new dataset, with spatial information, the usage of the neural harmonizer (that has already been used to align DNNs with human perceptual data) and extensive experimentation that gives strength to the conclusions.", "weaknesses": "I honestly can't find any that has not been already discussed in the Limitations section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Is it possible that, as a speculative question, the mismatch between the different features learned by DNNs and IT can be partially accounted for by the usage of backpropagation?\n\nFantastic question. As we wrote in our discussion, we believe that a wholesale revision of DNN training routines may be necessary for improving predictions of image-evoked neural responses in IT. In our paper we show that aligning DNNs with human behavior can partially achieve this goal, but we believe that great progress will be made in the future by identifying principles that could yield better predictions of neural activity without the need to co-train on human behavior. In our discussion we discuss opportunities for designing better datasets and objective functions for doing this, but more biologically-plausible learning algorithms are another approach that we will mention in our revision."}, {"review_id": "h9hP7qmJdt", "reviewer": "Reviewer_itup", "summary": "The paper discusses the observation that improved imagenet classification performance is no long correlated with neuron prediction performance in macaque IT. \n\nTo validate this claim, the authors perform experiments where an image was moved across the visual field while the monkey maintained fixation. This process allowed the authors to examine the relative importance of the spatially distributed features in an image. \n\nThe authors further compared against a set of architectures from pytorch timm, including those trained on imagenet, self-supervised models, and those trained on internet-scale datasets. Both CNNs and ViT-like models were tested. \n\nThe authors find that aligning the gradients of a model with human data improved the prediction performance of DNNs for IT neurons.\n\nIn the supplementary, the authors show that their stimuli is better aligned to imagenet, and thus their results cannot be explained by the dataset misalignment.", "questions": "See weaknesses section for other questions.\n\nFor figure 4, did you actually present upside down images to the monkeys? Or were they only used for the training of the neural harmonizer?", "limitations": "The authors largely address the limitations of their paper. However the authors could potentially emphasize that their approach applies to data collected from electrophysiology from macaque monkeys. Recent work on human fMRI data suggests that contrastive optimized DNNs (CLIP) are better encoding models of the human visual cortex than fully supervised models (imagenet).\n\n[1] Conwell, Colin, et al. \"What can 5.17 billion regression fits tell us about artificial models of the human visual system?.\" SVRHM 2021 Workshop@ NeurIPS. 2021.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper is interesting and timely.\n \nWith the widespread availability of datasets with hundreds of millions to billions of images (YFCC, Conceptual Captions, Datacomp-1B, LAION-2B), models have repeatedly achieved higher performance on zero-shot imagenet. Researchers building encoding models have increasingly embraced these models as backbones.\n\nThe paper shows that imagenet performance for a variety of different models is no longer strictly correlated with brain predictive performance. \n\nThe idea of shifting images while maintaining eye fixation is an interesting way to find biological importance of different features.", "weaknesses": "* On DNN fitting\n  * The paper provides few details on how exactly the images were presented to DNNs. For the monkey presentation, fixation was maintained and the images were shifted. Line 141 onwards provides an explanation, however it is very unclear how exactly you perform this step.\n* On the models used\n  * I find the division of models into CNN/Transformer/Robust/Self-supervised/DNN extra data to be confusing. For example CLIP models would presumably fall under DNN extra data. However CLIP models have both CNN and ViT variants, how are models classified? \n  * I suggest the authors use a mutually exclusive division by architecture (CNN/ViT/MLP-mixer etc), loss or dataset (supervised ImageNet, supervised large dataset e.g. CLIP on the billion+ images, self-supervised using patch/rotation/other prefix tasks)\n  * This applies to Figure 1 and Figure 3, as well as discussion within the text. \n* On Neural harmonizer\n  * The section starting from Line 116 provide very sparse details for how they actually train their neural harmonizer\n  * What is $g$? Is this the gradient operator? What is the gradient with respect to? Is it to the mean of the neurons in IT? \n  * Are you performing a second-order gradient optimization?\n\n* On the feature importance figures\n  * How exactly are you plotting feature importance for DNNs? What approach are you using? GradCAM? In text you mention CRAFT in line 215, however it is not clear if this is the approach you actually use for the visualizations\n  * Presumably you don't use CRAFT for the neural harmonizer? If so, please clarify what you use for the visualization and the neural harmonizer.\n\n* On imagenet accuracy\n  * Was imagenet accuracy top-1 accuracy? Was this used dataset ImageNet-1k? ImageNet-21k? ImageNetv2? \n  * How was accuracy for contrastive models computed? Were they zero-shot probes in the style of \"a photo of a x\" as in the CLIP paper?\n\n* Minor typos and grammatical errors\n  * Line 58 -> \"tjat\"\n  * Line 119 -> \"let $P$ be a function that a multi-scale Gaussian pyramid of a human feature importance map ...\", this line is super confusing.\n  * Supplementary code \"realising\" -> \"releasing\"\n\n* On related work\n  * I recommend the authors discuss at least one of the following papers:\n  * [1] which show how most DNNs trained on visual-language contrastive losses cannot perform compositional reasoning\n  * [2] which discusses how recognition and retrieval tasks, and the dataset used to train these tasks lead to compositional features not being emphasized in DNNs\n  * [3] which discusses how DNN initializations yield representational gaps between vision and concepts\n\n\nThe paper would be strengthened with additional details and clarifications.\n\n[1] Thrush, Tristan, et al. \"Winoground: Probing vision and language models for visio-linguistic compositionality.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n\n[2] Yuksekgonul, Mert, et al. \"When and Why Vision-Language Models Behave like Bags-Of-Words, and What to Do About It?.\" The Eleventh International Conference on Learning Representations. 2022.\n\n[3] Liang, Victor Weixin, et al. \"Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning.\" Advances in Neural Information Processing Systems 35 (2022): 17612-17625.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The paper provides few details on how exactly the images were presented to DNNs. For the monkey presentation, fixation was maintained and the images were shifted. Line 141 onwards provides an explanation, however it is very unclear how exactly you perform this step.\n\nWe apologize for the lack of clarity. We presented every image shown to a monkey to a DNN and then extracted the feature map at a given layer. Then, a feature map patch was extracted at each of the monkey’s receptive field locations. These feature map patches were used to determine neural predictivity. We will revise our description in the main text with these details. We also included a link to our code with an implementation of this procedure on line 276.\n\n> I find the division of models into CNN/Transformer/Robust/Self-supervised/DNN extra data to be confusing. For example CLIP models would presumably fall under DNN extra data. However CLIP models have both CNN and ViT variants, how are models classified?\n\nThank you for the suggestion! We have split the DNN extra data category into CNN extra data and Transformer extra data (Rebuttal Figs A and B). CLIP-ResNet falls into the former group and CLIP-Transformer falls into the latter group. We hesitate adding additional divisions as it will clutter the figures, but please take a look at our revised figures and let us know what you think.\n\n> The section starting from Line 116 provide very sparse details for how they actually train their neural harmonizer. What is g? What is the gradient with respect to? Are you performing a second-order gradient optimization?\n\nWe apologize for the oversight! Here, g(.) refers to any attribution method (heatmap method), and indeed, we utilize gradients. The loss is first differentiated with respect to the input, resulting in df(x)/dx (yielding a heatmap). Next, we compute the derivative of the difference between this heatmap and the desired heatmap with respect to our model's weights. This involves a mixed partial derivative process. We'd like to clarify that even with ReLU activations, the gradient of the loss is not zero (a trivial case to comprehend is calculating the closed-form of the gradient of the loss for a dense ReLU network). Therefore, the optimization occurs just as with any other training process. We will incorporate this explanation into our revision.\n\n> How exactly are you plotting feature importance for DNNs? What approach are you using? GradCAM? In text you mention CRAFT in line 215, however it is not clear if this is the approach you actually use for the visualizations\n\nWe plotted predicted and actual neural responses at every receptive field location in an image (see Figure 4 caption). These are not attribution maps. We will clarify this in the main text. We also applied CRAFT to harmonized and non-harmonized ResNet50 models. CRAFT provides a complementary insight: what are the actual features (as opposed to an activity heatmap) that each model thinks are driving IT responses? CRAFT shows that harmonized models predict that face features are more important for explaining neural responses, whereas nonharmonized models are less selective for face features.\n\n> Was imagenet accuracy top-1 accuracy? \n\nWe plotted DNN top-1 accuracy on ILSVRC12/ImageNet-1k. Some of the models in our zoo were trained on data beyond just ImageNet (CNN/Transformer extra data, original submission Figs. 3/4 and rebuttal figs A/B). We will clarify in the revision.\n\n> How was accuracy for contrastive models computed? Were they zero-shot probes in the style of \"a photo of a x\" as in the CLIP paper?\n\nAs noted in our Methods section, nearly all models in our zoo were taken from TIMM, and the rest are from the Neural Harmonizer. The CLIP models were taken from TIMM, where they were finetuned on ImageNet-1k.\n\n> Minor typos and grammatical errors\n\nThank you, these were fixed.\n\n>On related work, I recommend the authors discuss at least one of the following papers: [1] which show how most DNNs trained on visual-language contrastive losses cannot perform compositional reasoning, [2] which discusses how recognition and retrieval tasks, and the dataset used to train these tasks lead to compositional features not being emphasized in DNNs, [3] which discusses how DNN initializations yield representational gaps between vision and concepts. \n\nThank you for these references. We will include them in our discussion.\n\n> For figure 4, did you actually present upside down images to the monkeys?\n\nYes the monkeys actually saw the images as presented in Figure 4. We will clarify this point in an expanded methods section in the appendix of our submission.\n\n> The authors largely address the limitations of their paper. However the authors could potentially emphasize that their approach applies to data collected from electrophysiology from macaque monkeys. Recent work on human fMRI data suggests that contrastive optimized DNNs (CLIP) are better encoding models of the human visual cortex than fully supervised models (imagenet).\n\nThank you for this comment. As discussed in our main rebuttal, we will add a discussion of how model-based work on fMRI relates to our findings in our manuscript.\n"}]

Author rebuttal:
# Response to all reviewers

We thank the reviewers for their extensive feedback. We are confident that we have addressed their main critiques, which we summarize below along with our responses (the relevant reviewers are in parentheses):

**(nyjr) How was the best layer for each model selected?**
> Our neural data fitting procedure followed the original Brain-Score [1] procedure precisely, “The final neural predictivity score for the target brain region is computed as the mean across all train-test splits.” In other words, we (and the standard Brain-Score) did not use a held-out validation set to select the most predictive layer for each model. We agree with the reviewer, however, that this approach is potentially flawed. To test for this possibility we redid our analyses with train/val/test dataset partitions and selected a layer for each model as whichever one achieved the best performance on the validation set. For this procedure, we once again held one image out for testing, but in this version of the analysis, we also held out half of the training data for layer selection (validation). Our results from this approach strongly correlated with the ones reported in our original submission, and our overall conclusions remain the same (Monkey 1 ML Rebuttal Fig. A, $\rho = 0.98$, $p < 0.001$, Monkey 2 ML Rebuttal Fig. B, $\rho = 0.97$, $p < 0.001$, Monkey 1 PL, $\rho = 0.98$, $p < 0.001$). We will report both versions of results in our revision.

**(nyjr, itup) The only task considered is ImageNet. Would other datasets and tasks yield different or better predictions of neural responses?**
> We have added results on the ability of DNNs trained on the Taskonomy dataset and task set [2] and also the Ecoset [3] naturalistic object categorization dataset and task. We took 19 DNNs pretrained on each task in the Taskonomy, and 4 DNNs trained for classification on ecoset, and applied the standard Brain-Score fitting procedure that we describe in our original submission Methods to derive prediction accuracy scores for each model. These models were far less effective at explaining neural activity than any of the ImageNet-trained models we provided in the main text (Rebuttal tables C and D). **To summarize, our human behavior-aligned “Harmonized” DNNs are more accurate at predicting neural responses than any other DNN dataset or task we tested.**

**(nyjr, X5jQ) How novel is the top-line finding that ImageNet optimization is no longer effective for systems identification of primate IT?**
> The correlation between the accuracy of DNNs on object recognition tasks like ImageNet and their ability to predict IT responses has been used as evidence for long-standing theories in vision science, such as core object recognition [4] (our title is an homage to this paper). As we try to emphasize in our submission, this correlation has not only weakened in recent years [5] for nonhuman primate electrophysiology, it has begun to progressively **worsen** as DNNs improve on ImageNet. We will clarify that it is this worsening-with-accuracy neural alignment of DNNs that we believe is truly novel (and alarming!). We will adopt the reviewers’ suggestions for clarifying that the results scraped from the Brain-Score website are potentially an “observation by many” (even if it is not yet a published finding) vs. the data we introduce and model, which is a “finding by the authors”.

> The reviewers note that similar issues with performance optimization as we describe have been appeared in the human fMRI literature [6, 7], although those papers describe older network architectures from before 2022. We will include a section on human fMRI in our revision as well as an elaboration on the important differences regarding inferences about neural computations based on electrophysiology (as we do) vs. fMRI (which at best offers a very slow and indirect readout of neural populations [8]).

We have provided detailed comments to each of the reviewers’ responses below. We hope you agree that our manuscript has improved through your feedback and that our findings will have a significant impact on Computational Neuroscience.

[1] Schrimpf, M., Kubilius, J., Hong, H., Majaj, N.J., Rajalingham, R., Issa, E.B., Kar, K., Bashivan, P., 327 Prescott-Roy, J., Geiger, F., Schmidt, K., Yamins, D.L.K., DiCarlo, J.J.: Brain-Score: Which artificial 328 neural network for object recognition is most Brain-Like? (January 2020)

[2] Zamir, A., Sax, A., Shen, W., Guibas, L., Malik, J., Savarese, S. 2018. Taskonomy: Disentangling task transfer learning. IEEE Computer vision and pattern recognition conference (CVPR).

[3] Mehrer, J., Spoerer, C., Jobes, E., Kietzmann, T. 2021. An ecologically motivated image dataset for deep learning yields better models of human vision. Proc. Natl. Acad. Sci. U. S. A. 118(8).

[4] Yamins, D.L.K., Hong, H., Cadieu, C.F., Solomon, E.A., Seibert, D., DiCarlo, J.J.: Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sci. U. S. A. 111(23) 307 (June 2014) 8619–8624

[5] Schrimpf, M., Kubilius, J., Hong, H., Majaj, N.J., Rajalingham, R., Issa, E.B., Kar, K., Bashivan, P., 327 Prescott-Roy, J., Geiger, F., Schmidt, K., Yamins, D.L.K., DiCarlo, J.J.: Brain-Score: Which artificial neural network for object recognition is most Brain-Like? (January 2020)

[6] Jozwik, K., Schrimpf, M., Kanwisher, N., Dicarlo, James. 2019. To find better neural network models of human vision, find better neural network models of primate vision. BioRxiv.

[7] Conwell, Colin, et al. "What can 5.17 billion regression fits tell us about artificial models of the human visual system?." SVRHM 2021 Workshop@ NeurIPS. 2021.

[8] Heeger, D., Ress, D. 2002. What does fMRI tell us about neuronal activity? Nature reviews neuroscience.


Meta-review: The paper investigates the finding that more recent strong image recognition models have become worse models of monkey IT cortex and proposes a "neural harmonizer" to align model and brain responses. The reviewers raised some concerns about the exact contributions and potential controls with additional datasets, which the authors addressed during the discussion period. As a result of this interaction, the reviewers agree that the paper makes a valuable contribution and should be accepted.

As a note to the authors: To contextualize your results, you are expected to include the results on the BrainScore dataset which came out of the discussion with reviewer nyjr.
Decision: Accept (poster)