Harmonizing the object recognition strategies of deep neural networks with humans
Authors: Thomas FEL
Ivan F Rodriguez Rodriguez
Drew Linsley
Thomas Serre
Venue: NeurIPS 2022 Accept (2022)
Keywords: Cognitive science
human vision
explainable AI
models of biological vision
AI alignment
scaling laws

Abstract: The many successes of deep neural networks (DNNs) over the past decade have largely been driven by computational scale rather than insights from biological intelligence. Here, we explore if these trends have also carried concomitant improvements in explaining the visual strategies humans rely on for object recognition. We do this by comparing two related but distinct properties of visual strategies in humans and DNNs: where they believe important visual features are in images and how they use those features to categorize objects. Across 84 different DNNs trained on ImageNet and three independent datasets measuring the where and the how of human visual strategies for object recognition on those images, we find a systematic trade-off between DNN categorization accuracy and alignment with human visual strategies for object recognition. \textit{State-of-the-art DNNs are progressively becoming less aligned with humans as their accuracy improves}. We rectify this growing issue with our neural harmonizer: a general-purpose training routine that both aligns DNN and human visual strategies and improves categorization accuracy. Our work represents the first demonstration that the scaling laws that are guiding the design of DNNs today have also produced worse models of human vision. We release our code and data at https://serre-lab.github.io/Harmonization to help the field build more human-like DNNs.


Harmonizing the object recognition strategies of deep neural networks with humans
Thomas Fel-!.2, Ivan Felipe"), Drew Linsley* Thomas Serre 1.2.3
{thomas_for lellliian_ellllll welliperrodriueee drew ses__inssey))bbrn edu
Abstract
The many successes of deep neural networks (DNNs) over the past decade have largely been driven by computational scale rather than insights from biological intelligence. Here, we explore if these trends have also carried concomitant im- provements explaining the visual strategies humans rely on for object recognition. We do this by comparing two related but distinct properties of visual strategies in humans and DNNs: where they believe important visual features how they use those features to categorize objects. Across 84 different DNNs trained on ImageNet and three independent datasets measuring the where and the how of human visual strategies for object recognition on those images, we find a systematic trade-off between DNN categorization accuracy and alignment with human visual strategies for object recognition. State-of-the-de thtt-o DNNs are progressively becoming less aligned with humans as their accuracy improves. We rectify this growing issue with our neural harmonizer:
images and
training routine that both aligns DNN and human visual strategies and improves categorization accuracy. Our work represents the first demonstration that the scaling laws [13 that are guiding the design of DNNs today have also produced worse models of human vision. We release code and data httpps:: pestps://sereee //tp//eeeeeeabbbbeeee lab.gittubbb htt.aa/..tt......a.ooaooiiiiooao io/Harronizztion to help the field build more human-like DNNs.
Introduction
Rich Sutton stated (4) that the bitter lesson "from 70 years of Al research is that general methods that leverage computation are ultimately the most effective, and by a large margin." " Deep learning has been the standard approach to object categorization problems ever since the paradigm shifting success of AlexNet (5) on the ImageNet (6) benchmark decade ago. As deep neural network (DNN)
performance has continued to improve in the intervening years, Sutton's lesson has become more fitting than ever, with recent networks rivaling and likely outperforming humans on the benchmark (7) through brute-force computational scale: increasing the number of network parameters and number of images used for training orders-of-man maggnniuud beyond AlexNet (133). While the successes of so-called "scaling laws" are undeniable, this singular focus performance in the field has side-stepped an equally important question that will govern the utility of object recognition models for the brain sciences and industry applications alike: are the visual strategies learned by DNNS aligned with those used by humans?
The visual strategies that mediate object recognition in humans can be decomposed into two related but distinct processes: identifying where the important features for object recognition are a scene, and determining how to integrate the selected features into categorical decision [899 It has been
"Theee authors contributed equally. 'Department of Cognitive, Linguistic, Psychological Sciences, Brown University, Providence, RI 2 Artificial and Natural Intelligence Toulouse Institute (ANITI,, Toulouse, France *Camey Institute for Brain Science, Brown University, Providence, RI
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

known for nearly a century 13] that different humans attend to similar locations when asked find and recognize objects. After selecting these important features, human observers are also consistent in how they use those features to categorize objects the inclusion of a few pixels in an image can be the difference between recognizing an object or not (9 14].
Has the past decade of DNN development produced any models that are aligned with these human visual strategies for object recognition? Such a model could transform cognitive science by support- ing a better mechanistic understanding of how vision works. More human-like models of object recognition would also resolve the problems with predictablity and interpretablity of DNNs [I3 18). and control their alarming tendency to rely on "shortcuts" and dataset biases to perform well on tasks [19]. In this work, we perform the first large-scale and systematic comparison of the visual strategies of DNNs and humans for object recognition on ImageNet.
Contributions. In order to compare human and DNN visual strategies, we first turn to the human feature importance maps collected by Linsley al. (T) Their datasets, ClickMe and Clicktionary, contain maps of nearly 200,000 unique images in ImageNet that highlight the visual features humans believe are important for recognizing them. These datasets amount to a reverse inference on where important visual features are in ImageNet images (Fig. D We complement these datasets with new psychophysics experiments that directly test how important visual features are used for object recognition (Fig. D. As DNN performance has increased on ImageNet, their alignment with human visual strategies captured these datasets has worsened. This trade-off is found over 84 different DNNs representing all popular model classes from those trained for adversarial robustness to those pushing the scaling laws in network capacity and training data. To summarize our findings:
The trade-off between DNN object recognition accuracy and alignment with human visual strategies replicates across three unique datasets: ClickMe (20). Clicktionary (T.] and our psychophysics experiments.
We shift this trade-off with our neural harmonizer, novel drop-in module for co-training any DNN to align with human visual strategies while also achieving high task accuracy. Harmonized DNNs learn visual strategies that significantly more aligned with humans than any other DNN we tested.
We release our data and code at bhttps://serres lab. gi thub. io/Harronizationn/ to help the field tackle the growing misalignment between DNNs and humans.
2 Related work
Do DNNs explain human visual per- ception? Despite the continued suc- cess of DNNs on computer vision benchmarks, there conflicting ac- counts on their ability to explain hu- man vision. On the one hand, there is evidence that DNNs are improving as models of human visual perception on challenging tasks, such as recognizing objects obscured by noise [23] On the other hand, there is also evidence that DNNs struggle to explain percep- tual phenomena in human vision like contextual illusions 244]. perceptual grouping 19 25 26 and categorical prototypes 27J. Others have found
Where are the important features?
How the discovered features used?
Accuraoy topk' Human festines
Human
DNN
p ((Human, DNN)
Ao0
Accpen lle
Figure 1: Visual strategies of object recognition. We inves- tigate the alignment of human and DNN visual strategies in object categorization. We decompose human visual strate- gies into descriptions of where important features are (0 22 and how those features are integrated into visual decisions.
differences between human attention data and DNN models of visual attention 20 28 Moreover, DNNs have stopped improving as models recent years. While the original theory was that model explanations of object-evoked neural activity patterns improved alongside model categorization accuracy [29]. recent large-scale DNNs are worse at explaining neural data than older ones with lower ImageNet accuracy (30).
ventral visual system humans and primates over

---

What are the visual strategies underlying human object recognition? Ever since its inception, a goal of vision science has been to characterize the neural processes supporting object recognition in humans. It has been discovered that object recognition can be decomposed into different processing stages that emerge over time [13))36) where the earliest stage associated with processing through feedforward connections in the visual system, and the later stage is associated with processing through feedback connections. Since the DNNs used today mostly rely on feedforward connections, it is likely that they are better models for that rapid feedforward phase of processing than the subsequent feedback phase (3( To maximize the likelihood that the visual strategies learned by DNNs align with those used by humans, our experiments focus on the visual strategies of rapid feedforward object recognition in humans.
Most closely related to our work, are studies of "top-down" image saliency and where category diagnostic visual features are in images. These studies typically involve asking participants to search for an object in an image, or find visual features that are diagnostic for an object's category or identity [T0 20 22 38). In our work, we complement these descriptions of where important features in images with psychophysies testing how those features are used to categorize objects.
Comparing visual strategies of humans and machines. As methods in explainable artificial intelligence have developed over the past decade, they have opened up opportunities for comparing the visual regions selected by humans and DNNs when solving tasks. Many of these comparisons have focused on human image saliency measurements captured by eye tracking or mouse clicks during passive or active viewing 220 [2 |2| [2]) Others have compared categorical representation distances [30] or combined those distances with measures of human attention [28[44). The most direct comparisons between human and DNN visual strategies involved analyzing the minimal image patches needed to recognize objects
46). However, these studies were limited and compared humans with older DNNs on tens of images. To the best of our knowledge, the largest-scale evaluation of human and DNN visual strategies relied on the ClickMe dataset to compare visual regions preferred by humans and attention models trained for object recognition [20). What is noticeably missing from each of these studies an large-scale analysis spanning many images and models of how human and DNN alignment has changed as a function of model performance.
45
Improving the correspondence between humans and machines. Inconsistencies between human and DNN representations can be resolved by directly training models to act more like humans. DNNs have been trained to have more human-like attention, or human-like representational distances in their output layers (20] 40 43 47 48). Here, we add to these successes with the neural harmonizer, training routine that automatically aligns the visual strategies (Fig. D of any two observers by minimizing the dissimilarity of their decision ex explanations.
3 Methods
Human f'eature importance datasets. We focused on the ImageNet dataset to compare the visual strategies of humans and DNNs for object recognition at scale. We relied on the two significant efforts for gathering feature importance data from humans ImageNet: the Clicktionary (22) and ClickMe [20] games, which use slightly different methods to collect their data. Both games begin with the same basic setup: two players work together to locate features in an object image that they believe are important for categorizing it. As one of the players selects important image regions, those regions filled into blank canvas for the other observer to and categorize the image quickly as
possible. In Clicktionary (22). both players are humans, whereas in ClickMe (20). the player selecting features is human and the player recognizing images is a DNN (VGG16 [49].. For both games, feature importance maps depicting the average object category diagnosticity of every pixel was computed as the probability of it being clicked by participant. In total, Clicktionary [22] contained feature importance maps for 200 images from the ImageNet validation set, whereas ClickMe (20) contained feature importance maps for non-overlapping set of 196,499 images from ImageNet training and validation sets. Thus, ClickMe has far more data than Clicktionary, but Clicktionary data has more reliable human feature importance data than ClickMe. Our experiments measure the alignment between human and DNN visual strategies using ClickMe and Clicktionary feature importance maps captured on the ImageNet validation set. As we describe in $4 ClickMe feature importance maps from the ImageNet training set are used to implement neural harmonizer.
3

---

Psychophysies participants and dataset. We complemented the feature importance maps from Clicktionary and ClickMe with psychophysics experiments on rapid visual categorization. We recruited 199 participants from Amazon Mechanical Turk (mturk.com to complete the experiments. Participants viewed psychophysics dataset consisting of the 100 animal and 100 non-animal images in the Clicktionary game taken from the ImageNet validation set [22). We used the feature importance maps for each image as masks for the object images, allowing us to control the proportion of important features observers were shown when asked to recognize objects (Fig. 5p) We generated versions of each image that reveal anywhere between 1% to 100% (at log-scale spaced intervals) of the important object pixels against phase scrambled noise background (see Appendix $1 for details mask generation). The total number of revealed pixels was equal for every image at a given level of image masking, and the revealed pixels were centered against the noise background. Each participant saw only one masked version of each object image.
Psychophysics experiment. Participants were instructed to categorize images in the psychophysics dataset as animals or non-animals quickly and accurately as possible. Each experimental trial consisted of the following sequence of events overlaid onto a white background (SI Fig. 1): (i) fixation cross displayed for a variable time 1,600m:; (ii) an image for 400ms; (iii) an additional 150ms of response time. In other words, the experiment forced participants to perform rapid object categorization. They were given total of 550ms to view an image and press button to indicate its category (feedback was provided on trials in which responses were not provided within this time limit). Images were sized at 256 x 256 pixel resolution, which is equivalent to a stimulus size approximately between 5 - 11 degrees of visual angle across a likely range of possible display and seating setups we expect participants used for the experiment Similar paradigms and timing parameters have been shown to capture pre-attentive visual system processing (3 51 52] Participants provided informed consent electronically and were compensated $3.00 for their time (~ 10-15 min; approximately $15.00/hr).
Models. We compared humans with 84 different DNNs representing the variety of approaches
used in the field today: 50 CNNs trained on ImageNet [T 49
53
64 73). 6 CNNs trained on
other datasets in addition ImageNet (which refer to "CNN extra data") T 66 74). 10 vision transformers (75) 6 CNNs trained with self-supervision 80 81]. and 13 models trained for
robustness noise adversarial examples [3] We used pretrained weights for each of these models supplied by their authors, with a variety of licenses (detailed in SI $2), implemented in Tensorflow 2.0, Keras, PyTorch
4 Results
4.1 Where are diagnostic object features for humans and DNNs?
To systematically compare the visual strategies of object recognition for humans and DNNs on ImageNet, first turned to the ClickMe dataset of feature importance maps [20). In order to derive comparable feature importance maps for DNNs, we needed a method that could be efficiently and con- sistently applied to each of the 84 DNNs we tested without any idiosyncratic hyperparameters. This led us to choose a classic method for explainable artificial intelligence, image feature saliency (84). We prepared human feature importance maps from ClickMe by taking the average importance map produced by humans for every image that also appeared in ImageNet validation. We then used Spearman's 's rank-correlation measure the similarity between human feature maps and DNN feature maps for each image [50]. We also computed the inter-rater alignment of human feature importance maps as the mean salit-half correlation across 1000 random splits of the participant pool (p 0.66). We then normalized each human- -NNN correlation by this score (20).
There were dramatic qualitative differences between the features selected by humans and DNNs on ImageNet. In general, humans selected less context and focused more on object parts: for animals, parts of their faces; for moinansmals, parts that enable their usage, like the spade of a shovel (see Fig. [ and SI Fig. 5. The DNN that was most aligned with humans, the DenseNet121, was still only 38% aligned with humans (Fig. 3).
Plotting the relationship between DNNs' top-1 accuracy on ImageNet with their human alignment revealed a striking trade-off: as the accuracy of DNNs has improved beyond DenseNet121, their

---

Human (Cloatde)
Harmonized VIT
VIT
MLP-Mixer
ResNet50
SimCLR
Robust ResNet50
ConvNext
Snake
Bear
Hare
Leopard
Ball
Figure 2: Human and DNNs rely on different features to recognize objects. In contrast, our neural harmonizer aligns DNN feature importance with humans. We smooth feature importance maps from humans (ClickMe) and DNNs with a Gaussian kemnel for visualization.
alignment with humans has worsened (Fig. For example, consider the ConvNext (T). which achieved the best top-1 accuracy our experiments (85.8%), was only 22% aligned with humans equivalent the alignment of the BagNet33 (69] (63% top-1 accuracy). As additional control,
computed the similarity between the average ClickMe map, which exhibits a center bias [85 86) (SI Fig. 5), and each individual ClickMe map. This center-bias control was only outperformed by 42/84 CNNs we tested (f in Fig. 3. Overall, we observe that human and DNN alignment has considerably worsened since the introduction of these models.
The neural harmonizer. While scaling DNNs has immensely helped performance popular benchmark tasks, there are still fundamental differences in the architectures of DNNs and the human visual system (37) which could part of the reason blame for poor alignment. While introducing biological constraints into DNNs could help this problem, there is plenty of evidence that doing so would hurt benchmark performance and require bespoke development for every different architecture (87-89 Is il possible to align a DNN's visual strategies with humans without hurting its performance?
Such a general-purpose method for aligning human and DNN visual strategies should satisfy the following criteria: (i) The method should work with any dully-iifferentiable network architecture. (ii) should not present optimization issues that interfere with learning to solve a task, and the task-accuracy of model trained with the method should not be worse than model trained without the method. We created the neural harmonizer to satisfy these criteria.
Let us consider supervised categorization problem with input space, X' an output space y C R and predictor function fo + y parameterized by 0, which maps an input vector æ X' to an output fe (z). We denote F x X - X' an explanation functional that, given predictor fe E F and an input, returns a feature importance map Φ = g(fe, x). Here, we focus on DNN saliency g(fo. x) A fozfo()) as our method for computing feature importance DNNs, but the method can in principle work with any differentiable network explanation method.
To satisfy criterion (i), the neural harmonizer introduces a differentiable loss that will enforce alignment across feature importance map scales from any neural network. Let P.(() be the function mapping feature importance map & to it is representation the N levels of a Gaussian pyramid, with E {1. N}. The function P. (Φ) is computed by downsampling Pi-1()) using Gaussian
5

---

100
Human feature alignment (%)
CNN
80
Transformer Self-supervised Robust
VIT
CNN extra data
VGG
60
Neural harmonizer
ResNet50
EfficientNet
40
20
0
55
60
65
70
75
80
85
Top-1 ImageNet Accuracy (%)
The trade-off between DNN performance and alignment with human feature impor- tance from ClickMe [20]. Human feature alignment is the mean Spearman correlation between human and DNN feature importance maps, normalized by the average inter-rater alignment of hu- mans. The shaded region denotes the pareto frontier of the trade-off's between ImageNet accuracy and human feature alignment for unharmonized models. Harmonized models (VGGI6, ResNet50, ViT, and EfficientNeetBO) are more accurate and aligned than versions of those models trained only for categorization. Error bars are bootstrapped standard deviations over feature aligmment. Arrows show shift in performance after training with the neural harmonizer. The feature alignment of an average of ClickMe maps with held-out maps is denoted by f.
Figure 3:
kernel, with (i) We then seek minimize D ()) feature importance maps between humans and DNNs at every scale of the pyramid.
(.)))
which will align
To satisfy criterion (ii), the neural harmonizer should work well with training routines designed for large-scale computer vision challenges like ImageNet. This means that the neural harmonizer loss must avoid optimization issues scale. To do this, need way of comparing feature importance maps between humans and DNNs that is invariant to the norm of either map. We therefore standardize feature importance maps from humans and DNNs before comparing them, and only measure alignment on the most important areas of the image for each observer. Formally, let x..) be standardization function computed for each map & such that x(Φ) has 0 activation on average and unit standard deviation. To focus alignment important regions, let 2 denote the positive part of the standardized explanation =(Φ). Finally, we include task loss, the familiar cross entropy objective, to yield the complete neural harmonization loss and train models that are least as accurate as those trained without harmonization:
feature importance maps that takes the mean and standard deviation
Coomonizaaion =x1 Σ P:o (2 P
LCce(foo 2.)) + X2 Σ 0
(2)

---

100
Human feature alignment (%)
CNN
80
Transformer Self-supervised Robust
CNN extra data
VIT
60
Neural harmonizer
VGG
ResNet50
EfficientNet
40
20
55
60
65
70
75
80
85
Top-1 ImageNet Accuracy (%)
The trade-off between DNN performance and alignment with human feature impor- fance from Clicktionary [22]. Human feature alignment is the mean Spearman correlation between human and DNN feature importance maps, normalized by the average inter-rater alignment of humans. The shaded region denotes the pareto frontier of the trade-off's between ImageNet accuracy and human feature aligmment for unharmonized models. Harmonized models (VGG16, ResNet50, MobileNetV1. and EfficientNetBO) are more accurate and aligned than versions of those models trained only for categorization. Error bars bootstrapped standard deviations over feature alignment. Arrows denote a shift in performance after training with the neural harmonizer.
Figure 4:
Training. We trained four different DNNs with the neural harmonizer: VGG16, ViT, ResNet50, and EfficientNetBO. These models were selected because they are popular convolutional and transformer networks with open-source architectures that are straightforward to train and also sit near the boundary of the trade-off between DNN performance and alignment with humans. Models were trained using the neural harmonizer to optimize categorization performance on ImageNet and feature importance map alignment with human data from ClickMe. We trained models all images in the ImageNet training set, but because ClickMe only contains human feature importance maps for a portion those images, we computed the categorization loss but not the neural harmonizer loss for images without importance maps. Models were trained using cores V4 TPUs on the Google Cloud Platform, and training lasted upproximately recipe (built from and momentum over batches of 512 images, learning rate of 0.3, and label smoothing (90). Images were augmented with random left-right flips and mixup [91]. The learning rate was adjusted over the course of training with schedule that began with initial warm-up period of 5 epochs and then decaying according to cosine function over 90 epochs, with decay at step 30, 50 and 80. We validated that ResNet50 and VGGI6 trained with these hyperparameters and schedule using standard cross-entropy (but not the neural harmonizer) matched published performance.
day. Models were trained with augmented ResNet training com/ wwonm//essorffooo wwwensorfooww/oo tttp//pp///.//./ Models were optimized with SGD
The neural harmonizer aligns human and DNN visual strategies. We found that harmonized models broke the trade-off between ImageNet accuracy and model alignment with ClickMe human feature importance maps (Fig. 3) Harmonized models were significantly more aligned with feature importance maps and also performed better on ImageNet. The changes in where harmonized models find important features in images were dramatic: harmonized ViT had feature importance maps that far less reliant on context (Fig. 2 and approximately 150% more aligned with humans (Fig
7

---

100
Important human features (%)
Animal
Non-animal
Normalized performance (%)
80
1
60
40
20
20
100
0
20
40
60
80
100
(a)
(c)
Human features revealed (%)
Human
8
CNN
Transformer
VGG
-log(mean squared error)
Self-supervised
Robust
7
ResNet50
CNN extra data
Neural harmonizer
EfficientNet
6
5
4
3
(b)
55
60
65
70
75
80
85
Top-1 ImageNet Accuracy (%)
Comparing how humans and DNNs visual features during object recognition. (a) Humans and DNNs categorized ImageNet validation images animals or non- non-animals, The images revealed only portion of the most important visual features according to the Clicktionary game [92). (b) There was trade-off between DNN top-1 accuracy on ImageNet and alignment with human visual decision making. The shaded region denotes the pareto frontier of the trade-off between ImageNet accuracy and human feature alignment for unharmonized models. Arrows denote a shift in performance after training with the neural harmonizer. Error bars are bootstrapped standard deviations over decision-making alignment. (c) A state-of-the-an the-t--rt DNN like the ViT learned different strategy for integrating visual features into decisions than humans or a harmonized VIT.
Figure 5:
VIT goes from 28.7% to 72.6% alignment after harmonization). The same model also performed 4% better in top-1 accuracy without any changes to its architecture. Similar improvements were found for

---

the harmonized VGGI6 and ResNet50. While the EfficientNetBO had only minimal improvement in accuracy, too exhibited large boost in human feature alignment.
Clicktionary. To test the trade-off between DNN ImageNet accuracy and alignment with humans a general phenomenon we next turned Clicktionary (22). Indeed, we observed similar trade-off' on this dataset as we found for ClickMe: alignment with human feature importance from Clicktionary has worsened as DNN accuracy has improved on ImageNet (Fig. 4 As with ClickMe, harmonized DNNs shift the accuracy-alignner trade-off on this dataset.
4.2 How do humans and DNNs integrate diagnostic object features into decisions?
The trade-off we discovered between DNN accuracy on ImageNet and alignment with human visual feature importance suggests that the two different visual strategies for object classification. However, there is potential for an even deeper problem. Even if two observers deem the same regions of an image as important for recognizing it, there is no guarantee that they use the selected features in the same way to render their decisions. We posit that if two observers have aligned visual strategies, the will agree on both where important features are in an image and how they use those features for decisions.
We developed psychophysics experiment to measure how different humans use features in ImageNet images to recognize objects. Participants viewed versions of these images where only a proportion of the features that were deemed most important in the Clicktionary game were visible (Fig. 5). Participants had to accurately detect whether or not the image contained an animal within 550ms, which forced them rely feedforward processing as much possible (33) Each of the 200 images we used were shown to single participant only once. We accumulated responses from all participants to construct decision curves that showed how accurately the average human converted any given proportion of image features into an object decision. We performed the same experiment on DNNs as we did on humans, recording animal vs. non-an non-animal decisions according to whether or not the most probable category in the model's 1000-category output was an animal. Because the experiment was speeded, humans did not achieve perfect accuracy. Thus, we normalized performance for humans and DNNs to compare the rate at which each integrated features into accurate decisions.
We discovered similar trade-off between ImageNet accuracy and alignment with human visual decision making in this experiment as we did in ClickMe and Clicktionary (Fig.5p). Indeed, the model that was most aligned with human decision- - the BagNet33 (69] - only achieved 63.0% accuracy on ImageNet. Surprisingly, harmonized models broke this trend, particularly the harmonized ViT (Fig. 5p. top-right), despite no explicit constraints in that procedure which forced consistent decision-making with humans. In contrast, an unharmonized VIT integrates visual information into accurate decisions less efficiently than humans or harmonized models (Fig. 5c).
5 Conclusion
Models that reliably categorize objects like humans do would shift the paradigms of the cognitive sciences and artificial intelligence. But despite continuous progress over the past decade on the ImageNet benchmark, DNNs are becoming worse models of human vision. Our finding resembles a growing number concurrent works showing similar trade-offs between DNN performance and predictions of human perception on different tasks [93] 94). Our solution to this problem, the neural harmonizer, can be applied to any DNN to align their visual strategics with humans and even improve performance.
We observed the greatest benefit of harmonization on the visual transformer, the VIT. This finding is particularly surprising given that transformers eschew the locality bias of convolutional neural networks that has helped them become the new standard for modeling human vision and cognition (37]. Thus, we suspect that the neural harmonizer is especially well-suited suited for large-scale training on low- inductive bias models, like transformers. We also hypothesize that the improvements in human alignment provided by the neural harmonizer will yield variety of downstream benefits for a model like the VIT, including better predictions of perceptual similarity, stinulus-evoked neural responses, and even performance visual reasoning tasks. We leave these analyses for future work.

---

The field of computer vision today is following Sutton's prescient lesson: benchmark tasks can be scaling architectural capacity and the size of training data. However, as we have demonstrated here, these scaling laws are exchanging performance for alignment with human perception. We encourage the field to re-analyze the costs and benefits of this exchange, particularly in light the growing concerns about DNNs leveraging shorteuts and dataset biases to achieve high performance (19). Alignment with human vision need not be exchanged with performance if DNNs are harmonized. Our codebase (https: /tte:///eeeeeeee lab. github. io aoooaaaoonii zation/) can be used to incorporate the neural harmonizer into any DNN created and measure its alignment with humans on the datasets we describe in this paper.
Limitations. One possible explanation for the misalignment between DNNs and humans that observe is that recent DNNs have achieved superhuman accuracy on ImageNet. Superhuman DNNs have been described in biomedical applications (95]96] where there is definitive biological ground-truth labels, but ImageNet labels are noisy, making unclear if such an achievement is laudable (https: /tt:///abbnn htt//aabbeeerrrr..mm com/). Thus, an equally likely explanation is that the continued improvements of DNNs at least partially reflect their exploitation of shortcuts in ImageNet [19].
The scope of our work is also limited in that it focuses on object recognition in ImageNet. It is possible that models trained other tasks, such as segmentation, may be more aligned with humans.
Finally, our modeling efforts were hamstrung for the largest-scale models existence. Our work does not answer how much harmonization would help model like CLIP because of the massive investment needed to train it. The neural harmonizer can be applied to CLIP but it is possible that more ClickMe human feature importance maps are needed for successful harmonization.
Broader impacts. A persistent issue in the field of artificial intelligence the tendency of models to exploit dataset biases. A central theme of our work is that there are facets of human perception that are not captured by DNNs, particularly those which follow the scaling laws which have been so embraced by industry leaders. Forcing DNNs to rely on similar visual strategies as humans could represent scalable path forward to correcting the insidious biases which have assailed under-constrained models of artificial intelligence.
Acknowledgments and Disclosure of Funding
This work was supported by ONR 1000014-9--1-2220 NSF (112-1912200 and EAR-192548) 1). DARPA (D19AC00015), NIH/NINDS (R21 NS 12743), and the ANR-3IA Artificial and Natural Intelligence Toulouse Institute ANNR-19-P1AA-00U
Additional support provided by the Carney Institute for Brain Science and the Center for Computation and Visualization (CCV). We acknowledge the Cloud TPU hardware resources that Google made available via the TensorFlow Research Cloud (TFRC) program as well as computing hardware supported by NIH Office of the Director grant S100D025181.
References
[1] Liu, 7., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A ConvNet for the 2020s. (January 2022)
[2] Zhai, X., Kolesnikov, A., Houlsby, N., Beyer, L.: Scaling vision transformers. (June 2021)
[3] Kaplan, J., McCandlish, S., Henighan, T., Brown, T.B., Chess, B., Child, R., Gray. S., Radford, A., Wu, J., Amodei, D.: Scaling laws for neural language models. (January 2020)
Sutton, R.: The bitter lesson. (May 2019)
[5] Krizhevsky, A., Sutskever, I., Hinton, G.F.: ImageNet classification with deep convolutional neural networks. In Pereira, F., Burges, C.J., Bottou, L.,, Weinberger, K.Q., eds.: Advances in Neural Information Processing Systems. Volume 25., Curran Associates, Inc. (2012)
[6] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hicrarchical image databasc. In: 2009 IEEE Conference Computer Vision and Pattem Recognition. (June 2009) 248 255
[7] Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A.. Bernstein, M., Berg. A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. (September 2014)
10

---

[8] DiCarlo, J.J., Zoccolan, D., Rust. N.C.: How does the brain solve visual object recognition? Neuron 73(3)
(February 2012) 415-434
[9] Ullman, S., Assif, L., Fetaya, E.,, Harari, D.: Atoms of recognition in human and computer vision. Proc. Natl. Acad. Sei. U. S. A. 113(10) (March 2016) 2744-2749
[10] Buswell, G.T.: How people look at pictures: a study of the psychology and pereeption in art. 198 (1935)
[11] Yarbus, A.I.: Bye Movements and Vision. Springer US
[12] Posner, M.L: Orienting of attention. Q. J. Exp. Psychol. 32(() (February 1980) 3-25
[13] Mannan, S.K., Kennard, C., Husain, M.: The role visual salience in directing eye movements visual
object agnosia. Curr. Biol. 19(6 (March 2009) R247-8
[14] Gruber, L..., Ullman, S., Ahissar. E.: Oculo-retinal dynamics can explain the perception minimal recognizable configurations. Proc. Natl, Acad. Sci. U. S. A. 118(34) (August 2021)
[15] Fel, T., Colin, J.. Cadene, R., Serre, T.: What I cannot predict. do not understand: A Human--Cnterred evaluation framework for explainability methods. Advances Neural Information Processing Systems (NeurIPS) (December 2021)
[16] Fel, T., Cadene, R., Chalvidal, M.. Cord, M., Vigouroux, D., Serre, T:: Look the variance! efficient black-box explanations with sobol-based sensitivity analysis. Advances in Neural Information Processing Systems (NeurIPS) (November 2021)
[17] Fel, T., Ducoffe, M., Vigouroux, D., Cadene, R., Capelle, M., Nicodeme, C.,, Serre, T:: Don't lie me! robust and efficient explainability with verified perturbation analysis. Workshop, Proceedings of the International Conference on Machine Learning (ICML) .) (February 2022)
[18] Fel, T., Vigouroux, D., Cadène, R., Serre, T:: How good your explanation? algorithmic stability measures to assess the quality of explanations for deep neural networks. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (September 2020)
[19] Geirhos, R., Jacobsen, J.H., Michaelis, C., Zemel, R. Brendel, W., Bethge, M., Wichmann, F.A.: Shortcut learning in deep neural networks. Nature Machine Intelligence 2(11) (November 2020) 665 663
[20] Linsley, Shichler, D., Fberhardt, S., Serre, T: Learning what and where attend with humans in the loop. In: International Conference on .earning Representations, (2019)
[21] Lin. T., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongic, S.: Feature pyramid networks for object detection. In: 2017 IEBE Conference on Computer Vision and Pattern Recognition (CVPR). (July 2017)
[22] Linsley, D., Eberhardt, S., Sharma, T., Gupta, P., Serre, T.: What are the visual features underlying human versus machine vision? In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). (October 2017) 2706-226222 2714
[23] Geirhos, R., Narayanappa, K., Mitzkus, B., Thieringer, T., Bethge, M., Wichmann, F.A., Brendel, W.:
Partial success closing the gap between human and machine vision. (June 2021)
[24] Linsley, D., Kim, J., Ashok, A., Serre, T.: Recurrent neural circuits for contour detection. International Conference on Leamingg Representations (2020)
[25] Kim J. Linsley* D., Thakkar, K., Serre, T.: Disentangling neural mechanisms for perceptual grouping. International Conference on Representation Learning (2020)
[26] Linsley, D., Malik, G., Kim, J., Govindarajan, I.N., Mingolla, F.,, Serre, T.: Tracking without re-recognition in humans and machines. (May 2021)
[27] Golan, T., Raju, P.C., Kriegeskorte, N.: Controversial stimuli: Pitting neural networks against each other as models of human cognition. Proc. Natl. Acad. Sci. U. S. A. 117(47) (November 2020) 99330-29337
[28] Langlois, T., Zhao, H., Grant, E.,, Dasgupta, I., Griffiths, T., Jacoby, N.: Passive attention artificial neural networks predicts human visual selectivity. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P.S., Vaughan, J.W., eds.: Advances in Neural Information Processing Systems. Volume 34., Curran Associates, Inc. (2021) 27094-27777
[29] Yamins, D.I.K., Hong, H., Cadieu, C.F., Solomon, F.A., Seihert, D., DiCarlo, JJ::: Performance-pptimize hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sei. U. S. A. 111(23) (June 2014) 8619-8624
[30] Schrimpf, M., Kubilius, J., Lee, MJ., Ratan Murty, N.A., Ajemian, R., DiCarlo, JJ:: Integrative benchmarking to advance neurally mechanistic models of human intelligence. Neuron 108(3) (November 2020) 413-423
[31] Habre-Thorpe M.: The characteristics and limits of rapid visual categorization. Front. Psychol. 2 (October 2011) 243
11

---

[32] Roelfsema, P.R.. Lamme, V.A., Spekreijse, H.: The implementation of visual routines. Vision Res.
40(10-12) (2000) 1385-1411
[33] Serre, T., Oliva, A., Poggio, T.: A feedforward architecture accounts for rapid categorization. Proc. Natl. Acad. Sci. U. S. A. 104(15) (April 2007) 6424-6429
[34] Kietzmann, T.C.,, Spoerer, C.J., Sörensen, I..,.A., Cichy, R.M., Hauk, O., Kriegeskorte, N.; Recurrence
required to capture the representational dynamics of the human visual system. Proc. Natl, Acad. Sci. U. S.
A. 116(43) (October 2019) 21854-21 2163
[35] Jagadeesh, V., Gardner, J.L.: Texture-likk representation of objects in human visual cortex. Proceedings of the National Academy of Sciences 119(17) (2022) c21 15302119 19
[36] Berrios, W., Deza, A.: Joint rotational invariance and adversarial training of dual-stream transformer yields state of the art Brain-Score for area V4. (March 2022)
[37] Serre, T.: Deep learning: The good, the bad, and the ugly. Annu Rev Vis Sci 5 (September 2019) 399-426
[38] Koehler, K., Guo, F., Zhang, S., Eckstein, M.P. What do saliency models predict? J. Vis. 14(3) (March
2014) 14
[39] Jiang, M., Huang, S., Duan, J., Zhao, Q.: SALICON: Saliency context. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). (June 2015) 1072-1080
[40] Peterson, J.C., Abbott, J.T., Griffiths, T.L.: Evaluating (and improving) the correspondence between deep neural networks and human representations. Cogn. Sci. 42(8) (November 2018) 2648-20040 2266
[41] Lai, Q.. Khan, S., Nie, Y., Shen, J.. Sun, H. Shao. L.: Understanding more about human and machine attention in deep neural networks. (June 2019)
[42] Ebrahimpour, M.K., Falandays, J.B., Spevack, S., Noelle, D.C.: Do humans look where deep convolutional neural networks "attend"? In: Advances in Visual Computing, Springer International Publishing (2019) 53-65
[43] Roads, B.D., Love, B.C.: Enriching ImageNet with human similarity judgments and psychological embeddings. (November 2020)
[44] Marjich, R., van Rijn, P. Sucholutsky, I., Sumers, T.R., Lee, H., Griffiths, T.L., Jacoby, N.: Words are all you need? capturing human sensory similarity with textual descriptors. (June 2022)
[45] Funke, J., Tschopp, F.D., Grisaitis, W., Sheridan, A., Singh, C., Saalfeld, S., Turaga, S.C.: Large scale image segmentation with structured loss based deep learning for connectome reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. (2018) 1-1
[46] Srivastava, S., Ben- Yosef. G., Boix, X.: Minimal images in deep neural networks: Fragile object recognition in natural images. (February 2019)
[47] Boyd, A., Tinsley, P., Bowyer. K., Czajka, A.: CYBORG: Blending human saliency into the loss improves deep learning. (December 2021)
[48] Bomatter, P., Zhang, M., Karev, D., others: When pigs fly: Contextual reasoning in synthetic and natural scenes. Proceedings of the (2021)
[49] Simonyan, K., Zisserman, A.: Very deep convolutional networks for Large-Scale image recognition. (Septemher 2014)
[50] Eberhardt, S., Cader, J.G., Serre, T.: How de is the feature analysis underlying rapid visual categorization? In Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R., eds.: Advances in Neural Information Processing Systems 29, Curran Associates, Inc. (2016) 1100-1108
[51] Kirchner, H., Thorpe, S.J.: Ultra-rapid object detection with saccadic eye movements: visual processing
speed revisited. Vision Res. 46(11) (May 2006) 1762- 1776
[52] Muriel, B., Simon, T., Holle, K.: Rapid object categorization without conscious recognition: aneuropsy- chological study. J. Vis. 7(9) (June 2007) 1033-1033
[53] Chen, X., Yan, B., Zhu, J., Wang, D., Yang. X., Lu, H.: Transformer tracking. (March 2021)
[54] Tan, M.. Le, Q.V.: EfficientNet: Rethinking model scaling for convolutional neural networks. (May 2019)
[55] Radosavovic, I., Kosaraju, R.P., Girshick. R., He, K., Dollár, P:: Designing network design spaces. (March
2020)
[56] Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang. R., Vasudevan, V., Le, Q. Adam, H.: Searching for MobileNetV3. (May 2019)
[57] Huang, L., Zhao, X., Huang, K.: GOT-10k: A large High-Diversity benchmark for generic object tracking in the wild. (October 2018)
[58] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. (December 2015)
12

---

[59] Zhang, H. Wu, C., Zhang, Z.. Zhu, Y., Lin, H., Zhang, Z., Sun, Y.. He. T., Mueller, J., Manmatha, R., Li. M., Smola, A.: ResNeSt: Split-Attention networks. (April 2020)
[60] Gao, S.H., Cheng. M.M., Zhao, K., Zhang, X.Y., Yang, M.H., Torr, P:: Res2Net: A new Multi-Scale
backbone architecture. IEEE Trans. Pattern Anal. Mach. Intell. 43(2) (February 2021) 652 662
[61] Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J. Yung. J., Gelly, S., Houlsby, N.: Big transfer (BiT):
General visual representation learning. (December 2019)
[62]
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: Inverted residuals and linear bottlenecks. (January 2018)
[63]
Szegedy, C., loffe, S., Vanhoucke, V., Alemi, A:: Inception-v4, Inception ResNet and the impact of residual connections on learning. (February 2016)
[64]
Szegedy, C., Vanhoucke, V., loffe. S., Shlens, J.. Wojna, Z.: Rethinking the inception architecture for computer vision. (December 2015)
[65]
Chollet, F: Xception: Deep learning with depthwise separable convolutions. (October 2016)
[66] Radford, A., Kim, J.W. Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P.,
Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. (February 2021)
[67] Xie, C., Tan, M., Gong, B., Wang, J., Yuille, A., Le, Q.V.: Adversarial examples improve image recognition. (November 2019)
[68] Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. (November 2016)
[69] Brendel, W., Bethge, M.: Approximating CNNs with Bag-of-local-Feantee models works surprisingly well on ImageNet. (March 2019)
[70] Mehta, D., Sotnychenko, O., Mueller, F., Xu, W., Elgharib, M.. Fua, P., Seidel, H.P., Rhodin, H. Pons-Moll, G., Theobalt, C.: XNect: real-time multi-person 3D motion capture with single RGB camera. ACM Trans. Graph. 39(4) (July 2020) 82:1-82:17
[71] Chen, Y. Li, J., Xiao, H., Jin, X., Yan, S., Feng, J.: Dual path networks. (July 2017)
[72] Wang, C.Y., Liao, Y.M. Yeh, I.H., Wu, Y.H., Chen, P.Y., Hsieh, J.W.: CSPNet: new backbone that can enhance learning capability of CNN. (November 2019)
[73] Tan, M., Chen, B., Pang. R., Vasudevan, V.. Sandler, M., Howard, A., Le, Q.V: MnasNet: Platforn-Aware neural architecture search for mobile. (July 2018)
[74] Xie, Q.. Luong, M.T., Hovy, E., Le, Q.V.: Self-training with noisy student improves ImageNet classification. (November 2019)
[75] d' Ascoli, S., Touvron, H.. Leavitt. M.. Morcos, A.. Biroli, G., Sagun, L.: ConViT: Improving vision transformers with soft convolutional inductive biases. (March 2021)
[76] Touvron, H., Cord, M., Douze, M., Massa, F,, Sablayrolles, A., Jégou, H.: Training data-efficient image
transformers & distillation through attention. (December 2020)
[77] Tolstikhin, L., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung. J., Steiner, A..
Keysers, D., Uszkoreit, J., Lucic, M., Dasovitskiy, A.: MI.e- An all-MI.P architecture for vision.
(May 2021)
[78] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image worth 16x16 words: Transformers for image recognition at scale. (October 2020)
[79] Steiner, A., Kolesnikov, A., Zhai, X., Wightman, R., Uszkoreit, J., Beyer, I..: How to train your VIT? data, augmentation, and regularization in vision transformers. (June 2021)
[80] Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual
representations. (February 2020)
[81] Zeki Yalniz, I., Jégou, H., Chen, K., Paluri, M., Mahajan, D.: Billion-scale semi-supervised learning for
image classification. (May 2019)
[82] Geirhos, R., Rubisch, P.. Michaelis. C., Bethge, M.. Wichmann, F.A., Brendel, W.: ImageNett-rranned
CNNs are biased towards texture: increasing shape bias improves accuracy and robustness. (November 2018)
[83] Salman, H., Ilyas, A., Engstrom, L., Kapoor, A., Madry, A.: Do adversarially robust ImageNet models transfer better? (July 2020)
[84] Simonyan, K.. Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: Visualising image
classification models and saliency maps. (December 2013)
13

---

[85] Deza, A., Konkle, T.: Emergent properties of foveated perceptual systems. (June 2020)
[86]
Wang, P., Cottrell, G.W.; Central and peripheral vision for seene recognition: A neurocomputational
modeling exploration. J. Vis. 17(4) (April 2017) 9
[87] Tang, H., Schrimpt. M., Lotter, W., Moerman, C., Paredes, A., Ortega Caro, J., Hardesty, W. Cox. D., Kreiman, G.: Recurrent computations for visual pattern completion. Proc. Natl. Acad. Sci. U. 115(35)
(August 2018) 8835-8840
[88] Kubilius, J.. Schrimpf. M.. Kar, K., Rajalingham, R., Hong, H., Majaj. N., Issa, E., Bashivan, P., Prescott- Roy, J., Schmidt, K., Nayebi, A., Bear, D., Yamins, D.L., DiCarlo, J.J.; Brain-Like object recognition with High-Porforming shallow recurrent ANNs. In Wallach, H., Larochelle, H., Beygelzimer, A., d Alché-Bus F., Fox, E., Garnett, R., ods.: Advances in Neural Information Processing Systems 32. Curran Associates, Inc. (2019) 12805-12816
[89] Schrimpf. M., Kubilius. J., Lee, M.., Ratan Murty, N.A. Ajemian, R., DiCarlo, JJ:: Integrative benchmarking to advance neurally mechanistic models of human intelligence. Neuron 108(3) (Novemher 2020) 413-423
[90] Müller, R., Kornblith, S., Hinton, G.: When does label smoothing help? (June 2019)
[91] Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, mixup: Beyond empirical risk minimization. (October 2017)
[92] Linsley, D., Eberhardt, S., Sharma, T., Gupta, P., Serre, T. What the visual features underlying human
versus machine vision? (January 2017)
[93] Kumar, M., Houlsby, N., Kalehbrenner, N., Cubuk, E.D.: Do better ImageNet classifiers assess perceptual similarity better? (Septemher 2022)
[94] Muttenthaler, L.. Dippel, J., Linhardt, L., Vandermeulen, R.A., Kornblith, S.: Human alignment of neural
network representations. (November 2022)
[95] Linsley, J.W., Linsley, D.A., Lamstein, J., Ryan, G., Shah, K., Castello, N.A., Oza, V., Kalra, J., Wang, S., Tokuno, Z.. Javaherian, A., Serre. T., Finkbeiner, S.: Superhuman cell death detection with biomarker- optimized neural networks. Sci Adv 7(50) (December 2021) eabf8142
[96] Lee, K., Zung. J., Li, P., Jain, V., Sebastian Seung, H.: Superhuman accuracy on the SNEMI3D connec- tomics challenge. (May 2017)
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Discussion.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Discussion
(d) Have you read the ethics review guidelines and ensured that your paper conforms them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See SI.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes]
4. If you using existing assets (c.g., code, data, models) or curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
14

---

(b) Did you mention the license of the assets? [Yes]
(c) Did you include any assets either in the supplemental material or as URL? [Yes]
(d) Did you discuss whether and how consent obtained from people whose data you're using/curating? [Yes] See Methods
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes] See Methods
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [Yes] See SI.
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [Yes] See SI.
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [Yes] See Methods
15


Reviews:
[{"review_id": "sj10B557NrL", "reviewer": "Reviewer_cEc7", "summary": "Summary: This paper focusses on evaluating the alignment between visual strategies of DNNs and human perception. Specifically, the ask how this alignment evolves as architectures become more complex. For a fair comparison the paper focusses on only the rapid, feed-forward component of human perception, and suggests evaluating alignment on visual strategies as defined by a two step process---identifying regions with important features and how these regions are used to make a classification decision. For the first part, the paper evaluates alignment between DNNs and humans on three datasets. These include ClickMe and Clicktionary datasets with ground truth annotations for human visual importance, and a third dataset collected by the authors using a psychophysics experiment involving rapid classification of images. The paper shows that identifying important features peaks as architectures become better at classification, but then start dipping as they get more complex. To address this, the neural harmonizer methodology is proposed. Finally, the paper tests how these features are used by exposing different models to an increasing percentage of human-identified important regions, showing similar trends as the previous experiments.", "questions": "1. On the role of scene context: Human annotations highlighted lesser context areas (Page 5), but it is widely known that human visual recognition is strongly dependent on scene context. Does this suggest that what humans \"think\" of as visually important, may not essentially be the most important regions? Also may be suggested by work on minimal images, as it is counter-intuitive to humans that a few pixels can cause such large drops in accuracy. Would be good to see this discussed briefly in the limitations.\n\n\n2.  A broader question is that the evaluation of alignment necessarily requires a gold standard of knowing what humans rely on the first place. How confident are we in the current measured ClickMe/Clicktionary style data? Also, how do the psychophysics data collected here improve upon these? A short discussion on this would be helpful.", "limitations": "Yes.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. Extremely interesting problem. While accuracy on standardized datasets is a good diagnostic for progress, measuring if these improvements result in better alignment between biological and artificial intelligence is a fundamental goal for a multitude of reasons. This has also been very well motivated in the paper.\n\n2. The writing is clear and easy to follow. The work is placed well into the context of existing works, and is clear in elaborating how this work builds on top of this literature. \n\n3. Above all, the experimental design is very solid. Controls are sufficient, and claims are well supported with either experimental results or through existing literature. Psychophysics experiments are well designed, and good care has been taken to ensure as fair a comparison between humans and DNNs as possible.\n\n4. The neural harmonizer module is interesting as it increases alignment significantly.\n\n\nWeaknesses:\n\n1. The code and data are propriety and will NOT be released. The findings of this paper are interesting, but the most important contribution here seems to be an interesting benchmark for future researchers to focus on. Since the code and the data will not be released, it severely diminishes the utility for the research community. \n\nMinor Weakness:\n\nSome relevant recent literature is missing, and would be good to add to the paper:\n\nA. On minimal images: \n\n- Srivastava, S., Ben-Yosef, G. and Boix, X., 2019. Minimal images in deep neural networks: Fragile object recognition in natural images. arXiv preprint arXiv:1902.03227.\n\nB. On alignment in humans and machines in existing works:\n\n- Bomatter, P., Zhang, M., Karev, D., Madan, S., Tseng, C. and Kreiman, G., 2021. When pigs fly: Contextual reasoning in synthetic and natural scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 255-264).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "jSIEIZkAEOq", "reviewer": "Reviewer_Xrpg", "summary": "This paper re-introduces a very important topic in the ML + CV community that has not been studied enough: are computer vision models aligned to human visual perception (and if they are not, why should we care?). The authors introduce the neural harmonizer as a training procedure that aligns the human-relevant features with a CNN to train from scratch a couple of new models from classical CNN architectures (ResNet + VGG) and they show that their models improve human perceptual alignment while also maintaining ImageNet Test Accuracy and performance. Furthermore, they empirically show that networks trained with the Neural Harmonizer are more robust to a peripheral occlusion task, and most importantly -- that their change in performance rate is similar to that of a human without any additional fine-tuning on the network.", "questions": "See Weaknesses points (1) and (3).", "limitations": "None, the paper actually does a great job in introducing current limitations of modern DL systems. The paper is overall well written. Providing additional experiments and/or a good rebuttal may improve my rating. I am also generally curious to know what other reviewers have thought about the maturity of this paper. \n\nI'm really on the fence with this paper. It's amazingly well written, addresses an important problem in modern ML + CV -- via the proper cog-sci/psychology insights, but is still missing more results that back-up their claim with stronger evidence. There are no error bars for example, and authors could have definitely trained the other NN's with their Neural Harmonizer, to provide substantial evidence about how the Harmonizer provides better human-perceptual alignment.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper’s main strengths is the paper’s theme as well as the very accessible way the paper has been written. Furthermore the paper’s experiments seem properly articulated and well-posed to support the main hypothesis and question that the authors introduce at the beginning of the paper. The psychophysics is pretty amazing.\n\nHowever, the paper still has some weaknesses that should be addressed/clarified:\n* **Weak Results:** Why are there only 2 reported neural networks that have been trained with the neural harmonizer (there are 2 purple dots)? I would have expected that for the claim to be “the neural harmonizer successfully increases man-machine perceptual alignment”, that there would be nearly all blue CNN dots transcribed to purple, including (potentially) the yellow ones which are the Transformers. Without these extra data points, the results seem very weak, and almost suggests cherry picking.\n* **Missing References:**\n    * “Emergent Properties of Foveated Perceptual Systems”. Deza & Konkle. ArXiv, 2020.\n    * “Central and peripheral vision for scene recognition: A neurocomputational modeling exploration” Wang & Cottrell. JoV, 2017.\n    * “Joint rotational invariance and adversarial training of a dual-stream Transformer yields state of the art Brain-Score for Area V4”. Berrios & Deza. Brain-Score Workshop 2022.\n    * “Texture-like representation of objects in human visual cortex”. Jagadeesh & Gardner. PNAS 2022.\n* **Experimental Results may provide evidence for alternative explanation**: The occlusion results from Figure 5 may also suggest that the neural harmonizer has enforced a central image bias in CNNs :O ! (So a partial rather than full human-perceptual alignment). It is expected that Transformers would not a priori have this bias. That being said: How would the Transformer’s curve trained with a Neural Harmonizer look like? In addition, what would the curve look like if the occlusion patterns were inverted, and one where to begin by starting to show peripheral regions of the image, and slowly showing the most important features last? Do the same patterns of results hold between humans and neural harmonizer trained networks? See for example the work of Deza & Konkle (ArXiv, 2020) and also Wang & Cottrell (JoV, 2017)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Ks1jBZ2ONQ4", "reviewer": "Reviewer_xZSj", "summary": "The authors conduct an empirical evaluation of the alignment of human and artificial neural network (ANN) saliency maps, and propose a method for improving it. The authors use saliency maps collected from ImageNet by Linsley et al. and compare them to those estimated from a variety of ANN's. In a separate experiment, the authors obscure images to varying degrees according to the human saliency maps, and measure the resulting degradation in classification performance in humans and ANN's, again comparing the two. Finally, the authors propose a method for aligning the saliency maps of humans and ANN's by jointly training the network for the original classification objective, and a measure of alignment with human saliency. The resulting models' saliency maps are indeed better aligned with human ones. ", "questions": "As discussed above, the proposed neural harmonizer is of limited scientific interest, as it simply aligns training and testing for this particular metric (alignment with human saliency maps). \n\nAssuming this were removed from the paper, could the authors instead re-analyze their data to study which training procedure, architecture, or data enables a model to reproduce human saliency maps without resorting to training on this data? For example \n\n- Given the same model architecture and training data, which training procedure (supervised, self-supervised, etc) leads to the best alignment with human saliency?\n\n- Given the same training data and objective (e.g. supervised or self-supervised), which model architecture (ResNet, ViT, etc) leads to the best alignment with human saliency?\n\n- Given the same architecture and training objective, which training data (ImageNet, ImageNet21k, etc) leads to the best alignment with human saliency?", "limitations": "There are no ethical or societal concerns associated with this work. ", "rating": 3, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The authors propose a sound evaluation of the alignment between human and ANN saliency maps. In particular, the experiment they propose which incrementally reveals images according to human saliency maps appears to be a good way of comparing the features used by humans and ANN's in an apples-to-apples manner. Whether this experiments reveals \"how features are used\" rather than \"where these features are\" is less clear: it instead appears to be a better controlled version of the original comparison between saliency maps. \n\nSecondly, the authors make good use of this evaluation to show that improving ImageNet classification accuracy doesn't lead to better alignment with human saliency maps, underscoring the limitations of ImageNet as a benchmark. It would be interesting to see if other computer vision benchmarks, such as object detection or semantic segmentation are more predictive of human saliency.\n\nFinally, it is encouraging to see that co-training the network to reproduce saliency maps allows it to do so, without any cost to the ImageNet classification accuracy (or even with a slight improvement to it), however it is unclear what the scientific value of this result is. Aligning the training and testing objective usually improves test performance, hence improving the alignment with human saliency is hardly surprising. The fact that it improves ImageNet classification is more surprising, but is at odds with the first finding that models which improve classification accuracy do not improve the alignment with human saliency. \n\nA much more interesting point would be to investigate what training procedure, architecture, or data enables a model to reproduce human saliency **without resorting to training on this data**. This would indicate which of the above is most closely matches the conditions leading to the human visual system's choice of features for performing these tasks. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "7lM5hTjm2d0", "reviewer": "Reviewer_cZRY", "summary": "The authors address the problem of evaluating the alignment of deep learning vision models with humans. To do so, they compare feature importance maps by humans, with saliency maps from DNNs. Additionally, they propose a training method (neural harmonizer) to improve such alignment. Finally authors present a psychophysic experiment using the rapid categorization paradigm to gather information of human decisions based on partial information of important features.", "questions": "I would be incline to change ma opinion if authors:\n- provide strong motivations on the choice of the one method for saliency, or present results for at least another different approach.\n- provide strong motivations on the choice of the two models to be improved with neural harmonizer, or provide average results for all initial 85 models\n- clarify why using feature importance map is sufficient to explain and measure alignment, in particular w.r.t. research in saliency and scanpath that rely directly on eye-tracking data", "limitations": "The discussion of limitations is insufficient. The authors should point out that results are only provided for one explainability method, and only for two models. Additionally, they should point out how the notion of feature importance and what is actually captured by humans through the gaze might differ.", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- the topic is timely and relevant\n- the paper is clear and easy to read\n- the authors validate part of their claims with a human study\n- code is provided for reproducibility\n\nWeaknesses:\n- Structure of the paper is not optimal. The neural harmonizer should probably be described introduced in the method section.\n- The rapid categorization paradigm has already been used to compare humans and DNNs decisions and it could be beneficial to provide those references (the provided references are from the neurosc. field)\n- Author exclusively use saliency maps from Simonyan et al. to reveal feature importance. However, studies have found severe weaknesses of such metrics that can also be drastically attacked. Authors could provide more arguments on why they chose to rely specifically and exclusively on this method, or eventually use more.\n- Related to the previous point, the proposed neural harmonizer method optimises neural networks to change their feature importance (w.r.t. the selected method of saliency). I rather have a different interpretation than those provided by the authors on the effect of this approach: this might lead the network to learn more effective activation that lead to human-like feature importance (maybe leveraging spatial biases of human annotators) while not changing the underlying decision process. The improvement in accuracy might be due to the additional loss and training data (in fact, multi-tasking is performed). Which lead me to the following observation:\n- Why do authors provide results for only two networks (ResNet50 and VGG16) of the 85 presented in the first experiment, and why those two? Their explanation is given in page 6, they are \"selected because they are popular networks that sit on the boundary of the trade-off betweenDNN performance and alignment with humans\". This seems to be not fully true from figure 4. Additionally, it might be very useful to know the average performance on the other architectures (or at least a random sample of them, if training them all is prohibitive) in order to support the claim that the method improves alignment without decreasing the performance. Especially important are the performance on DenseNet121, mentioned by the authors as the very limit of the trade-off between alignment and accuracy.\n- Authors assume that feature importance maps from the two selected datasets represent where features which are important for the decisions are in a scene. However, I am not sure this aligns with locations people attend with their gaze. Authors should discuss this point in the paper. Eventually, an experiment involving saliency maps (using for example SALICON, a subset of imagenet with human visual saliency information) might be meaningful?\n\nMinor:\n- Page 4, row 2, \"was was\" (repetition)\n- The authors refer to feature maps also as \"visual strategies\" often during the paper, however, I found this to be confusing since in the field this is more often used for visual scanpaths than for (static) saliency. I suggest use a different terminology.\n- I am not sure wether presenting a partial image cropped is effective since, even when selecting a smaller area, people can see the rest of the image with peripheral vision. Trying to model this effect could have made the experiment more reliable.\n- Section 4.1: motivate the use of spearman's rank-correlation or provide references\n- In the paragraph \"The neural harmonizer aligns human and DNN visual strategies.\": authors mention \"All of the three neural harmonizer models...\", however they presented only two of them.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept