Partial success in closing the gap between human and machine vision
Authors: Robert Geirhos
Kantharaju Narayanappa
Benjamin Mitzkus
Tizian Thieringer
Matthias Bethge
Felix A. Wichmann
Wieland Brendel
Venue: NeurIPS 2021 Oral (2021)
Keywords: psychophysics
machine vision
human vision
out-of-distribution generalisation
visual perception
robustness
human behaviour

Abstract: A few years ago, the first CNN surpassed human performance on ImageNet. However, it soon became clear that machines lack robustness on more challenging test cases, a major obstacle towards deploying machines "in the wild" and towards obtaining better computational models of human visual perception. Here we ask: Are we making progress in closing the gap between human and machine vision? To answer this question, we tested human observers on a broad range of out-of-distribution (OOD) datasets, recording 85,120 psychophysical trials across 90 participants. We then investigated a range of promising machine learning developments that crucially deviate from standard supervised CNNs along three axes: objective function (self-supervised, adversarially trained, CLIP language-image training), architecture (e.g. vision transformers), and dataset size (ranging from 1M to 1B).

Our findings are threefold. (1.) The longstanding distortion robustness gap between humans and CNNs is closing, with the best models now exceeding human feedforward performance on most of the investigated OOD datasets. (2.) There is still a substantial image-level consistency gap, meaning that humans make different errors than models. In contrast, most models systematically agree in their categorisation errors, even substantially different ones like contrastive self-supervised vs. standard supervised models. (3.) In many cases, human-to-model consistency improves when training dataset size is increased by one to three orders of magnitude. Our results give reason for cautious optimism: While there is still much room for improvement, the behavioural difference between human and machine vision is narrowing. In order to measure future progress, 17 OOD datasets with image-level human behavioural data and evaluation code are provided as a toolbox and benchmark at: https://github.com/bethgelab/model-vs-human/

Partial success in closing the gap between human and machine vision
Robert Geirhos 1.2%
Kantharaju Narayanappa'
Benjamin Mitzkus'
Tizian Thieringer
Matthias Bethge
Felix A. Wichmann
Wieland Brendel
University of Tübingen
International Max Planck Research School for Intelligent Systems
"Joint senior authors
To whom correspondence should be addressed: robert. hewpeet.ggrrhhommm-tuueeeeeeeeee
Abstract
A few years ago, the first CNN surpassed human performance on ImageNet. How- ever, it soon became clear that machines lack robustness on more challenging test cases, major obstacle towards deploying machines "in the wild" and towards obtaining better computational models of human visual perception. Here ask: Are we making progress in closing the gap between human and machine vision? To answer this question, we tested human observers on broad range of out-of- distribution (OOD) datasets, recording 85,120 psychophysical trials across 90 participants. We then investigated a range of promising machine learning devel- opments that crucially deviate from standard supervised CNNs along three axes: objective function self-supprvised, adversarially trained, CLIP language-image training), architecture (e.g. vision transformers), and dataset size (ranging from IM to IB).
Our findings are threefold. tween humans and CNNs is closing, with the best models now exceeding human feedforward performance still a substantial image-level consistency gap, meaning that humans make different errors than models. In contrast, most models systematically agree in their cate- gorisation errors, even substantially different ones like contrastive self-supervised standard supervised models. (3.) In many cases, human-to- model consistency improves when training dataset size increased by one to three orders of mag- nitude. Our results give reason for cautious optimism: While there is still much room for improvement, the behavioural difference between human and machine vision is narrowing. In order to measure future progress, 17 OOD datasets with image-level human behavioural data and evaluation code are provided as a toolbox and benchmark htthtttp://ggiee www.:uu....ooobbbbeeeeeeee. wwom///ehhgela//odeelll vs wws.anannnsaa/a/a///aaaaa
(1.) The longstanding distortion robustness gap be-
most of the investigated OOD datasets. (2.) There is
Introduction
Looking back at the last decade, deep learning has made tremendous leaps of progress by any standard. What started in 2012 with AlexNet [1 as the surprise winner of the ImageNet Acaee Visual Recognition Challenge quickly became the birth of a new AI "summer", a summer lasting much longer than just a season. With it, just like with any summer, came great expectations: the hope that the deep learning revolution will see widespread applications in industry, that it will propel breakthroughs in the sciences, and that it will ultimately close the gap between human and machine
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

perception. We have now reached the point where deep learning has indeed become a significant driver of progress in industry [e.g. B. and where many disciplines are employing deep learning for scientific discoveries bul are we making progress in closing the gap between human and machine vision?
IID vs. OOD benchmarking. For a long time, the gap between human and machine vision was mainly approximated by comparing benchmark accuracies on IID (independent and identically distributed) test data: as long as models are far from reaching human-level performance on challenging datasets like ImageNet, this approach is adequate [IQ]. Currently, models are routinely matching and in many cases even outperforming humans on IID data. At the same time, it is becoming increasingly clear that models systematically exploit shortcuts shared between training and test data [TH44). Therefore we are witnessing a major shift towards measuring model performance on out-of-distribution (OOD) data rather than IID data alone, which aims at testing models on more challenging test cases where there is still a ground truth category, but certain image statisties differ from the training distribution. Many OOD generalisation tests have been proposed: ImageNet-C C [1]] for corrupted imaes, ImageNet-Skenn Sketch [16] for sketches, Stylized-ImageNet
ImageNet [17] for image style changes, [18] for unfamiliar object poses, and many more [1]]2]] While it is great to have many viable and valuable options to measure generalisation, most of these datasets unfortunately lack human comparison data. This is less than ideal, since we can no longer assume that humans reach near-ceiling accuracies on these challenging test cases as they do on standard noise-free III) object recognition datasets. In order to address this issue, we carefully tested human observers in the Wichmannlab's vision laboratory on a broad range of OOD datasets, providing some 85K psychophysical trials across 90 participants. Crucially, we showed exactly the same images to multiple observers, which means that we are able to compare human and machine vision on the fine-grained level of individual images (30 The focus of our datasets is measuring distortion robustness: we tested 17 variations that include changes to image style, texture, and various forms of synthetic additive noise.
Contributions & outlook. The resulting 17 OOD datasets with large-scale human comparion data enable us to investigate recent exciting machine learning developments that crucially deviate from "vanilla" CNNs along three axes: objective function (supervised vs. supervueerv,,,,, adversarially trained, and CLIP's joint language-iman and training dataset size (ranging from 1M 1B images). Taken together, these are some of the most promising directions our field has developed to date-but this field would not be machine learning if new breakthroughs weren't within reach in the next few weeks, months and years. Therefore, we open-sourced modee mnumann Python toolbox that enables testing both PyTorch and TensorFlow models on our comprehensive benchmark suite of OOD generalisation data in order to measure future progress. Even today, our results give cause for (cautious) optimism. After a method overview (Section? 2 we are able to report that the best models now match or in many cases even exceed human feedforward performance on most of the investigated OOD datasets (Section Bp. While there is still a substantial image-level consistency gap between humans and machines, this gap is narrowing on some but not all datasets when the size of the training dataset is increased (Section
training). architecture (convolutional vs. vision transformer)
distortion robustness gap is closing: the
2 Methods: datasets, psychophysical experiments, models, metrics, toolbox
OOD datasets with consistency-gradd human data. We collected human data for 17 generalisa- tion datasets (visualized in Figures 77 and in the Appendix, which also state the number of subjects and trials per experiment) on carefully calibrated screen in dedicated psychophysical laboratory (a total of 85,120 trials across 90 observers). Five datasets each correspond to a single manipulation (sketches, edge-filtered filtered images, silhouettes, images with a texture-shape cue conflict, and stylized images where the original image texture replaced by the style of a painting); the remaining twelve datasets correspond to parametric image degradations (e.g. different levels of noise or blur). Those OOD datasets have in common that they are designed test Ieaaneet- models. OOD images were obtained from different sources: sketches from ImageNet-Skedt Sketch [16), stylized images from

---

[17,, edge-filtered images, silhouettes and cue conflict images from 17 and the
remaining twelve parametric datasets were adapted from [33]. For these parametric datasets, [33 collected human accuracies but unfortunately, they showed different images to different observers implying that we cannot use their human data to assess image-level consistency between humans and machines. Thus we collected psychophysical data for those images ourselves by showing exactly the same images to multiple observers for each of those twelve datasets. Additionally, we cropped the images from 33 to 224 224 pixels to allow for a fair comparison to ImageNet models (all models included our comparison receive 224 x 224 input images: 33 showed 256 x 256 images to human observers in many cases).
Psychophysical experiments. 90 observers were tested in a darkened chamber. Stimuli were presented at the center of a 22" monitor with 1920 x 1200 pixels resolution (refresh rate: 120 Hz). Viewing distance was 107 em and target images subtended x 3 degrees of visual angle. Human observers were presented with an image and asked to select the correct category out of 16 basic categories (such as chair, dog, airplane, etc.). Stimuli were balanced w.r.t. classes and presented in random order. For ImageNet-trainedd models, in order to obtain choice from the same 16 categories, the 1,000 class decision vector was mapped to those 16 classes using the WordNet hierarchy (34). In Appendix
L we explain why this mapping is optimal. We closely followed the experimental protocol defined by [33,, who presented images for 200 ms followed by a 1/f backward mask to limit the influence of recurrent processing (otherwise comparing to feedforward models would be difficult). Further experimental details are provided in Appendix C
Why not use crowdsourcing instead? Our approach of investigating few observers in a high- quality laboratory setting performing many trials is known as the so-called "small-N design", the bread-and- buttterddttteraaea approach in high-qualin psychophysics- see, e.g., the review "Small is beautiful: In defense of the small-N design" (35). This is in contrast to the "crowdsourcing approach" (many observers in noisy setting performing fewer trials each). The highly controlled conditions of the Wichmannlab's s psychophysical laboratory come with many advantages over crowdsourced data collection: precise timing control (down to the millisecond), carefully calibrated monitors (especially important for c.g. low-contrast stimuli), controlled viewing distance (important for foveal presentation), full visual acuity (we performed an acuity test with every observer prior to the experiment), observer attention (e.g. no multitasking or children running around during an experiment, which may happen in crowdsourcing study). just name few [6]. Jointly, these factors contribute to high data quality.
Models. In order to disentangle the influence of objective function, architecture and training dataset size, we tested a total of 52 models: 24 standard ImageNet-trained CNNs (37). 8 self- supervised models
43 43
6 Big Transfer models [5). 5 adversarially trained models [6]. 5 vision
transformers 47 48) two semi-weakly supervised models [49] as well as Noisy Student [50] and CLIP [51] Technical details for all models provided in the Appendix.
Metrics. In addition to OOD accuracy (averaged across conditions and datasets), the following three metries quantify how closely machines are aligned with the decision behaviour of humans.
Accuracy difference A(m) is simple aggregate measure that compares the accuracy of machine m to the accuracy of human observers in different out-of-distribution tests,
A(m) R -> [0, 1), + D Σ
Σ
(acca,e(h) (m))
Ha
Ca
dED
KgHa
cECa
where acca,e()) is the accuracy of the model the human on dataset d E D and condition Ca (e.g. a particular noise level), and h E HD denotes a human observer tested on dataset d. Analogously, one can compute the average accuracy difference between a human observer h and all other human observers by substituting h1 for m and h E HD \ (h1) for E HD (which can also be applied for the two metrics defined below).
For those three datasets consisting of 160, 160 and 1280 images respectively, consistency -rade psy- chophysical data was already collected by the authors and included in our benchmark with permission from the authors.
We presented a preliminary and much less comprehensive version of this work at the NeurIPS 2020 workshop SVRHM [44)
3

---

Aggregated metrics like A(m) ignore individual image-level decisions. Two models with vastly different image-level decision behaviour might still end up with the same accuracies on each dataset and condition. Hence, we include two additional metrics in our benchmark that are sensitive to decisions on individual images.
Observed consistency ((m) [32] measures the fraction of samples for which humans and model m get the same sample either both right or both wrong. More precisely, let (s) be if both a human observer h and m decide either correctly or incorrectly on a given sample s, and zero otherwise. We calculate the average observed consistency as
O(m): : R - 10,11mm
1 Hd
Σ DECd
Σ &ESa.
bl.m
D
Cdl
Sdaee
hCHa
where 8 Sá,e denotes sample (in our case, image) of condition from dataset d. Note that this measure can only zero the accuracy of h and m are exactly the same in each dataset and condition.
Error consistency (m) 32 tracks whether there is above-chance consistency. This is an important distinction, since e.g. two decision makers with 95% accuracy each will have at least 90% observed consistency, even if their 5% eтrors occur on non-overlapping subsets of the test data (intuitively, they both get most images correct and thus observed overlap is high). To this end, error consistency (a. (a.k.a. Cohen's 's kappa, cf. (52]) indicates whether the observed consistency larger than what could have been expected given two independent binomial decision makers with matched accuracy, which we denote as op.m- This can easily be computed analytically (e.g. 32 equation 1]. Then, the average error consistency is given by
DeESa, bn Ôn (Sa.e)
E(m) R + [-1,1,,,
D
Ha
Cal
1 Oh,m (Sd.c)
DED
heHa
CECл
Benchmark & toolbox. A(m), (()) and E(m) each quantify a certain aspect of the human- machine gap. We use the mean rank order across these metries determine overall model ranking (Table in the Appendix). However, we would like to emphasise that the primary purpose of this benchmark is generate insights, winners. Since insights are best gained from detailed plots and analyses, we open-source modelvshuman a Python project to benchmark models against human data Pl The current model zoo already includes 50+ models, and an option add new ones (both PyTorch and TensorFlow). Evaluating a model produces 15+ page report on model behaviour. All plots in this paper can be generated for future models-te track whether they narrow the gap towards human vision, or to determine whether an algorithmic modification to a bascline model (e.g., an architectural improvement) changes model behaviour.
Robustness across models: the OOD distortion robustness gap between human and machine vision is closing
We are interested in measuring whether we are making progress in closing the gap between human and machine vision. For long time, CNNs were unable to match human robustness in terms of generalisation beyond the training distribution large OOD distortion robustness gap [14] 33 53 (5). Having tested human observers on 17 OOD datasets, we are now able to compare the latest developments in machine vision to human perception. Our core results are shown in Figure the OOD distortion robustness gap between human and machine vision is closing IIa lbp, especially for models trained on large-scale datasets. On the individual image level, human-machine consistency gap remains (especially ld]. which will be discussed later.
Self-supervieed models "If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning and the cherry on the cake is reinforcement learning". "
'f course, comparing human and machine vision is not limited to object recognition behaviour: other comparisons may be just as valid and interesting.

---

0.2
accuracy differecce
000 accuraty
nee
oce
OOD accuracy (higher better).
(b) Accuracy difference (lower = better).
observed con consistency
consistency
3
(c) Observed consistency (higher = better).
(d) ErTor consistency (higher better).
Figure 1: Core results, aggregated over 17 out-of-distribution (OOD) datasets: The OOD robustness gap between human and machine vision is closing (top), but an image-level consistency gap remains (bottom). Results compare humans, standard supervised CNNs, self-supervised models, adversarially trained models, vision transformers, noisy student, BiT, SWSL and CLIP. For convenience, 1 marks models that are trained on large-scale datasets. Metries defined in Section 2, Best viewed on screen.
Yann LeCun said in 2016 [56]. A few years later, the entire cake is finally on the table-the representations learned via self-supervised learning [4]
now compete with supervised methods on ImageNet 45) and outperform supervised pre-traine pre-training for object detection [4]. But how do recent self-supervised models differ from their supervised counterparts in terms of their behaviour? Do they bring machine vision closer human vision? Humans, too, rapidly learn to recognise new objects without requiring hundreds of labels per instance; additionally number of studies reported increased similarities between self-supervised models and human perception 67. Figure compares the generalisation behaviour of eight self-supervised models in orange (PIRI, MoCo, MoCoV2, InfoMin, InsDis, SimCLR-x! SimCLR-x22, SimCLR-x4) with 24 standard supervised models (grey). We find only marginal differences between self-supervised and supervised models: Across distortion types, self-supervised networks are well within the range of their poorly generalising supervised counterparts. However, there is one exception: the three SimCLR variants show strong generalisation improvements on uniform noise, low contrast, and high-pas images, where they are the three top- performing self supervuue networks quite remarkable given that SimCLR models were trained different set of augmentations (random crop with flip and resize, colour distortion, and Gaussian blur). Curious by the outstanding performance of SimCLR, asked whether the self-supervised objective function or the choice of training data augmentations was the defining factor. When comparing seff-supervised SimCLR models with augmentation-maln supervised fashion (Figure 15inn in the Appendix), we find that the augmentation scheme (rather than the self-supervised objective) indeed made the crucial difference: supervised baselines show just the same generalisation behaviour, a finding that fits well with [62], who observed that the influence of training data augmentations stronger than the role of architecture or training objective. In conclusion, our analyses indicate that the "cake" of contrastive self sule-supervieed learning currently (and disappointingly) tastes much like the "icing".
baseline models trained in the standard
"Unnuppevvsedd learning" and supervisd learning" are sometimes used interchangeably. We use the term "self-supervised learning"" since those methods (label-free) supervision.
5

---

30.
0.8
06
06
0.4
0.2
0.
(a) Colour greyscale
(b) True vs. false colour
(c)
Power equalisation
Power specinun
caulieed
0.0
(d) Rotation
Rocation argie
90
180
275
6.8
Clesit cacan soursey
9.
94
94
3.0
0.9
35
0.0
150
10 Contrsst pepereett
0.0
0.
3.0
Uniom notue witts
Filtrr standard deviation
Fiter siandard deviation
(e) Uniform noise
(f) Contrast
(g) Low-pass
(h) High
10
6
3.0
30 30 320
150 180
0.
0.
Phane nolie witth
100g reach parameter
Locy readh prameter
(i)
Phase noise
Eidolon I
(k) Eidolon
(1) Eidolon III
Figure 2: The OOD distortion robustness gap between human and machine vision is closing. Robust- ness towards parametric distortions for humans, standard supervised CNNs, self-supervised models, adversarially trained models, vision transformers, noisy student, BiT, SWSL, CLIP. Symbols indicate architecture type (O convolutional, V vision transformer, O human); best viewed on screen.
Adversarially trained models The vulnerability of CNNs to adversarial input perturbations is, arguably, one of the most striking shortcomings of this model class compared to robust human perception. A successful method increase adversarial robustness is adversarial training [e.g. The resulting models were found to transfer better, have meaningful gradients [65]. and enable interpolating between two input images [66]] "robust optimization as inducing a human prior over the features that models are able to learn" 667 10]. Therefore, tradeoffs,
63 64).
actually be viewed
include five models with ResNer-50 architecture and different accuracy-robustnee adversarially trained on ImageNet with Microsoft-scall resources by 46 to test whether models with representations" also show numan-alignar OOD generalisation behaviour-ae as we would hope. This is not the case: the stronger the model is trained adversarially (darker shades of blue in Figure 2 the more susceptible it becomes (random) image degradations. Most strikingly, simple rotation by 90 degrees leads to 50% drop in classification accuracy. Adyersarial robustness seems come the cost of increased vulnerability large-scale perturbations P On the other hand, there is silver lining: when testing whether models are biased towards texture or shape by testing them on cue conflict images (Figure 3). in accordance with [69 [70] we observe perfect relationship between shape bias and the degree of adversarial training, big step in the direction of human shape bias (and a stronger shape bias than nearly other models).
peerceptually-alignee
Vision transformers In computer vision, convolutional networks have become by far the dominat- ing model class over the last decade. Vision transformers [47] break with the long tradition of using convolutions and are rapidly gaining traction (71. We find that the best vision transformer (ViT-I. trained on 14M images) even exceeds human OOD accuracy (Figure Ta shows the average across 17 datasets). There appears be an additive effect architecture and data: vision transformers trained on IM images (light green) are already better than standard convolutional models; training on 14M images (dark green) gives another performance boost. line with 722 we observe a higher shape bias compared to most standard CNNs.
'This might be related to 68 who studied potentially related tradeoff between selectivity and invariance.

---

1.0
Shopp Dias
3000
1993
ves
1993
dencenetie
shulthent yo 20
resroxtioo 30x00
wide noir norrato
monmet
Ind Resile
Medac Ies
PiR:: Bed
MocoVZ: Rostee
Intoltn: Restie
Sincla Roskee 50x0
Sindutts lee s002
simci Restee-OOat
Resfiet 50 L2 ass 0.0
Restte 50 12 eps 0.5
1.0-
Res epx
eps 5.0
VTT
VIT
SUnE men 301 pp-tot
SWELL Restee (4000
us Me Rustee US LL-M0-
co 114M0
na th mite 11-06-
arT Reshee Str (1AM0-
VP-4 (3AM)-
vir-a [1 116A1-
Noay Studeot: mhett Onoe
VIT (400
humana
Resttee 12
Resites 50
Figure 3: Shape vs. texture biases of different models. While human shape bias is not yet matched,
several approaches improve over vanilla CNNs. Box plots show category-depende distribution of shape/ / texture biases (shape bias: high values, texture bias: low values).
Standard models trained on more data: BiT-M, SWSL, Noisy Student Interestingly, the biggest effect on OOD robustness we find simply comes from training on larger datasets, not from advanced architectures. When standard models are combined with large-scale training (14M images for BiT-M, 300M for Noisy Student and a remarkable 940M for SWSL), OOD accuracies reach levels not known from standard ImageNet-trained models; these models even outperform more powerful architecture (vision transformer ViT-S) trained on less data (IM) as shown in Figure Ta| Simply training on (substantially) more data substantially narrows the gap to human OOD accuracies (Tb). finding that we quantified in Appendix by means of a regression model. (The regression model also revealed a significant interaction between dataset size and objective function, well as significant main effect for transformers over CNNs.) Noisy Student in particular outperforms humans by a large margin Ta the beginning of new mumanne gap, this time in favour of machines?
overall (Figure
CLIP CLIP is special: trained on 400M images (more data) with joint ainguge--iaae supervision (novel objective) and vision transformer backbone inon-standard architecture), it scores close to humans across all of our metrics presented in Figure I. most strikingly in terms of error consistency (which will be discussed in the next section). We tested number of hypotheses to disentangle why CLIP appears "special". HI: because CLJP is trained on a lot of data? Presumably no: Noisy Student-a model trained
comparably large dataset of 300M OOD accuracy, but poorly on error consistency. A caveat in this comparison is the quality of the labels: while Noisy Student uses pseudolabeling, CLIP receives web-based labels for all images. H2: because CLIP receives higher-quality labels? About 6% of ImageNet labels are plainly wrong 774.. Could be the case that CLIP simply performs better since it doesn't 't suffer from this issue? In order to test this, we used CLIP to generate new labels for all 1.3M ImageNet images: (a) hard labels, i.c. the top-1 class predicted by CLIP: and (b) soft labels, i.c. using CLIP's full posterior distribution as a target. We then trained ResNet-50 from scratch on CLIP hard and soft labels (for details see Appendix D. However, this does not show any robustness improvements over a vanilla ImateNet-trained ResNet-50, thus different/better labels are not likely root cause. H3: because CLIP has a special image+text loss? Yes and no: CLIP training on ResNet-50 leads astonishingly poor OOD results, so training a standard model with CLIP loss alone is insufficient. However, while neither architecture nor loss alone sufficiently explain why CLIP is special, we find a clear interaction between architecture and loss (described in more detail in the Appendix along with the other "CLIP ablation" experiments mentioned above).
perefors very well on
4 Consistency between models: data-rich models narrow the substantial image-level consistency gap between human and machine vision
In the previous section we have seen that while self-supervised and adversarially trained models lack OOD distortion robustness, models based on vision transformers and/or trained on large-scale datasets now match or exceed human feedforward performance on most datasets. Behaviourally, a
'The boundary between IID and OOD data is blurry for networks trained on big proprietary datasets. We consider it unlikely that CLIP was exposed to many of the exact distortions used here (e.g. eidolon or cue conflict images), but CLIP likely had greater exposure some conditions such grayscale or low- low-contrast images.
7

---

VIT (ImageNet) self- self-supervised standard supervised CNNS
0013 30
50
shufflenct v2 x
resnextso 22840
wide resnis5o
0.8
IndDi Bashes-50
PinL: Rastui-s
etror consistenc
InfoMee: Restut-5o
adv. trained
Samcuut Reshet-30 50x
Destiee 50 wps
Desdlet-9 50 eps
(esdlet-53 1.2 aps 5.0
VIT-BI
humans
subject-11
subject-d
subject-d5
data rich models
subject-d7
SWS1: Resheet-100 101 (S40M)
B.T.: Resdiet 152bc (1401)
B1T-4t Reddict- 101x3 (2401)
GIT-M: Restiet 50x3 (2,60))
VT--L (34K)
Volty Student: ENet12 (300M)
Noby Student, ENeLL (R00
30 HIN ein
DOTT CP TH00 Bo BfT Best BITTN sest Bi
Figure 4: Data-rich models narrow the substantial image-level consistency gap between humans and machines. Error consistency analysis on a single dataset (sketch images; for other datasets see Appendix, Figures 9 errors) irrespective of their architecture and objective function; humans cluster differently (high human-to-human consistency, low human-to- model consistency); but some data-rich models including CLIP and SWSL blur the boundary, making more human-like errors than standard models.
shows that most models cluster (dark red highly consistent
natural follow-up question is to ask not just how many, but which errors models make-iae do they make errors on the same individual images as humans OOD data (an important characteristic of a "human-like model, cf. [32 ? This is quantified via error consistency (defined in Section [ which additionally allows to compare models with each other, asking e.g. which model classes make similar errors. In Figure 4 we compare all models with each other and with humans, asking whether they make errors on the same images. On this particular dataset (sketch images), we can see one big model cluster. Irrespective of whether one takes a standard supervised model, a self- supervised model, adversarially trained model or vision transformer, all those models make highly systematic errors (which extends the results of [32 76] who found similarities between standard vanilla CNNs). Humans, on the other hand, show very different pattern of errors. Interestingly, the boundary between humans and some data-rich models at the bottom of the CLIP (400M images) and SWSL (940M) is blurry: some (but not all) data-rich models much more closely mirror the patterns of errors that humans make, and we identified the first models to achieve higher error consistency with humans than with other (standard) models. Are these promising results shared across datasets, beyond the sketch images? In Figures Ic and [d] aggregated results over 17 datasets are presented. Here, we can see that data-rich models approach numan-toohumman observed consistency, but not error consistency. Taken in isolation, observed consistency is not good measure of image-level consistency since it does not take consistency by chance into account; error consistency tracks whether there is consistency beyond chance; here we see that there is still
8

---

0.5
0.40
0.35
0.4
0.30
efor consistency
0.3
errer conaistency
0.25
0.20
0.
0.15
0.10
0.1
0.05
0.0
expected
0.00
expected
0.45
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.2
0.4
0.6
OOD accuracy
0D0 accurary
(a) 12 "disappointing" datasets
(b) 5 "promising" datasets
Figure 5: Partial failure, partial success: Error consistency with humans aggregated over multiple datasets. Left: 12 datasets where model accuracies exceed human accuracies; here, there is still substantial image-level consistency gap to humans. Right: 5 datasets (sketch, silhouette, edge, cue conflict, low-pass, where humans are more robust. Here, OOD accuracy is a near near-perfect predictor of image-level consistency; especially data-rich models (e.g. CLIP, SWSL, BiT) narrow the consistency gap to humans. Symbols indicate architecture type (O convolutional, v vision transformer, ! human).
substantial image-level consistency gap between human and machine vision. However, several models improve over vanilla CNNs, especially BiT-M (trained on 14M images) and CLIP (400M images). This progress is non-trivial; at the same time, there is ample room for future improvement.
How do the findings from Figure (showing nearly human-level error consistency for sketch images) and from Figure [d (showing substantial consistency gap when aggregating over 17 datasets) fit together? Upon closer inspection, we discovered that there are two distinct cases. On 12 datasets (stylized, colour/greyscale, contrast, high-pass, phase-scrambling, power-equalisation, false colour, rotation, eidolonl, -II and -III as well as uniform noise), the human- machine gap is large; here, more robust models do not show improved error consistency (as can be in Figure Sa On the other hand, for five datasets (sketch, silhouette, edge, cue conflict, low-pass filtering). there completely different result pattern: Here, OOD accuracy is near-perfect predictor of error consistency, which means that improved generalisation robustness leads to more human-like errors (Figure 5b). Furthermore, training on large-scale datasets leads to considerable improvements along both axes for standard CNNs. Within models trained on larger datasets. CLIP scores best; but models with standard architecture (SWSL: based on ResNet-50 and ResNeXt-101) closely follow suit.
remains an open question why the training dataset appears to have the most important impact on model's decision boundary as measured by error consistency (as opposed to other aspects of model's 's inductive bias). Datasets contain various shortcut opportunities [14]. and two different models trained on similar data, they might converge
similar solution simply by exploiting the same shortcuts-which would also fit well to the finding that adversarial examples typically transfer very well between different models [77 78) Making models more flexible (such as transformers, a generalisation of CNNs) wouldn't i change much in this regard, since flexible models can still exploit the same shorteuts. Two predictions immediately follow from this hypothesis: (1.) error consistency between two identical models trained on very different datasets, such as ImageNet vs. is much lower than error consistency between very different models (ResNet-50 vs. VGG-16) trained on the same dataset. (2.) eтror consistency between ResNet-50 and highly flexible model (e.g., vision transformer) much higher than eтror consistency between Re and a highly constrained model like BagNet-9 79). We provide evidence for both predictions in Appendix B which makes the shortcut hypothesis of model similarity a potential starting point for future analyses. Looking forward, it may be worth exploring the links between shorteut learning and image difficulty, such as understanding whether many "trivially easy" images in common datasets like ImageNet causes models to expoit the same characteristics irrespective of their architecture (80).

---

5 Discussion
Summary We set out to answer the question: Are we making progress in closing the gap between human and machine vision? In order to quantify progress, we performed large-scale psychophysice experiments on 17 out-of-distribtiion distortion datasets (open-sourced along with evaluation code as a benchmark to track future progress). We then investigated models that push the boundaries of traditional deep learning (different objective functions, architectures, and dataset sizes ranging from 1M to IB), asking how they perform relative to human visual perception. We found that the OOD distortion robustness gap between human and machine vision is closing, as the best models now match or exceed human accuracies. At the same time, an image-level consistency gap remains; however, this gap that least in some cases narrowing for models trained on large-scale datasets.
Limitations Model robustness is studied from many different viewpoints, including adversarial robustness [77], theoretical robustness guarantees [e.g.81]] or label noise robustness [e.g. 82]. The focus of our study is robustmess towards non-adversarial out-o--distribution data, which is particularly well-suited for comparisons with humans. Since we aimed at a maximally fair comparison between feedforward models and human perception, presentation times for human observers were limited to 200 ms in order to limit the influence of recurrent processing. Therefore, human ceiling performance might be higher still (given more time); investigating this would mean going beyond "core object recognition". which happens within less than 200 ms during a single fixation [83]. Furthermore, human and machine vision can be compared in many different ways. This includes comparing against neural data [84 [5 contrasting Gestalt effects [e.g. 86), object similarity judgments [87), or mid-level properties [6]] and of course not limited to studying object recognition. By no means do we mean to imply that our behavioural comparison is the only feasible option- on the contrary. we believe it will be all the more exciting to investigate whether our behavioural findings have implications for other means of comparison!
Discussion We have to admit that we view our results concerning the benefits of increasing dataset size by one-to-three orders of magnitude with mixed feelings. On the one hand, "simply" training stan- dard models on (a lot) more data certainly has an intellectually disappointing given many rich ideas in the cognitive science and neuroscience literature which architectural changes might be required to bring machine vision closer to human vision (3). Additionally, large-scale training comes with infrastructure demands that are hard meet for many academic researchers. On the other hand, we find it truly exciting to see that machine models are closing not just the OOD distortion robustness gap humans, but that also, at least for some datasets, those models are actually making more human-like decisions on an individual image level; image-level response consistency
caannent--prticllaati
much stricter behavioural requirement than just c.g. matching overall accuracies. Taken together, our results give reason to celebrate partial success in closing the gap between human and machine vision. In those cases where there is still ample room for improvement, our psychophysical benchmark datasets and toolbox may prove useful in quantifying future progress.
Acknowledgments and disclosure of funding
We thank Andreas Geiger, Simon Kornblith, Kristof Meding, Claudio Michaelis and Ludwig Schmidt for helpful discussions regarding different aspects of this work: Lukas Huber, Maximus Mutschler, David-Elias Künstle for feedback on the manuscript: Ken Kahn for pointing out typos: Santiago Cadena for sharing a PyTorch implementation of SimCLR: Katherine Hermann and her collaborators for providing supervised SimCLR baselines; Uli Wannek and Silke Gramer for infrastructure/admmiitttaa their models publicly available; and our anonymous reviewers for many valuable suggestions.
support; the many authors who made
Furthermore, we are grateful to the International Max Planck Research School for Intelligent Systems (IMPRS- IS) for supporting R.G.; the Collaborative Research Center (Projektnummer 276693517- SFB 1233: Robust Vision) for supporting M.B. and F.A.W. W. This work was supported by German Federal Ministry of Education and Research (BMBF): Tübingen AI Center, FKZ: 011118039 (W.B. and M.B.). F.A. W. is member of the Machine Learning Cluster Excellence, EXC number 1ro4//1-rooeect number 390727645. M.B. and W.B. acknowledge funding from the MICTONS program of the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/Interior Business Center (Dol/IBC) contract number D16PC00003. W.B. acknowledges financial support via the Emmy Noether Research Group on The Role of Strong Response Consistency for Robust and Explainable Machine Vision funded by the German Research Foundation (DFG)
under grant no. BR 6382/1-1.
10

---

Author contributions
Project idea: R.G. and W.B.; project lead: R.G.; coding toolbox and model evaluation pipeline: R.G., K.N. and B.M. based on a prototype by R.G.; training models: K.N. with input from R.G., W.B. and M.B.; data visualisation: R.G., B.M. and K.N. with input from M.B., F.A. W. and W.B.; psychophysical data collection: T.T. (12 datasets) and B.M. (2 datasets) under the guidance of R.G. and F.A.W. curating stimuli: R.G.; interpreting analyses and findings: R.G., M.B., F.A.W. and W.B.; guidance, feedback, infrastructure & funding acquisition: M.B., F.A.W. W. and W.B.; paper writing: R.G. with help from F.A.W. and W.B. and input from all other authors.
References
[1] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Sysiems, pages 1097-1105, 2012.
[2] Jinjiang Wang. Yulin Ma, Laibin Zhang, Robert X Gao, and Dazhong Wu. Deep learning for smart manufacturing: Methods and applications. Journal of Manufacturing Systems, 48:144-156, 2018.
[3] Javier Villalba-Diez, Daniel Schmidt, Roman Gevers, Joaquín Ordieres- Meré, Martin Buchwitz, and Wanja Wellbrock. Deep learning for industrial computer vision quality control in the printing industry 4.0. Sensors, 19(18)39877, 2019.
[4] Christof Angermueller, Tanel Pämamaa, Leopold Parts, and Oliver Stegle. Deep learning for computational biology. Molecular Systems Biology, 12(7):878, 2016.
[5] Adam H Marblestone, Greg Wayne, and Konrad P Kording. Toward an integration of deep learning and neuroscience. Frontiers in Computational Neuroscience, 10:94, 2016.
[6] Garrett B Goh, Nathan O Hodas, and Abhinav Vishnu. Deep learning for computational chemistry. Journal 2017.
of Computational Chemistry, 38(16):1211-))
[7] Travers Ching, Daniel S Himmelstein, Brett K Beaulicu-lones Alexandr A Kalinin, Brian T Do, Gregory P Way, Enrico Ferrero, Paul-Mine Michael Agapow, Michael Zietz, Michael M Hoffman, et al. Opportumities and obstacles for deep learning in biology and medicine. Journal of The Royal Society Interface, 15(141): 20170387, 2018
[8] Dan Guest, Kyle Cranmer, and Daniel Whiteson. Deep learning and its application to LHC physics. Annual Review of Nuclear and Particle Science, 68:161-181, 2018.
[9] Andrew W Semior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Židek, Alexander WR Nelson, Alex Bridgland, et al. Improved protein structure prediction using potentials from deep learning. Nature, 577(7792):706-77 710, 2020.
[10] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander Berg, and Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 115(3):21 1-252, 2015.
[11] Jason Jo and Yoshua Bengio. Measuring the tendency of enns to learn surface statistical regularities. arXiv preprint arXiv:1711.1 11611, 2017.
[12] Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European Conference on Computer Vision, pages 456-473, 2018
[13] Timothy Niven and Hung- Yu Kao. Probing neural network comprehension of natural language arguments. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4658 4664, 2019.
[14] Robert Geirhos. Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2:665 673, 2020.
[15] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019.
[16] Haohan Wang, Songwei Ge, Eric P Xing, and Zachary C Lipton. Learning robust global representations by penalizing local predictive power. arXiv preprint ArXiv:9905. 13549, 2019.
[17] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. ImageNet-traiine CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference Learning Representations, 2019.
11

---

[18] Michael A Alcor, Qi Li. Zhitao Gong, Chengfei Wang. Long Mai, Wei-Shinn Ku, and Anh Nguyen. Strike (with) pose: Neural networks are easily fooled by strange poses of familiar objects. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, 2019.
[19] Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through prohabilistic program induction. Science, 25006266):1332-13333 2015.
[20] Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang. Dan Gutfreund, Josh Tenenbaum, and Boris Katz. ObjectNet: a large-scale bias-controlled dataset for pushing the limits of object recognition models. In Advances in Neural Information Processing Systems, pages 9448 9458, 2019.
[21] Claudio Michaelis. Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S Ecker, Matthias Bethge, and Wieland Brendel. Benchmarking robustness in object detection: Autonomous driving when winter is coming. arXiv: /007:0997.07700 2019.
[22] Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Mea- suring robustness to natural distribution shifts image classification. In Advances in Neural Information Processing Systems, 2020.
[23] Josip Djolonga, Jessica Yung. Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D' Amour, Dan Moldovan, et al. On robustness and transferability of convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 16458-16468, 2021.
[24] Isaac Dunn, Hadrien Pouget, Daniel Kroening, and Tom Melham. Exposing previously undetectable faults in deep neural networks. arXiv preprint arXiv:2106.0 00576, 2021.
[25] Yue He, Zheyan Shen, and Peng Cui. Towards non-iid image classification: A dataset and baselines. Pattern Recognition, 110: 107383, 2021.
[26] Dan Hendryeks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEÉE Conference on Computer Vision and Puttern Recognition, pages 8340-8349, 2021.
[27] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 12262-15271, 2021.
[28] Spandan Madan, Tomotake Sasaki, Tzu-Mao Li, Xavier Boix, and Hanspeter Pfister. Small in-distribution changes in 3D perspective and lighting fool both CNNs and transformers. arXiv preprint arXiv:2106.1 16198, 2021.
[29] Vaishaal Shankar, Achal Dave, Rebecca Roelofs, Deva Ramanan, Benjamin Recht, and Ludwig Schmidt. Do image classifiers generalize across time? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9661-9669, 2021.
[30] David M. Green. Consistency of auditory detection judgments. Psychological Review, (155)392--407, 1964.
[31] Kristof Meding, Dominik Janzing, Bernhard Schölkopf, and Felix A. Wichmann. Perceiving the arrow of time in autoregressive motion. Advances in Neural Information Processing Systems (NeurIPS), 32: 2303-2314, 2019.
[32] Robert Geirhos, Kristof Meding, and Felix A Wichmann. Beyond accuracy: quantifying trial-by-trial be- haviour CNNs and humans by measuring error consistency. Advances in Neural Information Processing Systems, 33, 2020.
[33] Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko Schüit, Matthias Bethge, and Felix Wichmann. Generalisation in humans and deep neural networks. In Advances in Neural Information Processing Systems, 2018.
[34] George A Miller. WordNet: a lexical database for English. Communications of the ACM, 38(1 38(11):39-4) 1995.
[35] Philip L Smith and Daniel R Little. Small is beautiful: In defense of the small-N design. Psychonomic Bulletin & Review, 25(6))2083-211 2101,
12

---

[36] Siavash Haghiri, Patricia Rubisch, Robert Geirhos, Felix Wichmann, and Ulrike von Luxburg. Comparison- based framework for psychophysics: lab versus crowdsoureing, arXiv preprini arXiv:1905.07234, 2019.
[37] Sébastien Marcel and Yann Rodriguez. Torchvision the machine-vision package of torch. In Proceedings of the 18th ACM International Conference on Multimedia, pages 1485-1488. 2010.
[38] Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non- parametric instance discrimination. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 3733-3742, 2018.
[39] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Gitshick. Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages
2020.
[40] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint wrXi::0033004299 2020.
[41] Ishan Misra and Laurens van der Maaten. Self supersupe learning of pretext-invarian representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6707-6717,
2020.
[42] Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning, arXiv preprint arXiv:2005. 2020.
[43] Ting Chen, Simon Korblithh, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709, 2020.
[44] Robert Geirhos, Kantharaju Narayanappa. Benjamin Mitzkus, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. On the surprising similarities between supervised and self-supervised models. arXiv preprint arXi::2010.08377 2020.
[45] Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung. Sylvain Gelly, and Neil Houlsby. Big transfer (BiT): General visual representation learning. arXiv preprint 111.::191. 6 (2):8, 2019.
[46] Hadi Salman. Andrew Ilyas. Logan Engstrom. Ashish Kapoor, and Aleksander Madry. Do adversarially robust ImageNet models transfer better? arXiv preprint arXiv:2007.08455 2020.
[47] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner. Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.1190 2020.
[48] Ross Wightman.
PyTorch image
models.
https://github..com//zu
pytor sytorch-image-odeee image-models
2019.
[49] Zeei Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. Billion-scale sumi-super learning for image classification. arXiv preprint /90500054 2019.
[50] Qizhc Xie, Minh-Thang Luong, Eduard Hovy, and Quoc Le. Self-training with noisy student im- proves ImageNet classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 10687-10698, 2020.
[51] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.0000 2021.
[52] Jacob Cohen, A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37-46, 1960.
[53] Samuel Dodge and Lina Karam. study and comparison of human and deep learning recognition performance under visual distortions. In 26th International Conference on Computer Communication and Networks, pages 1 1-7. IEEE, 2017.
[54] Felix A Wichmana, David HJ Janssen, Robert Geirhos, Guillermo Aguilar, Heiko H Schütt, Marianne Maertens, and Matthias Bethge. Methods and measurements to compare men against machines. Electronic Imaging, Human Vision and Electronic Imaging. 2017(14):36 45, 2017.
[55] Thomas Serre. Deep learning: the good, the bad, and the ugly. Annual Review of Vision Science, 2019.
13

---

[56] Yann LeCun. Predictive learning. 2016. htt.ttt:t//..y.oottttt wwwpatuub..co///ttccca-ouuuuuuaa
[57] William Lotter, Gabriel Kreiman, and David Cox. A neural network trained for prediction mimics diverse features of biological neurons and perception. Nature Machine Intelligence, 2(4):210-219, 2020.
[58] A Emin Orhan, Vaibhav V Gupla, and Brenden M Lake. Self-supervised learning through the eyes of a child, arXiv preprint 20t.:::0007.16888 2020.
[59] Talia Konkle and George A Alvarez. Instance-level contrastive learning yields human brain-like represen- tation without superviiss...i.inn... bioRxiv, 2020.
[60] Chengxu Zhuang, Siming Yan, Aran Nayebi, Martin Schrimpf, Michael Frank, James DiCarlo, and Daniel Yamins. Unsupervised neural network models of the ventral visual stream. bioRxiv, 2020.
[61] Katherine R Stors, Barton L Anderson, and Roland W Fleming. Unsupervised learning predicts human perception and misperception of gloss. Nature Human Behaviour, pages 1-16, 2021.
[62] Katherine Hermann, Ting Chen, and Simon Kornblith. The origins and prevalence of texture bias in convolutional neural networks. Advances in Neural Information Processing Systems, 33, 2020.
[63] Ian Gondfellow, Jonathon Shlens, and Christian Szegedy. Explaining and hamessing adversarial examples. arXiv preprint urXiv:1412.6577 2014.
[64] Ruitong Huang. Bing Xu, Dale Schuurmans, and Csaba Szepesvári. Learning with a strong adversary. arXiv preprint (tii::5111030344 2015.
[65] Simran Kaur, Jeremy Cohen, and Zachary C Lipton. Are perceptually-aligned gradients general property of robust classifiers? arXiv preprint 1911v:1910.08640, 2019.
[66] Shibani Santurkar, Dimitris Tsipras, Brandon Tran, Andrew Ilyas, Logan Engstrom, and Aleksander Madry. Image synthesis with a single (robust) classifier. 0rXiv:1906.090 2019.
[67] Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Aleksander Madry, Adversarial robustness as prior for learned representations. arXiv preprint 1906v 00945, 2019.
[68] Florian Tramèr, Jens Behrmann, Nicholas Carlini, Nicolas Papernot, and Jörn-Henrie Jacobsen. Fundamen- tal tradeoff's between invariance and sensitivity to adversarial perturbations. In International Conference on Machine Learning, pages 9561-9571. PMLR, 2020.
[69] Tianyuan Zhang and Zhanxing Zhu. Interpreting adversarially trained convolutional neural networks. In International Conference on Machine Learning, pages 7502-7511. PMLR, 2019.
[70] Peijic Chen, Chirag Agarwal, and Anh Nguyen. The shape and simplicity biases of adversarially robust ImageNet-trained CNNs. arXiv preprint arXiv:2006.09377 09373, 2020.
[71] Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo. Zhenhua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, et al. A survey on visual transformer. arXiv preprint arXiv:2012.12550, 2020.
[72] Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-HSsuan Yang. Intriguing properties of vision transformers. arXiv preprint arXiv:2105. 10497, 2021.
[73] Shikhar Tuli, Ishita Dasgupta, Frin Grant, and Thomas I. Griffiths. Are convolutional neural networks transformers more like human vision? arXiv preprint 1rxti:2105.07199 2021.
[74] Curtis G Northeutt, Anish Athalye, and Jonas Mueller. Pervasive label errors in test sets destabilize machine learning benchmarks. arXiv preprint arXiv:2103. 14749, 2021.
[75] Rishi Rajalingham, Elias Issa, Pouya Bashivan, Kohitij Kar, Kailyn Schmidt, and James J DiCarlo. Large-scale, high-resolution comparison of the core visual recognitio behavior of humans, monkeys, and state-of thr- deep artificial neural networks. Journal of Neuroscience, 38(33):7255 -266, 2018.
[76] Horia Mania, John Miller, Ludwig Schmidt, Moritz Hardt, and Benjamin Recht. Model similarity mitigates test set overuse. Advances in Neural Information Processing Systems, 2019.
[77] Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. urXiv:1312.6199, 2013.
[78] Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Bonch, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint arXiv:1704.0355, 2017.
14

---

[79] Wieland Brendel and Matthias Bethge. Approximating CNNs with bag-of-locallfeeture models works surprisingly well ImageNet. In International Conference Learning Representations, 2019.
[80] Kristof Meding, Luca M Schulze Buschoff, Robert Geirhos, and Felix A Wichmann. Trivial or impossibee-dcchotomool dichotomous data difliculty masks model differences (on ImageNet and beyond). arXiv preprint arXiv:21 2021.
[81] Matthias Hein and Maksym Andriushchenko. Formal guarantees the robustness of classifier against adversarial manipulation. arXiv preprint arXiv: 074.04555 2017.
[82] Arash Vahdat. Toward robustness against label noise in training deep discriminative neural networks. arXiv preprint arXiv: 00038, 2017.
[83] James J DiCarlo, Davide Zoccolan, and Nicole Rust. How does the brain solve visual object recognition? Neuron, 73(3):115 434,, 2012.
[84] Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James DiCarlo. Perfornance-opimmized hierarchical models predict neural responses higher visual cortex. Proceedings of the National Academy of Sciences, 11 (1(()))8611 19-8644, 2014.
[85] Jonas Kubilius, Martin Schrimpf. Kohitij Kar, Rishi Rajalingham. Ha Hong, Najib Majaj, Elias Issa, Pouya Bashivan, Jonathan Prescott- Roy, Kailyn Schmidt, et al. Brain-like object recognition with high-performing shallow recurrent ANNs. Advances in Neural Information Processing Systems, 32: 12800-122816, 2019.
[86] Been Kim, Emily Reif, Martin Wattenberg, Samy Bengio, and Michael C Mozer. Neural networks trained on natural scenes exhibit Gestalt closure. Computational Brain & Behavior, pages 1 1-13, 2021.
[87] Martin N Hebart, Charles Y Zheng, Francisco Pereira, and Chris Baker. Revealing the multidimen- sional mental representations of natural objects underlying human similarity judgements. Nature Human Behaviour, 4(1 (11):1173-1183 2020.
[88] Nikolaus Kriegeskorte. Deep neural networks: a new framework for modeling biological vision and brain information processing. Annual Review of Vision Science, 1:417-446, 2015.
[89] Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. Building machines that learn and think like people. Behavioral and Brain Sciences, 40, 2017.
[90] Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh. Timothée Masquelier. and Anthony Maida. Deep learning in spiking neural networks. Neural Networks, 111:47-63, 2019.
[91] Fabian H Sinz, Xaq Pitkow, Jacob Reimer, Matthias Bethge, and Andreas S Tolias. Engineering less artificial intelligence. Neuron, 103(6))967-977 2019.
[92] Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David D Cox, and James J DiCarlo. Simulating primary visual cortex at the front of CNNs improves robustness image perturbations. bioRxiv, 2020.
[93] Benjamin Evans, Gaurav Malhotra, and Jeffrey S Bowers. Biological convolutions improve dnn robustness to noise and generalisation. bioRxiv, 2021.
[94] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. PyTorch: An imperative style, high-perfornance deep learning library. In Advances in Neural Information Processing Systems, pages 8026-8037, 2019.
[95] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXivv1807.03722 2018
[96] Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: a large-scale hicrarchical image database. In JEEE Conference on Computer Vision and Puttern Recognition, pages 248-255, 2009.
197] Evgenia Rusak, Steffen Schneider, Peter Gehler, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Adapting ImageNet-scale models to complex distribution shifts with self-learning. arXiv preprint arXiv:2104. 12928, 2021.
[98] Eleanor Rosch. Principles of categorization. In E. Margolis and S. Laurence, editors, Concepts: core readings, pages 189-206. 1999.
15


Reviews:
[{"review_id": "acFFa_vKxf", "reviewer": "Reviewer_m3s7", "summary": "The authors performed a large set of behavioral experiments to measure human psychophysical performance on an image classification task using a broad range of out-of-distribution datasets. They then evaluated a pool of convolutional neural networks (CNNs) on how well they matched human judgements and observed that while there is still a big gap between human and machine vision, models trained on very large datasets are starting to close that gap. The main contribution of the study is the release of the behavioral data for benchmarking models of computer vision. EDIT: Score updated 5 -> 6\n", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is very well written, clear, and addresses one of the most important problems in computer vision: the difference in robustness between human and machine vision. It does so by directly comparing the performance of models in several out-of-distribution datasets with human psychophysics. Despite this, I have some concerns\n\nThis paper follows a trend of studies comparing human behavioral performance on an object recognition task with machine vision models. Studies such as  Rajalingham et al 2018 The Journal of Neurosciences, Geirhos et al 2018 NeurIPS, Geirhos et al. 2019 ICLR and Geirhos et al 2020 NeurIPS have all compared CNN performance with humans in different datasets. In that sense this study can be seen not entirely original but more as an extension of this line of research. It is difficult to see what is exactly original in this study since most of the comparisons with human behavior described here had already been done elsewhere: there is a large overlap with datasets from Geirhos et al 2018 and Geirhos et al 2020.\n\nFurthermore, the methodology that the authors have used is also not without issues. To compare CNN models with 1,000 classes outputs with human behavioral performance in a task with 16 object categories, the authors have used a simple fixed behavioral decoder to map these different output spaces using the WordNet hierarchy. Other researchers (e.g. Rajalingham et al 2018) have used different strategies to decode behavioral choices from the model features and it is not clear which option is better. Even if the authors do not implement other strategies for decoding behavioral outputs from the models, they should at least discuss this in the paper and justify their choice. \n\nFinally, it is not clear in which format the datasets will be made available. While I salute the authors for providing an evaluation platform to facilitate access to the benchmarks described here, it would also be important that the authors provide the raw behavioral data as well as the stimuli used so that other researchers can implement alternative metrics for comparing models to humans. \n\nMinor issues:\nI was surprised to see the omission of a citation to Rajalingham et al 2018, as it was one of the first studies to perform a detailed comparison between models to human behavioral performance in an object recognition task.\n\nEDIT: Score updated 5 -> 6\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors properly addressed some of the study's limitations. The authors do not address potential negative societal impact since these are non-existent. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "WjsKuS14_2x", "reviewer": "Reviewer_AEmo", "summary": "The authors employ a large-scale comparison between human classification behavior and a diverse range of machine vision models on 17 out-of-distribution (OOD) datasets as a way of attempting to comprehensively measure whether the field is making progress in closing the human-machine gap in this domain. They find that particular architectures and training objectives, as well as larger training datasets appear to be increasingly closing the gap when measuring OOD accuracy overall, but less so at the level of error patterns for individual images. They further identify subsets of their OOD task set where the human-machine gap is narrowing versus those where it is not.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While many smaller experiments have previously compared humans to CNNs, the current work is one of the most ambitious and comprehensive comparisons to date, including many architectures that have received far less study in the cognitive sciences. The conclusions are interesting, and the task suite is said to be planned for public release, which would be a very useful tool for the community.\n\nGiven the breadth of the work, it's hard to complain about many of the details. My only real concern was that the conclusions may be severely limited by the behavioral dataset used. That is, the authors used a single experimental paradigm wherein a very small set of laboratory subjects provide a larger number of judgements for a much smaller range of classes than the models which are points of comparison. There are existing behavioral datasets that are larger and pool information across many more annotators to assess image-level error patterns without having to alter the categorization task (e.g., CIFAR-10H). Other datasets focus on much richer OOD problems (ObjectNet, Omniglot) that may elicit very different human-machine deviations. For these reasons, I would have gladly traded some of the depth in model exploration in the paper for additional depth in behavioral comparison.\n\nIn the end, I have little doubt that the work represents an important analysis of the field, but I don't agree that it provides the so-called \"missing human baseline\", nor that datasets employed allow one to draw conclusions about the current human-machine vision gap beyond the sort of image augmentations explored by the authors.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors acknowledge many limitations and there doesn't appear to be much potential for negative social impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "Ggga_wruH1", "reviewer": "Reviewer_bjmw", "summary": "This paper addresses a very interesting and important question: to what extent the recent improvements in object recognition models make them closer to human perception. The authors do this by collecting human data on a wide range of ood benchmarks modeled after ImageNet, evaluating the behavior of a large number of recently developed ImageNet models on these benchmarks, and comparing them with the human responses. The authors also open-source all their data and analysis tools so people can evaluate their own models on the same benchmarks and compare them with human behavior in a fine-grained manner (on an image-by-image basis).", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think this is overall a solid paper packed with important and, to my knowledge, novel contributions and insights:\n\n1) The human data itself and the open-source tools made available for evaluating and analyzing new models are very useful contributions.\n\n2) The authors show that several recent models make significant progress in closing the gap with human behavior. This seems to be true even with respect to the most stringent comparison metric: i.e. image-by-image error consistency with humans. The most important contributor to this progress seems to be pretraining on large datasets (e.g. CLIP, SWSL, BiT).\n\n3) Self-supervised models do not perform any differently from their supervised counterparts when differences in data augmentation pipelines are accounted for.\n\n4) Transformer models in general appear to be better than comparable convnets both in terms of ood performance and in terms of consistency with human observers.\n\nPerhaps, one small suggestion for improvement I can make is some kind of factorial quantitative analysis of the effects of different factors considered here both on ood performance and on consistency with human observers. Currently the authors are making these judgements qualitatively by looking at some plots and concluding, for example, “models pre-trained on large datasets perform better on ood benchmarks and are more consistent with human observers” or “transformers perform better than convnets” etc., which is fine, but I think a more quantitative, factorial analysis would be even better, because, among other things, it would, for example, also potentially give us more information about the interactions and comparative effect sizes.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I think the authors do discuss the limitations of this work adequately in their discussion section. For me, the most important limitation is that static object recognition is an important, but limited facet of human perception. Extending this kind of detailed analysis to a broad range of other human perceptual skills in the future would be important. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "4z__SZ7vOde", "reviewer": "Reviewer_6Nw6", "summary": "This paper compares human and model performance on a variety of OOD benchmarks (e.g., texture-swapped images, blurred images, silhouettes, sketches) and for a variety of the most recent models (eg., CLIP, ViT, BiT, SimCLR) and shows that the best models are now on par with humans on these tasks. The paper also shows interesting remaining differences between humans and models in terms of the errors made (humans make different errors than models) and the texture bias (much stronger in models than humans).", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*Summary of the review:*\nI recommend acceptance of this paper for its important and non-trivial conclusions, and for its impressive thoroughness (52 recent models tested on 17 OOD datasets, 90 human observers, 85,120 psychophysical trials). However, I believe that the conclusion that 'the longstanding robustness gap between humans and CNNs is closing, with the best models now matching or exceeding human performance on most OOD datasets' is too broad in scope and not currently supported by the results. I would feel comfortable increasing my rating if this claim was re-worded to more adequately reflect the scope of the results (see Main Concern).\n\n*Strengths*\n- The study is very thorough (52 recent models, 17 OOD datasets, 90 human observers, 85,120 psychophysical trials) and its main conclusions are non-trivial and have far-reaching implications for the field of computer vision.\n- I found the article very well and clearly written, and pleasant to read.\n\n*Main Concern*\n\nI believe that the main conclusion of the study, as currently stated in the abstract  ('The longstanding robustness gap between humans and CNNs is closing, with the best models now matching or exceeding human performance on most OOD datasets'), lacks support for two reasons:\n- There isn't a clear consensus on what constitutes relevant OOD benchmarks for images. The OOD tasks chosen in this paper follow a somewhat narrow definition of OOD, with the distortions only affecting pixel-level details of the images and object textures, as opposed to the overall structure of the image. For example, it would be important to see if models are now robust to unusual poses of objects in images (Alcorn et al.). See references below for more OOD datasets which affect the global structure of images and other properties.\n- Humans were presented images for a short 200ms followed by a backward mask and in a 3x3 visual angle frame. I understand the motivation to remove recurrent processing, but given this potentially strong impairment to human vision, I believe that the claim that 'best models now matching or exceeding human performance' currently lacks support.\n\n*Suggestions to improve clarity*\n- Is OOD a well-defined notion for networks trained on proprietary datasets such as CLIP? For example, are we sure that the distortions presented in the OOD datasets were not present in the training set? What is the exact definition of OOD used here?\n- How to reconcile the strong difference in shape/texture bias between human and models in fig3, with the overall performance match in fig1? Is the stylized dataset a particularly difficult dataset for the models tested?\n- 'Observed consistency' seems to be a less informative quantity than 'Error consistency' because the overall accuracy factors in 'Observed consistency'. Why not just report 'Error consistency'?\n\n*Other suggestions*\n- Error consistency: it would be useful to have a qualitative interpretation and illustrations of the images that confuse networks and not humans and vice-versa.\n- Figure 3 apparently shows a drastic failure of models to capture shape information, but would a simple linear readout trained on top of the representations of the last layer change these results? Maybe the shape information is readily available at the level of these representations but the network chose to ignore it to focus on texture. Conversely, humans naturally pay attention to shape but could probably switch to a texture-bias if asked to focus on texture.\n\n*References* \n\nStrike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects\nMichael A. Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, Anh Nguyen\nhttps://arxiv.org/abs/1811.11553\n\nExposing Previously Undetectable Faults in Deep Neural Networks\nIsaac Dunn, Hadrien Pouget, Daniel Kroening, Tom Melham\nhttps://arxiv.org/abs/2106.00576\n\nSmall in-distribution changes in 3D perspective and lighting fool both CNNs and Transformers\nSpandan Madan, Tomotake Sasaki, Tzu-Mao Li, Xavier Boix, Hanspeter Pfister\nhttps://arxiv.org/abs/2106.16198\n\nNatural Adversarial Examples\nDan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, Dawn Song\nhttps://arxiv.org/abs/1907.07174\n\nTowards Non-I.I.D. Image Classification: A Dataset and Baselines\nYue He, Zheyan Shen, Peng Cui\nhttps://arxiv.org/abs/1906.02899\nhttp://nico.thumedialab.com/\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "No.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers were enthusiastic about the breadth and quality of this work. I agree; this is an exceptional paper that will be an important resource for the community in comparing human and machine vision, especially with the open-source evaluation software.

The reviewers offered important suggestions about the wording of some of the conclusions, which the authors were receptive to in the rebuttal. The regression analysis suggested by R-bjmw is also very useful. These changes will further improve the final manuscript.
Decision: Accept (Oral)