Human alignment of neural network representations
Authors: Lukas Muttenthaler
Jonas Dippel
Lorenz Linhardt
Robert A. Vandermeulen
Simon Kornblith
Venue: ICLR 2023 poster (2023)
Keywords: Human Alignment
Robustness
Neural Network Representations
Human Concepts
Object Similarity
Computer Vision

Abstract: Today’s computer vision models achieve human or near-human level performance across a wide variety of vision tasks. However, their architectures, data, and learning algorithms differ in numerous ways from those that give rise to human vision. In this paper, we investigate the factors that affect the alignment between the representations learned by neural networks and human mental representations inferred from behavioral responses. We find that model scale and architecture have essentially no effect on the alignment with human behavioral responses, whereas the training dataset and objective function both have a much larger impact. These findings are consistent across three datasets of human similarity judgments collected using two different tasks. Linear transformations of neural network representations learned from behavioral responses from one dataset substantially improve alignment with human similarity judgments on the other two datasets. In addition, we find that some human concepts such as food and animals are well-represented by neural networks whereas others such as royal or sports-related objects are not. Overall, although models trained on larger, more diverse datasets achieve better alignment with humans than models trained on ImageNet alone, our results indicate that scaling alone is unlikely to be sufficient to train neural networks with conceptual representations that match those used by humans.

Published as a conference paper at ICLR 2023
HUMAN ALIGNMENT OF NEURAL NETWORK REPRE- SENTATIONS
Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A. Vandermeulen Machine Learning Group, Technische Universität Berlin & BIFOLD Berlin, Germany
Simon Kornblith
Google Research, Brain Team
ABSTRACT
Today's computer vision models achieve human or near-human level performance across wide variety of vision tasks. However, their architectures, data, and learn- ing algorithms differ in numerous ways from those that give rise to human vision. this paper, we investigate the factors that affect the alignment between the repre- sentations learned by neural networks and human mental representations inferred from behavioral responses. We find that model scale and architecture have essen- tially effect the alignment with human behavioral responses, whereas the training dataset and objective function both have a much larger impact. These find- ings are consistent across three datasets of human similarity judgments collected using two different tasks. Linear transformations of neural network representa- tions learned from behavioral responses from one dataset substantially improve alignment with human similarity judgments on the other two datasets. In addition, we find that some human concepts such food and animals are well-represented by neural networks whereas others such as royal or sports-related objects are not. Overall, although models trained on larger, more diverse datasets achieve better alignment with humans than models trained on ImageNet alone, results in- dicate that scaling alone unlikely to be sufficient to train neural networks with conceptual representations that match those used by humans.
INTRODUCTION
Representation learning is fundamental part of modern computer vision systems, but the paradigm has its roots in cognitive science. When Rumelhart al. (1986) developed backpropagation, their goal was to find method that could learn representations of concepts that are distributed across neurons, similarly to the human brain. The discovery that representations learned by backpropa- gation could replicate nontrivial aspects of human concept learning was a key factor in its rise to popularity in the late 1980s (Sutherland) 1986 Ng & Hinton 2017 A string of empirical successes has since shifted the primary focus of representation learning research away from its similarities to human cognition and toward practical applications. This shift has been fruitful. By some metrics, the best computer vision models now outperform the best individual humans on benchmarks such as ImageNet (Shankar et al. 2020 Beyer et al.) 2020) Vasudevan al., 2022). As computer vision systems become increasingly widely used outside of research, we would like to know if they see the world in the same way that humans do. However, the extent to which the conceptual representations learned by these systems align with those used by humans remains unclear.
Do models that are better at classifying images naturally learn more human-like conceptual represen- tations? Prior work has investigated this question indirectly, by measuring models' error consistency with humans (Geirhos et al., 2018 Rajalingham et al. 2018 Geirhos et al.) 2021 and the ability of their representations to predict neural activity in primate brains (Yamins et al. 2014 Güçlü & van Gerven 2015) Schrimpf et al. 2020), with mixed results. Networks trained on more data make somewhat more human like errors (Geirhos al. 202I), but do not necessarily obtain better fit to brain data (Schrimpi et al., (220)). Here, we approach the question of alignment between human and machine representation spaces more directly. We focus primarily on human similarity judgments

---

Published as a conference paper at ICLR 2023
collected from an odd-one-out task, where humans saw triplets of images and selected the image most different from the other two (Hebart et al., 2020). These similarity judgments allow us to infer that the two images that were not selected are closer to each other in individual's concept space than either is to the odd-one-out. We define the odd-one-out in the neural network represen- tation space analogously and measure neural networks' alignment with human similarity judgments in terms of their odd-one-out accuracy, i.c., the accuracy of their odd-one-out "judgments" with respect to humans', under a wide variety of settings. We confirm our findings on two independent datasets collected using the multi- uulti-arrangement task, in which humans arrange images according to their similarity Cichy al.] (2019): King et al. (2019). Based on these analyses, we draw the following conclusions:
Scaling ImageNet models improves ImageNet accuracy, but does not consistently improve align- ment of their representations with human similarity judgments, Differences in alignment across ImageNet models arise primarily from differences in objective functions and other hyperparam- eters rather than from differences in architecture or width/depth.
Models trained on image/text data, or on larger, more diverse classification datasets than Ima- geNet, achieve substantially better alignment with humans.
linear transformation trained improve odd-one-out accuracy on THINGS substantially in- creases the degree of alignment on held-out THINGS images as well as for two human similarity judgment datasets that used a aulti-arrangement task to collect behavioral responses.
We use a sparse Bayesian model of human mental representations (Muttenthaler et al. 2022) to partition triplets by the concept that distinguishes the odd-one-out. While food and animal- related concepts can easily be recovered from neural net representations, human alignment is weak for dimensions that depict sports-related or royal objects, especially for ImageNet models.
2 RELATED WORK
Most work comparing neural networks with human behavior has focused on the errors made during image classification. Although Imagenet- models appear to make very different errors than humans (Rajalingham et al. 20188] Geirhos et al. 2020 202 1), models trained on larger datasets than
ImageNet exhibit greater error consistency (Geirhos a 2021) Compared to humans, ImageNet-
trained models perform worse on distorted images (Richard Webster et al. 2019; Dodge & Karam 2017 Hosseini et al., 2017 Geirhos et al. [2018) and rely more heavily on texture cues and less on
object shapes (Geirhos et al., 2019 Baker et al. (018). although reliance on texture can be mitigated through data augmentation (Geirhos et al. 2019: Hermann et al. 2020 Li et al. 202T), adversarial
training (Geirhos et al. 2021). or larger datasets (Bhojanapalii et al. 2021
Previous work has also compared human and machine semantic similarity judgments, generally us- ing smaller sets of images and models than we explore here. Jozwik et al. (2017) measured the similarity of AlexNet and VGG-16 representations to human similarity judgments of 92 object im- ages inferred from
auutti-araaggement task. Peterson al. (2018) compared representations of five neural networks to pairwise similarity judgments for six different sets of 120 images. [Aminoff et al.) (2022) found that, across 11 networks, representations of contextually associated objects (e.g., bicy- cles and helmets) were more similar than those of non- oon-associt objects; similarity correlated with both human ratings and reaction times. Roads & Love (2021) collect human similarity judgments for ImageNet and evaluate triplet accuracy on these similarity judgments using 12 ImageNet networks. Most closely related our work, Marjieh al. (2022) measure aligment between representations of networks that process images, videos, audio, or text and the human pairwise similarity judgments of Peterson al. (2018). They report a weak correlation between parameter count and alignment, but do not systematically examine factors that affect this relationship.
Other studies have focused on perceptual rather than semantic similarity, where the task mea- sures perceived similarity between a reference image and a distorted version of that reference im- age (Ponomarenko et al., 2009; Zhang et al., 2018), rather than between distinct images as in our task. Whereas the representations best aligned with human perceptual similarity are obtained from intermediate layers of small architectures (Berardino et al. 2017 Zhang et al. 2018 Chinen et al.) 2018 Kumar al. 2022), the representations best aligned with odd-one-out jud judgments are obtained at final model layers, and architecture has little impact. Jagadeesh & Gardner (2022) com- pared human odd-one-out judgments with similarities implied by neural network representations and
judgmmt--
2

---

Published as a conference paper at ICLR 2023
brain activity. They found that artificial and biological representations distinguish the odd one out when it differs in category, but do not distinguish natural images from synthetic scrambled images.
Our work fits into broader literature examining relationships between in-distribution accuracy of image classification and other model quality measures, including accuracy on out-of-distribution (OOD) data and downstream accuracy when transferring the model. OOD accuracy correlates nearly linearly with accuracy on the training distribution (Recht et al. 2019: Taori et al. 2020] Miller et al.) 2021 although data augmentation can improve accuracy under some shifts without improving in- distribution accuracy (Hendrycks et al. 202T). When comparing the transfer learning performance across different architectures trained with similar settings, accuracy on the pretraining task correlates well with accuracy on the transfer tasks (Kornblith et al.] 2019b), but differences in regularization, training objective, and hyperparameters can affect linear transfer accuracy even when the impact on pretraining accuracy is small (Kornblith et al. 2019b 2021 Abnar et al. (022). In our study, we find that the training objective has significant impact, as it does for linear transfer. However, in contrast to previous observations regarding OOD generalization and transfer, we find that better-performing architectures do not achieve greater human alignment.
3 METHODS
Human
3.1 DATA
Our primary analyses use images and corre- sponding human odd-one-out triplet judgments from the THINGS dataset (Hebart et al. 2019). THINGS consists of collection of 1,854 object categories, concrete nameable nouns in the En- glish language that can be easily identified as a central object in a natural image, along with representative images for these categories. For presentation purposes, we have replaced the im- ages used Hebart et al. (2020) with images (Stoinski et al., 20022). We additionally consider two datasets of images with human similarity judg- ments obtained from a nulti--arangeemett task (Cichy et al., 2019 King et al., 2019). We briefly describe the procedures that were used to obtain these datasets below.
Figure I: An example triplet from Hebart al. 22020), where neural nets choose different odd-one-our than a human. The images in this triplet are copyright-free images from THINGS + (Stoinski et al. 2022)
similar in appearance that are licensed under CCO
THINGS triplet task Hebart al. 2020)) collected similarity judgments from human participants on images in THINGS in the form of responses to triplet task. In this task, images from three distinct categories are presented to participant, and the participant selects the image that is most different from the other two (or equivalently the pair that most similar). The triplet task has been used to study properties of human mental representation for many decades (e.g., Fukuzawa et al., 1988] Robilotto & Zaidi 2004 Hebart et al. 2020). Compared tasks involving numerical/Likert-se pairwise similarity judgments, the triplet task does not require different subjects to interpret the scale similarly and does not require that the degree of perceived similarity is cognitively accessible.
Hebart et al. 2020) collected 1.46 million unique responses crowdsourced from 5,301 workers. See Figure for an example triplet. Some triplets offer an obvious answer to the triplet task, e.g. "cat", "dog" "candelabra", whereas others can be ambiguous, e.g. "knife", "table", "candelabra." To estimate the consistency of triplet choices among participants Hebart et al. (2020) collected 25 responses for each triplet in randomly selected set of 1,000 triplets. From these responses, Hebart ct al. 2020) determined that the maximum achievable odd-one-out accuracy is 67.22% 1.04%.
Multi-arrangement task The multi-arrangement task is another task commonly used to measure human similarity judgments (Kriegeskorte & Mur] (012)). In this task, subjects arrange images on computer screen so that the distances between them reflect their similarities. We use multi- arrangement task data from two recent studies. Ciihy et al. (2019) collected similarity judgments from 20 human participants for 118 natural images from ImageNet (Deng et al.) 2009). and King et al. (2019) collected similarity judgments from 20 human participants for two natural image sets with 144 images per image set. The 144 images correspond to 48 object categories, each with three images. For simplicity, we report results based on only one of these two sets of images.
3

---

Published as a conference paper at ICLR 2023
3.2 METRICS
Zero-shot udd-one-out accuracy To measure alignment between humans and neural networks on the THINGS triplet task, we examine the extent which the odd-one-out can be identified directly from the similarities between images in models' representation spaces. Given representations æ1, 22 and Dy of the three images that comprise the triplet, we first construct similarity matrix S E R3x3 where S.jj == z; / [[] x. (lll|ll))l)) the cosine similarity between a pair of repre- sentations We identify the closest pair of images in the triplet as arg max;,>> St.f; the remaining image is the odd-one-out. We define zero-shot odd-one-out accuracy as the proportion of triplets where the odd-one-out identified in this fashion matches the human odd-one-out response. When evaluating zero-shot odd-one-out accuracy of supervised ImageNet models, we report the better of the accuracies obtained from representations of the penultimate embedding layer and logits; for self- supervised models, we use the projection head input: for image/text models, we use the representa- tion from the joint image/text embedding space; and for the JFT-3B model, we use the penultimate layer. In Figure B.I we show that representations obtained from earlier layers performed worse than representations from top layers, as in previous work (Montavon et al.) 2011).
Probing addition to measuring zero-shot odd-one-out accuracy on THINGS, we also learn linear transformation of each neural network's 's representation that maximizes odd-one-out accuracy and then measure odd-one-out accuracy of the transformed representation on triplets comprising a held- out set of images. Following Alain & Bengio (2017), we refer to this procedure as linear probing. To learn the linear probe, we formulate the notion of the odd-one-out probabilistically, as in Hebart et al. )020). Given image similarity matrix S and triplet {i. k) (here the images indexed by natural numbers), the likelihood of a particular pair, {a, b} C {i, k). being most similar, and thus the remaining image being the odd-one-out, is modeled by the softmax of the object similarities, p( {a, (i j. k}; S) == cxp(Sa,t) (exp(S.,)) + exp(St,s) + exp(S,.)))
(1)
We learn the linear transformation that maximizes the log- likelihood of the triplet odd-one-ond out judg- ments plus an lg regularization term. Specifically, given triplet responses (fe,, b, (is. Js. k2 Di= we find square matrix W yielding similarity matrix Sig- (Wx,) (Wz,) that optimizes
1 n n
arg min W
k.). +X|
odd-one-out prodiction
Here, we determine via grid-search during k-fold gross-validation (CV). To obtain minimally biased estimate of the odd-one-out accuracy of a linear probe, we partition the m objects into two disjoint sets. Experimental details about the optimization process, k-foold CV, and how we partition the objects can be found in Appendix A. and Algorithm respectivvey.
RSA To measure the alignment between human and neural net representation spaces on multi- arrangement datasets, following previous work, we perform representational similarity analysis (RSA; |Kriegeskorte et al.] (2008)) and compute correlation coefficients between neural network and human representational similarity matrices (RSMs) for the same sets of images (Kriegesko- rte & Kievit| 2013] Cichy et al. 2019). We construct RSMs using a Pearson correlation kernel and measure the Spearman correlation between RSMs. We measure alignment multi-arrangent datasets in a zero-shot setting as well as after applying the linear probe W learned on THINGS.
3.3 MODELS
Vision models In our evaluation, we consider a diverse set of pretrained neural networks, including wide variety of self-supervised and supervised models trained ImageNet- IK and ImageNet- 21K; three models trained on EcoSet (Mehrer et al. 2021), which is another natural image dataset; "gigantic" Vision Transformer trained on the proprietary JFT-3B dataset (ViT-G/14 JFT) (Zhai eet al. 2022); and image/text models CLIP (Radford et al. 2021). ALIGN (Jia et al. 2021). and BA- SIC (Pham al. 2022). A comprehensive list of models that we analyze can be found in Table B. To obtain image representations, we use iniiiisviss sion, a Python library for extracting activa- tions from neural nets (Muttenthaler & Hebart, 202T). We determine the ImageNet top-1 accuracy for networks not trained on ImageNet- IK by training a linear classifier on the network's 's penultimate layer using I-BBGS (Liu & Nocedal 1989).
We use cosine similarity rather than dot products because it nearly always yields similar or better zero-shot add-one-out accuracies, as shown in Figure B.2
4

---

Published as a conference paper at ICLR 2023
(llearrt al, 20201
Varying architecture
Zero 100--hot 000 100000000-----0 000- Sccuracy
6.76
0.35
Beayeter
646-
6.00
6.55
0.50
0.50
646-
032
0.72
0.00
Imagehet accuracy
ImageNet accuracy
ImageNNet accuracy
Sof-superwisd models
Image//enct models
Model
Z000 Cue--ahurt cdd. 2010030000000000000000000000220 -U BOCUNNT
Swer
3.50
0.00
VoG
0.50
6:5
0.25
3.50
600
0.00
6.00
55
153
330
250
Imagehet accuracy
ImageNet accuracy
#Parametees (Mliliin))
Figure 2: Zero-shot odd-one-out accuracy THINGS only weakly correlates with ImageNet accuracy and varies with training objective but not with model architecture. Top left: Zero-shot accuracy as a function of ImageNet accuracy for all models. Diagonal line indicates least-squares fit. Top center: Models with the same architecture (ResNct-50) trained with a different objective function or diffcrent data augmentation. Since MixUp alters both inputs and targets, it listed under both objectives and augmentations. Top right: Models trained with the same objective (softmax pooss-entropy) but with different architectures. Bottom left: Perfor- mance of different SSL models. Bottom center: Zero-shot accuracy is negatively correlated with ImageNet accuracy for image/text models. Bottom right: A subset of ImageNet models with their number of parameters, colored by model family. Note that, in this subplot, models that belong to different families come from different sources and were trained with different objectives. hyperparameters, etc.; thus, models are only directly compa- rable within a family. In all plots, horizontal lines reflect chance-leve or ceiling accuracy. See also Table [.I
VICE Several of our analyses make of human concept representations obtained by Variational Interpretable Concept Embeddings (VICE), an approximate Bayesian method for finding represen- tations that explain human odd-one-out responses a triplet task (Muttenthaler et al. 2022). VICE uses mean-field VI to learn sparse representation for each image that explains the associated be- havioral responses. VICE achieves an odd-one-out accuracy of ~64% on THINGS, which only marginally below the ceiling accuracy of 67 27% (Hebart et al. 2020). The representation dimen- sions obtained from VICE are highly interpretable and thus give insight into properties humans deem important for similarity judgments. We use the representation dimensions to analyze the alignment of neural network representations with human concept spaces. However, VICE not a vision model, and can only predict odd-one-out judgments for images included in the training triplets.
EXPERIMENTS
Here, investigate how closely the representation spaces of neural networks align with humans' concept spaces, and whether concepts can be recovered from a representation via linear transfor- mation. Odd-one-out accuracies are measured on THINGS, unless otherwise stated.
4.1 ODD-ONE-OUT vs. IMAGENET ACCURACY
We begin by comparing zero-shot odd-one-out accuracy for THINGS with ImageNet accuracy for all models in Figure 2 (top left). ImageNet accuracy generally is good predictor for transfer learning performance (Korbblih et al. 2019b; Djolonga al., 2021 EEicsson et al.) (021). However, while ImageNet accuracy is highly correlated with odd-one-out accuracy for a reference triplet task that uses the CIFAR- 100 superclasses (see Appendix C). its correlation with accuracy on human odd- one-out judgments is very weak (r = 0.099). This raises the question of whether there are model, task, or data characteristics that influence human alignment.
Architecture or objective? We investigate odd-one-out accuracy as a function of ImageNet accu- racy for models that vary in the training objective/final layer regularization, data augmentation, or architecture with all other hyperparameters fixed. Models with the same architecture (ResNet-50) trained with different objectives (Komblith et al.) 202T) yield substantially different zero-shot odd- one-out accuracies (Figure 2 top center). Conversely, models with different architectures trained
5

---

Published as a conference paper at ICLR 2023
with the same objective (Korbblihh et al. 2019b) achieve similar odd-one-out accuracies, although their ImageNet accuracies vary significantly (Figure 2 top right). Thus, whereas architecture does not appear to affect odd-one-out accuracy, training objective has significant effect.
Training objective also affects which layer yields the best human alignment. For networks trained with vanilla softmax cross-entropy, the logits layer consistently yields higher zero-shot odd-one-out accuracy than the penultimate layer, but among networks trained with other objectives, the penulti-
mate layer often provides higher odd-one-out accuracy than the logits (Figure E.2 The superiority
of the logits layer of networks trained with vanilla softmax cross-entropy is Specific to the odd-one- out task and RSA and does not hold for linear transfer, as we show Appendix [
sele-super learning Jigsaw (Noroozi & Favaro 2016) and RotNet (Gidaris et al. 2018 show substantially worse alignment with human judgments than other SSL models (Figure 2 bottom left). This is not surprising given their poor performance on ImageNet. Jigsaw and RotNet are the only SSL models in our analysis that are non-Siamese, i.e., they were not trained by connecting two augmented views of the same image. For Siamese networks, however, ImageNet performance does not correspond to alignment with human judgments. SimCLR (Chen et al. (020) and MoCo-v2 (He) et al. at 2020), both trained with contrastive learning objective, achieve higher zero-shot odd-one-out
accuracy than Barlow Twins [boonarr et al., 202I). SwAV (Caron et al. 2020), and VICReg (Bardes) ct al. 2022) of which all were trained with a non-contrastive learning prrettvve-athoure their ImageNet performances are reversed. This indicates that contrasting positive against negative exam- ples rather than using positive examples only improves alignment with human similarity judgments.
Model capacity Whereas one typically observes a positive correlation between model capacity and
task performance in computer vision (Tan & Le| 22019) Kolesnikov et al. 2020 Zhai et al. 2022), we observe no relationship between model parameter count and odd-one-out accuracy (Figure) 2 bottom right). Thus, scaling model width/depth alone appears to be ineffective improving alignment.
4.2 CONSISTENCY OF RESULTS ACROSS DIFFERENT DATASETS
Although the aaatiieerran task is quite different from the triplet odd-one-out task, we observe similar results for both human similarity judgment datasets that leverage this task (see Figure 3). Again, ImageNet accuracy is not correlated with the degree of human alignment, and objective function and training data, but not architecture or model size, have substantial impact.
4.3 How MUCH ALIGNMENT CAN LINEAR PROBE RECOVER?
We next investigate human alignment of neural network representations after linearly transforming the representations to improve odd-one-out accuracy, as described in $ In addition to evaluating probing odd-one-out accuracies, we perform RSA after applying the transformation matrix obtained from linear probing on the triplet odd-one-out task to model's 's raw representation space. Note that the linear probe was trained exclusively on subset of triplets from Hebart et al.] (2020) (see Appendix F), without access to human responses from the two other human similarity judgments datasets Cichy et al. 2019: King et al. 2019).
Across all three datasets, we observe that the transformation matrices obtained from linear probing substantially improve the degree of alignment with human similarity judgments. The probing odd- one-out accuracies are correlated with the zero-shot odd-one-out accuracies for both the embedding (Figure 4 left; P 0.774) and the logit layers (Figure E.T p 0.880). Similarly, we observe strong correlation between the human alignment of raw and transformed representation spaces for the embedding layer for both multi-arrangement task datasets from Cichy et al. (2019) (Figure 4 center, P = 0.749) and King et al. (2019) (Figure 3 transformation matrices neural nets' representations, we find that image/text models and ViT-G /14 JFT are better aligned than ImageNet or EcoSet models for all datasets and metrics.
right; = 0.519) respectively. After applying the
As we discuss further in Appendix L the relationship between probing odd-one-out accuracy and ImageNet accuracy is generally similar to the relationship between zero-shot odd-one-out accuracy and ImageNet accuracy. The same holds for the relationship between Spearman's 's p and ImageNet accuracy and Spearman's 's (+ transform) and ImageNet accuracy. The correlation between Ima- geNet accuracy and probing odd-one-out accuracy remains weak (r = 0.222). Probing reduces the variance in odd-one-out accuracy Spearman's 's p among networks trained with different loss functions, self-supervised learning methods, and image/text models, yet we still fail to see improve- ments in probing accuracy with better-performing architectures or larger model capacities. However,

---

Published as a conference paper at ICLR 2023
A
(Cicyy al. 2019)
objertice
Varying archilecture
Demuiee:
Pactee
Spearmen's
0.5
CC-
0.
C2
032
0.26
020
6.0
0.2
0.54
3.75
0.70
0.00
ImageiNet accuracy
Imagelet accuracy
ImageNet
Sell-suerndsed models
Image//sst modiss
Model size
3.5
tall
Busker
Inapeeo--Ls
Sparrmn'ss
14
2.55
50
0.75
100
150
56
ImageNet
Imageltei
#Parameters (Milliin)
IKing al., 2019:
Varying objective
Varving architecture
Devucton Moblen
6.0-
Labe
Spcaman's
Mang
Booki
0.6
L6-
00
0.5
67
65
8.2
0.76
0.00
632
8.4
Imagehest accuracy
Imageeett accuracy
ImagelMett accurasy
Self supervised models
Image/taat models
Madel sime
Pazdle
Sely
3.5
0.5
VDC
201
Sparrman's
245
130
6.56
0.00
0.50
0.73
235
1000
200
26
20c
14
ImageNet accuraty
ImageNec accuracy
ePasameters (Million)
Figure 3: Spearman correlation between human and neural network representational similarity matrices is not correlated with ImageNet accuracy for ImageNet models and is negatively correlated for image/text models. Alignment varies with training objective but not with model architecture or number of parameters for both sim- ilarity judgment datasets (Cichy et al. 2019 King et al. 2019). See caption of Figure 2 for further description of panels. Diagonal lines indicate least-squares fits.
whereas image/text models exhibit negative correlation between ImageNet accuracy and zero-shot odd-one-out accuracy is negative in Figures[and]] the correlation between ImageNet accuracy and probing odd-one-out accuracy is small but positive.
Interestingly, for EcoSet models, transformation matrices do not improve alignment as much as they do for architecturally identical ImageNet models. Although one goal of EcoSet was to provide data that yields better alignment with human perception than ImageNet (Mehrer et al. (002]) we find that models trained on EcoSet are less aligned with human similarity judgments than ImageNet models.
4.4 How WELL DO PRETRAINED NEURAL NETS REPRESENT HUMAN CONCEPTS?
Below, we examine zero-shot and linear probing odd-one-out accuracies for individual human con- cepts. To investigate how well neural nets represent these concepts, we filter the original dataset D to produce new dataset D' containing only triplets correctly predicted by VICE. Thus, the best attain- able odd-one-out accuracy for any model is as opposed to the upper-bound of 0.6722 for the full data. We further partition D into 45 subsets according to the 45 VICE dimensions, ....... Dis- A triplet belongs to D when the sum of the VICE representations for the two most similar objects in the triplet, Ba Ly, attains its maximum in dimension j. j arg max Fo.j Tb.z'-
7

---

Published as a conference paper at ICLR 2023
(Hebart et ol., 2020)
(Cichy et al,, 2019)
(King et al., 2019)
Spearman p Prnssoorm
Spearmnn's p trans Sersarrmm
Seanidd (naprae-10
Aupenused Inagaler-20
Seciep
Sastiied Inapeae-210
Sestriin Inspaler-22 lell sel-iuperted
Superiia basgater-220
Sel Seannee
Zero-
accuracy
Speatman's
Figure 4: Left panel: Zero-shot and probing odd-one-out accuracies for the embedding layer of all neural nets. Right panels: Spearman rank correlation coefficients with and without applying the transformation matrix obtained from linear probing to model's raw representation space. Dashed lines indicate y = 2.
4.4.1 HUMAN ALIGNMENT CONCEPT-SPECIHGG
In Figure 5 we show zero-shot and linear probing odd-one-out accuracies for subset of three of the 45 VICE dimensions for a large subset of the models listed in Table B.I Zero-shot and probing odd- one-out accuracies for larger set of dimensions can be found Appendix J Since the dimensions found for THINGS are similar both in visual appearance and in their number between Muttenthaler ctt al. (2022) Hebart et al. (2020), we infer labeling of the human dimensions from Hebart et al. 2020 who have evidenced the interpretability of these dimensions through human experiments.
Although models trained on large datasets - image/text models and ViT-G/14 JFT - generally show higher zero-shot odd-one-out accuracy compared to supervied models models trained on ImageNet, the ordering of models is not entirely consistent across concepts. For dimension 10 (vehicles), ResNets trained with a cosine softmax objective were the best zero-shot models, whereas image/text models were among the worst. For dimension 4, an animal-related concept, models pretrained ImageNet clearly show the worst performance, whereas this concept is well represented in image/text models. Differences in the representation of the animal concept across models additionally corroborated by the t-SNE visualizations in Appendix H
Linear probing yields more consistent patterns than zero-shot performance. For almost every human concept, image/text models and ViT-G/14 JFT have the highest per-concept odd-one-out accuracies, whereas AlexNet and EfficientNets have the lowest. This difference is particularly apparent for dimension 17, which summarizes sports-related objects. For this dimension, image/text models and ViT-G/14 JFT perform much better than all remaining models. As shown in Appendix G even for triplets where VICE predicts that human odd-one-out judgments are very consistent, ImageNet models make substantial number of errors. By contrast, image/text models and ViT-G/14 JFT achieve a near-zero zero-shot odd-one-out error for these triplets.
4.4.2 CAN HUMAN CONCEPTS RECOVERED VIA LINEAR REGRESSION?
To further understand the extent to which human concepts can recovered from neural networks' representation spaces, we perform fz-regularized linear regression to examine models' ability to predict VICE dimensions. The results from this analysis - which we present in detail in Appendix]] I corroborate the findings from $4.3 models trained image/text data and VIT-G/14 JFT consis- tently provide the best fit for VICE dimensions, while AlexNet and EfficientNets show the poorest regression performance. We compare odd-one-out accuracies after linear probing and regression spectively. The two performance measures are highly correlated for the embedding (r 0.982) and logit (r 0.972; Figure F33) layers, which additionally supports probing. Furthermore, we see that the leading VICE dimensions, which are the most important for explaining human triplet responses, could be fitted with R score of 0.7 for most of the models the quality of the regression fit declines with the importance of a dimension (see Figure F.4)
observations from linear
DISCUSSION
In this work, evaluated the alignment of neural network representations with human concepts spaces through performance in an odd-one-out task and by performing representational similarity analysis. Before discussing our findings will address limitations of our work. One limitation is
8

---

Published as a conference paper at ICLR 2023
Dimension 4 Animal-related
Dimension 10 / Vehicles
Dimension 17 Sports
0.9
0.5
Zere
0.3
dig Reshort
Rostign
Haso
CLub
citammeter
Russ
Resslent
Dannerter
Edfciintie
Offic
0.0
0.0
Probing
0.4
0.2
Clip
Stitte Ansiried
Figure 5: -cro-shot and linear probing odd-one-out accuracies differ across VICE concepts. Results are shown for the embedding layer of all models for three of the 45 VICE dimensions. See Appendix for additional di- mensions. Color-coding is determined by training data/objective. Violet: Image/Text. Green: Self-supervised. Orange: Supervised (ImageNet-IIK) Cyan: Supervised (ImageNet-21K). Black: Supervised (JFT-3B).
that we did not consider non-linear transformations. It is possible that simple non non-lineat transfor- mations could provide better alignment for the networks we investigate. We plan to investigate such transformations further in future work. Another limitation stems from our use of pretrained models for our experiments, since they have been trained with a variety of objectives and regularization strategies. We have mitigated this by comparing controlled subsets of models in Figure 2
Nevertheless, we can draw the following conclusions from our analyses. First, scaling ImageNet models does not lead to better alignment of their representations with human similarity judgments. Differences in human alignment across ImageNet models are mainly attributable to the objective function with which model was trained, whereas architecture and model capacity are both in- significant. Second, models trained on image/text or more diverse data achieve much better align- ment than ImageNet models. Albeit not consistent for zero-shot odd-one-out accuracy, this is clear in both linear probing and regression results. These conclusions hold for all three datasets we have investigated, indicating that they are true properties of human/machine alignment rather than id- iosyncrasies of the task. Finally, good representations of concepts that are important to human similarity judgments can be recovered from neural network representation spaces. However, rep- resentations of less important concepts, such as sports s and royal objects, are more difficult to recover.
How can we train neural networks that achieve better alignment with human concept spaces? Al- though our results indicate that large, diverse datasets improve alignment, all image/text and JFT models we investigate all attain probing accuracies of 60-61.5%. By contrast, VICE representations achieve 64%, and a Bayes-optimal classifier achieves 67%. Since our image/text models are trained on datasets of varying sizes (400M to 6.6B images) but achieve similar alignment, suspect that further scaling of dataset size is unlikely to close this gap. To obtain substantial improvements, may be necessary to incorporate additional forms of supervision when training the representation itself. Benefits of improving human/machine alignment may extend beyond accuracy on our triplet task, transfer and retrieval tasks where it is important to capture human notions of similarity.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENTS
LM. LL, and RV acknowledge support from the Federal Ministry of Education and Research (BMBF) for the Berlin Institute for the Foundations of Learning and Data (BIFOLD) (011S18037A). This work is in part supported by Google, We thank Klaus- Robert Müller, Robert Geirhos, Katherine Hermann, and Andrew Lampinen for their helpful comments on the manuscript.
REFERENCES
Samira Abnar, Mostafa Dehghani, Behnam Neyshabur, and Hanie Sedghi. Exploring the limits of large scale pre-training. In International Conference on Learning Representations, 2022.
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings. OpenReview.net, 2017.
Elissa M Aminoff, Shira Baror, Eric W Roginek, and Daniel D Leeds. Contextual associations represented both in neural networks and human behavior. Scientific reports, 12(1):1-12, 2022.
Nicholas Baker, Hongjng Lu, Gennady Erlikhman, and Philip J. Kellman. Deep convolutional networks do not classify based on global object shape. PLOS Computational Biology, 14(12): 1-43, 12 2018. doi: 10.137 1/oo3////oraal.pbbb 1006613.
Adrien Bardes, Jean Ponce, and Yann Lecun, Vicreg: Marianc-invarrinne-ccvaaaa regularization for self- supervisd learning. In ICLR 2022-10th International Conference on Learning Represen- tutions, 2022.
Alexander Berardino, Valero Laparra, Johannes Ballé, and Eero Simoncelli. Sigen-distortions of hierarchical representations. I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Thomas Berg, Jiongxin Liu, Seung Woo Lee, Michelle L Alexander, David W Jacobs, and Peter N Belhumeur. Birdsnap: Large-scale fine mue-grained visual categorization of birds. In IEEE Confer- on Computer Vision and Pattern Recognition (CVPR), pp.
IEEE, 2014.
Lucas Beyer, Olivier J. Hénaff, Alexander Kolesnikov, Xiaohua Zhai, and Aäron van den Oord. Are done with ImageNet? CoRR, abs/2006.07159, 2020.
Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, and An- dreas Veit. Understanding robustness of transformers for image classification. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pp. 10231-10241, October 2021.
Lukas Bossard, Matthicu Guillaumin, and Luc Van Gool. nents with random forests. In European conference on computer vision, pp. 446-461. Springer, 2014.
discriminative compo-
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Pro- cessing Systems, volume 33, pp. 9912-9924. Curran Associates, Inc., 2020.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Hal Daumé IIII and Aarti Singh (eds.), Proceed- ings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 1597-1607. PMLR, 13-18 Jul 2020.
Troy Chinen, Johannes Ballé, Chunhui Gu, Sung Jin Hwang. Sergey loffe, Nick Johnston, Thomas Leung, David Minnen, Sean O' Malley, Charles Rosenberg, et al. Towards semantic perceptual image metric. In 2018 25th IEEE International Conference on Image Processing (ICIP), pp- 624-628. IEEE, 2018
10

---

Published as a conference paper at ICLR 2023
Radoslaw M. Cichy, Nikolaus Kriegeskorte, Kamila M. Jozwik, Jasper J.F. van den Bosch, and Ian Charest. The spatiotemporal neural dynamics underlying perceived similarity for real- world objects. Neurolmage, 194:12-24, 2019. ISSN 1053-8119. doi: https://doi.or//00.000 seturommage.2119.03.22
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 3606-3613. IEEE, 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-iFFe, ImageNet: A large-scale hier- archical image database. In 2009 IEEE Conference Computer Vision and Pattern Recognition, pp. 248-255, 2009, doi: 10. WWW.VVPR.2009520665..
Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D' Amour, Dan Moldovan, Sylvain Gelly, Neil Houlsby, Xiaohua Zhai, and Mario Lucic. On robustness and transferability of convo- lutional neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 16458-16468, June 2021.
Samuel Dodge and Lina Karam. A study and comparison of human and deep learning recogni- tion performance under visual distortions. In 2017 26th International Conference on Computer Communication and Networks (ICCCN), pp. 1-7, 2017. doi: 10.1 10.100/CCCN.22117. 8033446..
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x 16 words: Transformers for image recognition at scale. In International Conference Learning Representations, 2021.
Linus Ericsson, Henry Gouk, and Timothy M. Hospedales. How well do self-supervised models transfer? Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR), pp. 5414 5423, June 2021.
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88 (2):303-338 2010.
Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR) Workshop on Generative-Mode Based Vision, 2004.
Kazuyoshi Fukuzawa, Motonobu Itoh, Sumiko Sasanuma, Tsutomu Suzuki, Yoko Fukusako, and Tohru Masui. Internal representations and the conceptual operation of color in pure alexia with color naming defects. Brain and Language, 34(1):98-126, 1988. ISSN 0093-934X. doi: https:
Robert Geirhos, Carlos R. M. Temme, Jonas Rauber, Heiko H. Schütt, Matthias Bethge, and Fe- lix A. Wichmann. Generalisation in humans and deep neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett (eds.), Advances in Neural Infor- mation Processing Systems, volume 31. Curran Associates, Inc., 2018
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In 7th International Conference on Learning Representations, ICLR 2019. OpenReview.net, 2019.
Robert Geirhos, Kristof Meding, and Felix A. Wichmann. Beyond accuracy: quantifying trial-by- trial behaviour of cnns and humans by measuring error consistency. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 13890-13902. Curran Associates, Inc., 2020.

---

Published as a conference paper at ICLR 2023
Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 23885-23899 Curran Associates, Inc., 2021.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenRe- view.net, 2018.
Priya Goyal, Quentin Duval, Jeremy Reizenstein, Matthew Leavitt, Min Xu, Benjamin Lefaudeux, Mannat Singh, Vinicius Reis, Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Ishan Misra. VISSL, 2021.
Unut Güşlü and Marcel J. van Gerven. Deep neural networks reveal gradient in the com- plexity of neural representations across the ventral stream. Journal of Neuroscience, 35(27): 10005-10014, 2015, ISSN 0270-6474. doi: 10.5220NEUROSCC.5033 14.2015.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On Calibration of Modern Neural Networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Con- ference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 1321-1330 PMLR, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IFFE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Martin N. Hebart, Adam H. Dickter, Alexis Kidder, Wan Y. Kwok, Anna Corriveau, Caitlin Van Wicklin, and Chris I. Baker. THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images. PLOS ONE, 14(10):1-24, 2019. doi: 10.137 10.rra//jurrnl..one 0223792.
Martin N. Hebart, Charles Y. Zheng, Francisco Pereira, and Chris I. Baker. Revealing the multi- dimensional mental representations of natural objects underlying human similarity judgements. Nature Human Behaviour, 4(1 4(11):1173-118: October 2020. doi:
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song. Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8340-8349, October 2021.
Katherine Hermann, Ting Chen, and Simon Kornblith. The origins and prevalence of texture bias in convolutional neural networks. Advances Neural Information Processing Systems, 33:19000- 19015, 2020.
Hossein Hosseini, Baicen Xiao, Mayoore Jaiswal, and Radha Poovendran. On the limitation of convolutional neural networks in recognizing negative images. In Xuewen Chen, Bo Luo, Feng Luo, Vasile Palade, and M. Arif Wani (eds.). 16th IEEE International Conference Machine Learning and Applications, ICMLA 2017, pp. 352-358. IEEE, 2017. doi: 10. 10.1109//CLL..211. 0-136.
Akshay V. Jagadeesh and Justin L. Gardner. Texture-likk like representation of objects in human visual cortex. Proceedings of the National Academy of Sciences, 119(17):e2 (1t1117))2111002111 153021 2022. 10.1073/pnas.21 15300119
doi:
12

---

Published as a conference paper at ICLR 2023
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In Marina Meila and Tong Zhang (eds.). Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 4904 4916. PMLR, 18 24 Jul 2021.
Kamila M. Jozwik, Nikolaus Kriegeskorte, Katherine R. Storrs, and Marieke Mur. Deep Con- volutional Neural Networks outperform feature-based but not categorical models in explain- ing object similarity judgments. Frontiers in Psychology, 8, 2017. ISSN 1664-1078. doi:
Marcie I.. King, Iris I.A. Groen, Adam Steel, Dwight J. Kravitz, and Chris I. Baker. Similarity judgments and cortical visual responses reflect different properties of object and scene categories in naturalistic images. Neurolmage, 197:368-388 -826, 2019. ISSN 1053-8119, doi: https://doi.or// www.:/f..enoommee22101110
Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference Learning Representations, ICLR 2015, Conference Truck Proceedings, 2015.
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.). Computer Vision - ECCV 2020, pp- 491-507, Cham, 2020. Springer International Publishing. ISBN 978-3-030-58558-7.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton. Similarity of neu- ral network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning. ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 3519-3529, PMLR, 2019a.
Simon Kornblith, Jonathon Shlens, and Quoc V. Le. Do Better ImageNet Models Transfer Better? In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA. June 16-20 2019. pp. 2661-2671. Computer Vision Foundation IEEE, 2019b. doi: 10.1 N..1199CVVR.22190000 990077..
Simon Kornblith, Ting Chen, Honglak Lee, and Mohammad Norouzi. Why Do Better Loss Func- tions Lead to Less Transferable Features? In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Anmual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, volume 34, pp. 18648-28666 2021.
Jonathan Krause, Jia Deng, Michael Stark, and Li Fei-Fei. Collecting a large-scale dataset of fine- grained cars. In Second Workshop on Tine-Grained Visual Categorization, 2013.
Nikolaus Kriegeskorte and Rogier A. Kievit. Representational geometry: integrating cognition, computation, and the brain. Trends in Cognitive Sciences, 17(8):401-413 2013. ISSN 1364-
6613. doi: https://doi.org orr//1...r//1...00 wrgg10.1116/..1. 1i..1//.iixs.2133.6..0
Nikolaus Kriegeskorte and Marieke Mur. Inverse mds: Inferring dissimilarity structure from
multiple item arrangements. Frontiers in Psychology, 3. 2012. ISSN 1664-1078, doi:
10.3889fissy.22112.00255
Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini. Representational similarity analysis connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 2008. ISSN 1662-5137. doi: 04338//nerro.000004.20000
Alex Krizhevsky. One weird trick for parallelizing convolutional neural networks. arXiv e-prints, art. arXiv:1404.5997. April 2014.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical Report University of Toronto, Toronto, Ontario, 2009.
13

---

Published as a conference paper at ICLR 2023
Manoj Kumar, Neil Houlsby, Nal Kalchbrenner, and Ekin D Cubuk. Do better ImageNet classifiers assess perceptual similarity better? Transactions on Machine Learning Research, 2022.
Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan L. Yuille, and Cihang Xie. Shape-texture debiased neural network training. In 9th International Conference on Learning Representations, ICLR 2021. OpenReview.net, 2021.
Dong C Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical programming, 45(1):503-522, 1989.
S. Maji, J. Kannala, E. Rahtu, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. Technical report, 2013.
Raja Marjieh, Pol Rijn, Ilia Sucholutsky, Theodore R. Sumers, Harin Lee, Thomas L. Griffiths, and Nori Jacoby. Words are all you need? Capturing human sensory similarity with textual descriptors, 2022.
Johannes Mehrer, Courtney Spoerer, Emer C. Jones, Nikolaus Kriegeskorte, and Tim C. Kiet- zmann. An ecologically motivated image dataset for deep learning yields better models of hu- man vision. Proceedings of the National Academy of Sciences, 11888))2011411118, 2021. doi:
John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy on the line: on the strong correlation between out-of-disttibutton and in-distribution generalization. In Marina Meila and Tong Zhang (eds.). Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 7721-7735. PMLR, 18 24 Jul 2021.
18-24
Grégoire Montavon, Mikio I.. Braun, and -laus-Robert Müller. Kernel Analysis of Deep Networks. doi: 10.5555/1953048.25 208.8.07888...
Journal of Machine Learning Research, 12:2563-255 2581,, 2011.
Lukas Muttenthaler and Martin N. Hebart. THINGSvision: A Python toolbox for streamlining the extraction of activations from deep neural networks. Frontiers in Neuroinformatiics 15:45, 2021.
ISSN 1662-5196. doi: 10.3389/fninf.2221.77999 277.6898388.
Lukas Muttenthaler, Charles Yang Zheng. Patrick McClure, Robert A Vandermeulen, Martin N Hebart, and Francisco Pereira. VICE: Variational Interpretable Concept Embeddings. In Ad- vances Neural Information Processing Systems, volume 35, 2022.
Andrew Ng and Geoffrey E. Hinton. Heroes of Deep Learning: Andrew Ng interviews Geoffrey Hinton, 2017.
Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Computer Vision, Graphics & Image Processing, 2008. ICVGIP '08. Sixth Indian Conference on, pp. 722-729. IEEE, 2008.
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (eds.), Computer Vision ECCV 2016 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI, volume 9910 of Lecture Notes in Computer Science, pp. 69 84. Springer, 2016. doi: 10.1007/978- 3-319--31----9 46469-464644) -..5.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3498-3505. IEEE, 2012.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zachary Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' AAlcé--Bue, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Process- ing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14 2019, Vancouver, BC, Canada, pp. 8025-8035, 2019.
14

---

Published as a conference paper at ICLR 2023
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Seikit-learn: Machine learning in python. Journal of machine learning research, 12(0ct):2825-2887) 2011.
Joshua C. Peterson, Joshua T. Abbott, and Thomas L. Griffiths. Evaluating (and improving) the correspondence between Deep Neural Networks and Human Representations. Cogn. Sci., 42(8): 2648-2666 2018. doi: Sal1l11/oos..100
Hieu Pham, Zihang Dai, Golnaz Ghiasi, Kenji Kawaguchi, Hanxiao Liu, Adams Wei Yu, Jiahui Yu, Yi-Ting Chen, Minh-Thang Luong, Yonghui Wu, Mingxing Tan, and Quoe V. Le. Combined scaling for open- vhcebvocyy image classification. arXiv e-printx, art. arXiv:211 111. 11. 10000, 2022.
Nikolay Ponomarenko, Vladimir Lukin, Alexander Zelensky, Karen Egiazarian, Marco Carli, and Federica Battisti. TID2008 A Database for Evaluation of Full-Reference Visual Quality Assess- ment Metries. Advances of Modern Radioelectronics, 10(4):30-45 2009.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learming, volume 139 of Proceedings of Machine Learning Research, pp. 8748-8763, PMLR, 18-24 24 Jul 2021.
Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do vision transformers see like convolutional neural networks? In M. Ranzato, A. Beygelz- imer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 12116-12128.
Curran Associates, Inc., 2021.
Rishi Rajalingham, Elias B. Issa, Pouya Bashivan, Kohitij Kar, Kailyn Schmidt, and James J. Di- Carlo. Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks. Journal of Neuroscience, 38(33):7255-7269) 2018. ISSN 0270-6474.
doi: 10. (0.152//NEUROSSL.0888 18.2018.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do ImageNet classifiers generalize to ImageNet? In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.). Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 5389-5400. PMLR, 09-15 Jun 2019.
Brandon RichardWebster, Samuel E. Anthony, and Walter J. Scheirer. PsyPhy: psychophysics driven evaluation framework for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9):2280-22 22222222882222,,,_. 2019. doi: 10.1 10.119//PAML.20188284999
Brett D. Roads and Bradley C. Love. Enriching ImageNet with human similarity judgments and psychological embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3547-3557, June 2021.
Rocco Robilotto and Qasim Zaidi. Limits of lightness identification for real objects under natural viewing conditions. Journal of Vision, 4(9):9-9, 09 2004. ISSN 1534-7362. doi: 10.116774.9.9,
Rebecca Roelofs, Nicholas Cain, Jonathon Shlens, and Michael C. Mozer. Mitigating bias in cali- bration error estimation. In Gustau Camps- Francisco J. R. Ruiz, and Isabel Valera (eds.), International Conference on Artificial Intelligence and Statistics, AISTATS 2022, 28-30 March 2022, Virtual Event, volume 151 of Proceedings of Machine Learning Research, pp. 4036 4054. PMLR, 2022.
David E Rumelhart, Geoffrey Hinton, and Ronald J Williams. Learning representations by back- propagating errors. Nature, 323(6088) 1986.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Fei- Fei Li. ImageNet large scale visual recognition challenge. Int. Comput. Vis., 1 115(3):211-252, 2015. doi: 10.7007/s1 123-011-008166
15

---

Published as a conference paper at ICLR 2023
Martin Schrimpf, Jonas Kubilius, Ha Hong. Najib J. Majaj, Rishi Rajalingham, Elias B. Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy, Roy, Franziska Geiger, Kailyn Schmidt, Daniel L. K. Yamins, and James J. DiCarlo. Brain- Which artificial neural network for object recognition is most brain-like? bioRxiv, 2020. doi: 10.1 101/407007
Vaishaal Shankar, Rebecca Roclofs, Horia Mania, Alex Fang, Benjamin Recht, and Ludwig Schmidt. Evaluating machine accuracy on ImageNet. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 8634 8644. PMLR, 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceed- ings, 2015.
Andreas Peter Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How train your ViT? Data, Augmentation, and Regularization in Vision Transformers. Transactions on Machine Learning Research, 2022.
Laura M Stoinski, Jonas Perkuhn, and Martin N Hebart. THINGS+:
New norms and metadata for the THINGS database of 1,854 object concepts and 26,107 natural object images, Jul 2022.
Stuart Sutherland. Cognition: Parallel distributed processing. Nature, 323(6088):486-486 Oct 1986. ISSN 1476-4687. doi: 10. 10.1038/323486a0.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 2818- 2826. IEEE Computer Society, 2016. doi: 20.11099.
Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural net- works. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th ternational Conference Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 6105-6114. PMLR, 09-15 Jun 2019.
Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrIPS 2020, volume 33, pp. 18583-18599. Curran As- sociates, Inc., 2020.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE, Journal of Machine Learning Research, 9(1)), 2008
Vijay Vasudevan, Benjamin Caine, Raphael Gontijo Lopes, Sara Fridovich-Ker and Rebecca Roclof'. When does dough become a bagel? Analyzing the remaining mistakes on ImageNet. CoRR, abs/2205.0452 2022. doi: 00.48550/arxiv.225..444 04456..
Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3485-3492. IEEE, 2010.
Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), July 2017.
Daniel L. K. Yamins, Ha Hong, Charles F. Cadieu, Ethan A. Solomon, Darren Seibert, and James J. DiCarlo. Performance-optimin hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111(23):861 8624, 2014. doi: 10.
2073/pnas.140311 1403112114
16

---

Published as a conference paper at ICLR 2023
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stephane Deny. Barlow twins: Self Self-supe learning via redundancy reduction. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 12310-12320. PMLR, 18-24 Jul 2021.
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transform- ers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12104-12113, June 2022.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
17

---

Published as a conference paper at ICLR 2023
A EXPERIMENTAL DETAILS
A.1 LINEAR PROBING
Initialization We initialized the transformation matrix W E RPxP used in Eq. 2 with values from tight Gaussian centered around 0, such that W ~ ~V 10 II) at the beginning of the optimization process.
Training We optimized the transformation matrix W via gradient descent, using Adam (Kingma & Ba 2013) with a learning rate of 7) 0.001. We performed grid-search the learning rate 7. where E {0.0001, 0.001, 0.01} and found 0.001 to work best for all models in Table B.1 We trained the linear probe for maximum of 100 epochs and stopped the optimization process early whenever the generalization performance did not change by factor of 0.0001 for T 10 epochs.
For most of our evaluation and linear probing experiments, we use PyTorch (Paszke et al. 2019).
Cross-validatiin To obtain a minimally biased estimate of the odd-one-out accuracy of a linear probe, we performed k-fold CV over objects rather than triplets. We partitioned the m objects into two disjoint sets for train and test triplets. Algorithm demonstrates how object partitioning was performed for each of the & folds.
Algorithm Algorithm for object partitioning during k-fold CV
Input: m)
> Here, D == ((a,, baj, {ia, Ja e ka )=)>= and m is the number of objects
[m] {l....,m}
[[m]] m
Cuain ~ U((m) Sample number of train objects uniformly at random without replacement A> Test objects are the remaining objects Initialize an empty set for the train triplets > Initialize an empty set for the test triplets
Owea [m] Drain
Drrain
Diea ==
for E {1, {L...., n2 do
assignments & list() ) > For each triplet initialize empty list to control object assignments
for x E Js, kg do if (xr E Orrin) then assignment 2 "train"
else assignment A "test" end if assignments + assignment
Append current assignment the list of assignments
end for if (len(set (asignnsggmmtttss * 1) then
continue If not all objects in triplet belong to the same set of objects, discard triplet
else
assignment - pop(sct (assinntas))))))tss) (assignment is "train") then D Dirain D.
Get object assignment of current triplet
Assign current triplet to the train set
else
D Disa D Dust U DD
Assign current triplet to the test set
end if
end if
end for
Output: (Dtrann Dres))
Return both train and test triplet sets
Note that the number of train objects that are sampled uniformly at random without replacement from the set of all objects is dependent on k. We performed a grid-search search over k, where l E 3,,3,44 4, 5), and observed that 3-fold and 4-fold CV lead to the best linear probing results. Since the objects the train and test triplets are not allowed to overlap, loss data was inevitable (see Algorithm D. One can easily see that minimizing the loss of triplet data comes at the cost of disproportionally decreasing the size of the test set. We decided to proceed with 3-fold CV in final experiments since using 2/3 of the objects for training and 1/3 for testing resulted in a proportionally larger test set than using 3/4 for training and 1/4 for testing (~ 433k train and 54k test triplets for 3-fold CV 616k train and ~ 23k test triplets for 4-fold CV). In
18

---

Published as a conference paper at ICLR 2023
general, the larger test set, the more accurate the estimate of a model's generalization performance. To find the optimal strength of the l regularization for each linear probe, we performed a grid- search over À for each k value individually. The optimal À varied between models, where À (0.0001, 0.001, 0.01, 0.1, }--
A.2 TEMPERATURE SCALING
is widely known that classifiers trained to minimize cross-entropy tend to be overconfident their predictions (Szegedy al. 2016 Guo al. 2017] [Roelofs et al. 2022), which is in stark contrast to the high-entropy predictions of VICE. For this purpose, we performed temperature scaling (Guo al. 2017) the model outputs for THINGS and searched the scaling parameter for each model. In particular, we considered temperature-scaled predictions
exp(TSa.b)
p(a, bHf(,,,k), exp(rS..3) + exp(TS..x) exp(TS,x))
where multiply S in Eq. I by constant > 0 and St, is the inner product of the model representations for images i and j. i.e. the zero-shot similarities. There are several conceivable criteria that could be minimized to find the optimal scaling parameter т from of candidates. For our analyses, we considered the following,
Average lensen-Shanoon (JS) distance between model zero-shot probabilities and VICE proba-
bilities over all triplets
Average Kullback-Leein Leibler divergence (KLD) between model zero-shot probabilities and VICE probabilitics over all triplets
Expected Calibration Error (ECE) Guo et al. 2017
The ECE is defined as follows. Let (fa,, }. (ies Je.JJ. je k2) +==) be the of triplets and human responses from Hebart et al. (2020). For a given triplet {i,j,k] and similarity matrix S we define confidence
conf
max p ((a,, (a.b)C(i.j.k]
This corresponds to the expected accuracy of the Bayes classifier for that triplet according to the probability model from S with Eq. [ We define Bm to be those training triplets where
m 10
m
conf (fis) k.) S)
10
For similarity matrix, S, and set triplets with responses, D' C. we define acc (D', to be the portion of triplets in D' correctly classified according to the highest similarity according to S. Finally for a set triplets C and similarity matrix S we define conf to be the average confidence over that set (triplet responses simply ignored). The ECE is defined as
10 Σ
Bm (TS) n
ECE (T,
(cc(Bm (r5)) conf (Bm (T5)).-
Intuitively, the ECE is low if for each subset, (T the model's accuracy and its confidence are near each other. A model will be well-calibrated if its confidence in predicting the odd-one-out in triplet corresponds to the probability that this prediction is correct.
Of the three considered criteria, ECE resulted in the clearest optima when varying т, whereas KLD plateaued with increasing and JS distance was numerically unstable, most probably because the model output probabilities were near zero for some pairs, which may result in very large JS distance.
For all models, we performed a grid-search over E 10 7.5 10 10-1,3 10 2.5 10 10-1,111 10 2.5 5. 10 10 3.5 10.10 10-40,1--
10 7.5- 10 2. 10-2 2,5.
2.5 10 2, 10 2 7.5 10
10 4.5 10 10 10-53.
A.3 LINEAR REGRESSION
Cross-validation We used ridge regression, that is lz-regularized linear regression, to find the trans- formation matrix A,, and bias by that result in the best fit. We employed nested k-fold CV for each of the d VICE dimensions. For the outer CV we performed grid-search over k. where E {2, 3, 4, 5), similarly to how k-fold CV was performed for linear probing (see Appendix(A.I] For our final experiments, used 5-fold CV obtain minimally biased estimate for the Ra score
19

---

Published as a conference paper at ICLR 2023
of the regression fit. For the inner CV, leveraged leave-one-our CV to determine the optimal for Eq. using RidgeCV from Pedregosa et al. 2011). We performed grid search over a, where (0.01,0.1, 1, 10, 100, 1000, 10000, 100000, 1000000}.
B MODELS
First, we evaluate supervised models trained on ImageNet (Russakovsky et al. 2015), such as AlexNet (Krizhevsky] 2014), various VGGs (Simonyan & Zisserman 2015), ResNets (He et al.)
2016). EfficientNets Tan & Le 2019), ResNext models (Xie et al. 2017). and Vision Transformers
(ViTs) trained ImageNet-IK (Dosovitskiy al. (021) or ImageNet-2IK (Steiner et al. 2022)
respectively. Second, we analyze recent state-of-the-art models trained on image/text data, CLIP-
RN & CLIP-ViT (Radford al. [021), ALIGN (Jia al. 2021) and BASIC-L (Pham al. 2022
Third, we evaluate self supervsueee models that were trained with a contrastive learning objective such SimCLR (Chen al. 2020) and MoCo-v2 (He al.) 2020), recent SSL models that were trained with a non-contrastive learning objective (no negative examples). Barlow Twins (Zhontar al. 2021), SwAV (Caron et al. 2020), and VICReg (Bardes et al.) 2022), well earlier SSL, non-Siamese models, Jigsaw (Noroozi & Favaro 2016). and RotNet Gidaris et al. 2018). All self- supervised models have a ResNet-50 architecture. For SimCLR, MoCo-v2, Jigsaw and RotNet, leverage model weights from the VISSL library (Goyal et al.] (021). For the other models we use weights from their official GitHub repositories. Last, we evaluate the largest available ViT (Zhai) et al. 2022), trained on the proprietary JFT-3B image classification dataset, which consists of ap- proximately three billion images belonging to approximately 30,000 classes (Zhai al. 2022). See Table B.I for further details regarding the models evaluated.
Zero-shot odd-on odd-one-out accuracy
0.525
Reslict 50 BarlowTwins
ResNet 50
VS619
Alextet VITU16
ResNet-152
0.425
0.400
0.375
Layer Depth
Figure B.1: Zero-shot odd-on odd-one-out accuracy for different layers for a subset of selected models.
Seapon and (mageste--3) Seseriied Eneoptes--333
0.525
Imagetei-33) Sepenined
0.500
Savenieed Drepgeie-220 Sapervied hale
0.500
0.475
0.475
Cosine
Cosine
0.450-
0.425
0.425
0.400
0.400
0.375
0.350
0.325
0.35
0.40
0.45
0.50
0.35
0.40
0.45
0.50
Product
Dot Product
Figure B.2: Zero-shot odd-one-out accuracy using the cosine similarity nearly always is better than using the dot product similarity measure.
20

---

Published as a conference paper at ICLR 2023
Model
Source
Architecture
Dutaset
Objective
ImageNet Zero Shot
Probing
AlexNet
Krobevaky
2014
AletNer
ImigeNas-IK
Supervised (sothmes)
56.12%
50.47%
53.84%
AlexNet
2021
AlexNet
Bcoset
Supervisd (sottmax)
50.00%
51.30%
ALIGN
ElicienNee
AIJGNN dotaset
Imuge/Test (conte)
85.11%
43.60%
60.81
Bese-1.
2022
CoAiNer
ALIGN JFT-58
Image/Test (conte)
89,45%
50,6555
61.24%
CLIP RNIOI
Karlord
7021
ResNet
CLIP datavet
Imuge/Text (contr)
75. 70%
51. 11.26%
60.22%
CLIP RN50
ResNet
JP
Image/Test
73.30%
52.78%
50.92%
CLIP R6
2021
ResNer
dataver
Image/Test (oonte)
81.505
49,699
60.86%
CLIP RN50.4
Karlonn et
7021
ResNit
CLIP datuset
Image/Test (contr)
78.20%
50.24%
60. 38%
CLIP RN50664
Karlerd
70:11
ResNet
CL dataver
Image/Teu (contr)
83.60%
47.64%
61.07%
CLIP VIT-R/16
VIT
Image/Test
80.20%
60.72%
CLIP VIT-B/32
Maillond 21
7071
VIT
CLIP dataset
Imuge/Test (conte)
10%
51.41%
60. 54%
CLIP /1T:1/14
Kadlird 21
7121
VIT
CL.1P datnect
Image/Test (cont))
83 13.900%
46.711%
60 645
DenseNet- 121
Koenbnnh
20146
DemseNet
ImageNet-1K
Supervised (so(maa)
75.6/1%
50.37%
55.18%
DenseNat-16 169
DensaNet
ImggoNe-1K
Supersised (soflma)
76.73%
55.366
Den4NNa-201
Krempooh S0
3019
DosseNet
ImugeNes-1K
Supervised (sofmaa)
77.145
50.45%
55.37%
Elticiem DO
Tim Le
7014
ElicieniNet
ImageNe-1IK
Supervised (sofmax)
77.69%
45.42%
50.82%
EfficientNat B1
Ta
7019
EficieniNet
ImigeNe-II
Superiised (sofhmas)
79,849
45.00%
51. 11.300
EffleienNNat B2
ImigoNa-1K
Supervised (soflmax)
80.61%
45.23%
40.33%
ElficientNet B3
Tin Jim,1 1x
7014
Elicien/Net
ImageNe-1I
Supervised (so0maa)
82.01%
94%
50.79%
EfficiemNNet B4
Tinn & 1a
E/icienNNet
ImigeNe-IK
Supervised (sofmaa))
83.38%
38, 52%
50.65%
EfficientNer R5
Tmi To
1319
EfiiiennNet
ImigeNat-1K
Supervised (sofmmss
83.4498
45, 10%
51.47%
EfficientNet B6
Tin
EllicieniNet
ImageNe-1X
Supervised (soflmax)
$4.01%
51.56%
EfficientNat R7
Tin 12
771
EfficioNNel
ImugeNe--IK
Supervised (sohmax)
125%
45.77%
52.44 52-411%
Inception RN V2
Aperibii
20196
Incepiion
ImageNe: -K
Supervised (sofimas)
S0.26/%
51.31%
55.78%
Inceplion V1
Keenhhnhh
AU19
Incepiion
ImugeNe--1 IK
Supervised (sofmmas)
73.63%
50.47%
54.97%
Incopiio-V22 V2
Inception
ImugoNe-1K
Supersised (soflmac)
54.97%
Inception V3
Keenopih 2.
20196
Incepiion
ImageNet-1X
Supervised (solimas)
78.641%
51.59%
55.80%
Inceplion-V4
Keenbhith 41 m1
A114
Incepiun
ImageNe-IK
Supervied (soflmas)
79.92%
51.58%
55. 55.475
Krennon mD
2019
MobileNet
ImigeNa-11
Supervised (sofmmas
72.39%
50.70%
54 1899
MebilNNe-V
Kornblith
20196
MobileNet
ImageNe-1X
Supervised (sofimas)
71.67%
50.45%
55.17%
MobileNe--V2 (1.4)
Kemnhihh a ml.
AT
MobileNel
ImageNes-1
Supervised (soflma)
74.66%
50.67%
53.1
NASNet-L
Acenbiii ct dl.
30196
NASSe:
ImageNer LK
Supervised (sotimas)
S0,7716
51.68%
55.78%
KASNet Mobile
A0196
NASNet
ImageNet- IK
Supervised (sofhmax)
73.57%
51.23%%
55, 18%
ResNei
ImugoNe-1K
Sel-sup. (son-conir)
50, 50%
RN- Jigraw
Gova
A144
ResNet
ImageNer: LK
Self sup. inon Siumese)
48.57%
39.56%
50.1118
RN 50 50-MoCo-12
Lival 4 3F
ResNlet
ImageNe:-1 IK
Self-sup. (contr)
66.10%
47.85%
55.911%
RN-50- RocNee
ResNiet
ImugeNa-11 IK
Self-sup. (non-Siamewe)
54.93%
39.839 83.8
50.82%
SinCLR
Goval
ResNet
ImageNer LK
Self sup. (contz)
69.681
47.29/%
56.37%
RN- RN-50--WAV
Lanon 30
A1A1
ResNet
ImageNes- IK
Self-wip. (nom contr.)
74.92%
42.27%
51.79%
RN RN-50-VIC S0-VCC
Buroet 0t 10
+132
ResNet
ImageNer-1K -K
Setf-sup. (non contr.)
73.20%
42.44%
20.30%
2015
ResNet
ImageNet LK
Supervised (sofmaax)
69.76%
51.05%
54.97%
RN-34
ImageNe-1X
Supervised (sofimas)
73.31%
50.90%
55.30%
RN-50
150 2
2015
ResNet
ImageNet-1K
Supervesed (sottmas)
80.861
29.445
53.72%
RN 101
135 61 31
2015
ResNeet
ImageNet-1X
Supervised (sofimas)
$1.89%
47.40%
52.061%
RN-152
ResNet
ImageNes- IK
Supervised (sofmmar)
82.28%
46.61%
50,74%
20196
ResNect
ImageNes11
Supervised (sotimas)
73.56%
50.36%
53.95%
RN-152
Kocnolil 3.
A019
ResNet
ImageNer: LK
Supervised (solmaa)
79.29%
50.95%
56.011
RN-50
Acenonii 10
3019
ResNet
ImageNer-1K
Supervsed (sottmas)
76.993%
51.00%
56.05%
(dropout)
Jcenooin
ResNet
ImageNet. LK
Supervised (softmaxe)
77.42%
51.26%
40%
RN- (extra WD)
Korbbluhh
ImageNe: IK
Supervised (sofimace)
77.82%
52.62%
56. 56.1
RN-50 (label smnoothing)
Arcmoni 24 mk
ResNer
Im
Supervise (sottmaxe)
77.6.3%
45.785
55.52%
RN 50 (logit penality)
Acenobii 2 M1.
3041
ResNet
ImageNer-
Supervesed (sotimaxe)
77.67%
47.67%
54.21%
RN- (mixup)
Jombblih
ResNet
ImageNer. LK
Supervised (sofmmae))
77.92%
66.70%
56.29%
RN-50 (AunoAugmest)
3121
ResNet
ImageNet-1
Supervised (sotimax)
77.64%
50.67%
10.1
RN- 50 (logit norm)
KeenbliB 24 d.
A04
ResNet
ImageNet. L.
Supervised (sotimaxe)
77.83%
51.05%
53.63%
RN-50 (cosine softmax)
Kcemnniii 10
ResNer
Im
Supervied (sottmax))
77.86%
52.82%
56.73%
(siganold)
ResNet
ImageNer-1
Supervised (sqgmoid)
78.18%
44.72%
33.3
RN- 50 (sotimas)
Kombiii
ResNet
ImageNer- LK
Supervised (sotimas)
76.94%
50.39%
55.97%
RN-50 (squared enor)
Kocmooii 2 m0
ResNer
ImageNes-1K
Supervised (sq] error)
37.13%
41.04%
49.87%
RN-50
H2b21
2021
ResNet
scoset
Supervised (sottmas)
46.50%
50.63%
ResNeXi 50 32x4d
ResNeXi
ImageNer LK
Supervised (sofimax)
81.20%
46.97
51.40%
ResNeXi-101 32x8d
ResNeX:
ImageNet-1
Supervised (sofimas)
81.89%
50.82%
VGG-11
Sulnayya & Absemtin
2013
VGG
ImageNet- LK
Supervised (sottmax)
09.024
52.04%
35.91%
VGG-13 13
VGG
ImageNer LK
Supervised (sotimax)
69.93%
52.24%
55.84%
VGG-16
VGG
ImageNet-1K
Supervised (sothmax)
71.59%
52.09%%
56.
1GG-19
Suooeyas Zi5entas
VGG
ImageNer-1X
Supervised (sotimax)
72.38%
52.09%
55.80%
VGG-16
Maller olle & Motun
VOG
Fonwet
Supervised (sofmaax)
31,9648
53.59%
VIT-B/16 IIK
Steme 2
VIT
ImageNer-1K
Supervise (siguood))
77.66%
38.31%
50.48%
12K
Stllee
VIT
ImageNet 21K
Supervised (sqgmoid)
83.77%
44.62%
36.49%
VIT-B
VIT
ImageNot-1 IK
Supervised (signsid)
72.08%
37.45%
46.09%
VIT-B/32 121K
VIT
ImageNes-21K
Supervised (signood)
79.16%
44.74%
56.79%
V/E-L/16 IIK
Soitee
VIT
ImageNer. LK
Supervised (sggmoid)
75.111
37.00%
5 11.42%
VIT-L/16 121K
VIT
ImageNes-21K
Supervised (signoid)
83.13%
40.99%
51.27%
VIE-S/32 IIK
VII
ImageNet-1K
Supervised (sqgnnod)
72.18%
48.31%
VIT-S/32 DIK
Namee 15 m1
VIT
Imi
Supervised (sigmeid)
72.93%
46 16%
56 566
VIT-GH14 IFT
Zia 01
VIT'
JFT.38
Supervised (signnnid)
89.01%
53.94%
61.18%
VIL---16
2011
VIT
ImageNer-IK
Supervisd (sottmax)
81.07%
42.02%
47.89%
VIT-B-12
Dowwiieyy of
7027
VIT
ImageNis-1K
Supervised (sofimax)
75.91%
42.525
48.69%
Table B.I: Pretrained neural networks that considered in our analyses. "RN" ResNet, "Self-sup."
superviie "contr"" contrastive. "RN-50 (extra WD)" is a ResNet-50 with higher weight decay on the final network layer. The "ImageNet" column contains the accuracy on the ImageNet dataset. The "Zero-Shot" column contains the THINGS zero-shot odd-one-out accuracy of the better of the embedding and logits layer. The "Probing" column contains the THINGS probing odd-onc-out accuracy of the embedding layer.
21

---

Published as a conference paper at ICLR 2023
Figure B.1 shows the odd-one-out accuracy as function of layer depth in neural network for few different network architectures. Later layers generally perform better which is why we performed analyses exclusively for the logits peauliimatc/emboddingg layers of the models in Table[B.] Figure [.2 compares the odd-one-out accuracy of using dot product versus cosine similarity and shows that cosine similarity generally yields better alignment.
C CIFAR- 100 TRIPLET TASK
Figure C.I: An example triplet from the CIFAR-100 coarse dataset. The left two images are from one of the two CIFAR-100 "vehicle" superclasses, so the rightmost image is the odd-one-out.
In a similar vein to the THINGS triplet task, we constructed a reference triplet task from the CIFAR- 100 dataset (Krizhevsky & Hinton 2009). To show pairs of images that are similar to each other, but do not depict the same object, we leverage the 20 coarse classes of the dataset rather than the original fine-grained classes. For each triplet, we sample two images from the same and an one odd-one-out image from a different coarse class. We restrict ourselves to examples from the CIFAR- 100 train set and exclude the validation set. We randomly sample a total of 50, 50,000 triplets which is equivalent to the size of the original train set. Figure C shows an example triplet for this task.
We find that ImageNet accuracy highly correlated with odd-one-out accuracy for the CIFAR- 100 coarse task (see Figure C.2 r = 0.70), which is in stark contrast to its cor- relation with accuracy human odd-one-out judgments, which is significantly weaker (see Figure
Ord-onee out accuracy
Sueenses (Pi/
The main reason for constructing this task was to exam- ine whether or not any findings from comparing human to neural network responses for the THINGS triplet odd- one-out task can be attributed to the nature of the triplet task. Instead of using the CIFAR-100 class labels, we specifically used the coarse super-classes that are possibly comparable to higher-level concepts that are relevant to human similarity judgments on the THINGS odd-one-out task. Hebart et al. (2020) and Muttenthaler et al.] (2022) have shown that humans only use small set of concepts for making similarity judgments in the triplet odd-one-out task. These concept representations are sparse representations. That is, there are only i objects that are important for concept, where < m. Recall that m denotes the number of objects in the data (c.g.. 1854 for THINGS). The importance of objects is defined in Hebart et al. (2020) and Muttenthaler al. (2022). Similarly, the coarse super-classes CIFAR- 100 are sparse. Although the CIFAR- 100 triplet task may be dif- ferent, we believe that additionally testing models on this task is one reasonable way figure out whether findings (e.g., the correlation of ImageNet accuracy with triplet odd-one-out accuracy) are attributable to the nature of the triplet task itself rather than variables related to alignment.
2.4
0.5
0.3
5.7
0.8
0.2
ImageNet accuracy
Figure C.2: Zero-shot odd-one-oo accuracy on triplet task based on CIFAR-100 coarse exhibits a strong correlation with ImageNet accuracy. Diagonal line indicates a least- squares fit.
22

---

Published as a conference paper at ICLR 2023
D TRANSFERABILITY OF PENULTIMATE LAYER VS. LOGITS
In Figure E.2 we show that the logits typically outperform the penultimate layer in terms of zero- shot odd-one-out accuracy. In this section, we perform similar comparison of the performance of the penultimate layer and logits in the context of transfer learning. We find that, contrary to odd-one-out accuracy, transfer learning accuracy is consistently highest the penultimate layer.
Following Kornblith et al. 2201bb, we report the accuracy of fg-regularized multinomial logistic re- gression classifiers on datasets: Food-101 dataset (Bossard et al. 2014), CIFAR-1O and CIFAR- 100 (Krizhevsky & Hinton 2009), Birdsnap (Berg et al. [2014). the SUN397 scene dataset (Xiao ct al. 2010), Stanford Cars (Krause et al. 2013). FGVC Aircraft (Maji et al. 2013), the PAS- CAL VOC 2007 classification task (Everingham al. 2010), the Describable Textures Dataset (DTD) (Cimpoi et al.] 2014), Oxford-IIIT Pets (Parkhi et al. 2012). Caltech-10) (Fei-Fei et al.
2004). and Oxford 102 Flowers (Nilsback & Zisserman 2008 We use representations of the 16 models previously studied by Kornblith et al. (2019b) (see Table B. and follow the same procedure for training and evaluating the classifiers.
Results are shown in Figure D.I For nearly all models and transfer datasets, the penultimate layer
provides better representations for linear transfer than the logits layer.
Food-101
CIFAR-10
CIFAR-100
SUN397
Stanford
0.08
Logis
0.75
0.75
0.70
0.80
0.5
FGVC Aircraft
VOC2007
DTD
Oxford-III Pets
Callech-101
102 Flowers
0.60
0.74
0.94
0.84
0.58
0.82
0.72
0.82
0.92
0.50
0.63
0.5
0.03
0.85
0.70
075
0.575 0.900 0.925
0.50
0.55
0.50
0.35
Entoddingg
Enbeddings
Emieddinge
Embedddngg
Embeddinga
Emiedddngg
Figure D.I: Penultimate layer emheddings consistently offer higher transfer accuracy than the logits layer. Points reflect the accuracy of a multinomial logistic regression classifier trained on the penultimate layer em- beddings (r-axis) or logits (y-axis) of the 16 models from Kornblith et al, (2019b). which were all trained with vanilla softmax cross-entropy. Dashed lines reflect
23

---

Published as a conference paper at ICLR 2023
E LINEAR PROBING
In Figure E.1 we compare probing odd-one-out accuracy with zero-shot odd-one-out accuracy for models pretrained on ImageNet-1K or ImageNet-21K, We observe a strong positive correlation of 0.963 between probing and zero-shot odd-one-out accuracies.
Logits
Probing odd -o---oe-- -utt accuracy
0.49
0.45
0.41
0.37
Supervised [ImageNet-1K,
Supervised (maggeNet-210
0.33
Zero-shot odd-one-out accuracy
Figure E.1: Probing odd-one-out accuracy function of zero-shot odd-one-out accuracy for the logits layer of all ImageNet models in Table B.I
In the top left panel of Figure E.3 we show probing odd-one-out accuracy as function of ImageNet Similarly to the findings depicted in the top left panel of Figure we observe a low Pearson correlation coefficient (r 0.213). The remaining panels of Figure F.3 visualize probing odd-one-out accuracy function of ImageNet accuracy for the same subsets of models as shown in Figure 2 Again, the relationships are similar to those observed for zero-shot odd-one-out accuracy in Figure
accuracy for all models in Table B.1
Probing accuracies show less variability than zero-shot accuracies among the networks trained with the different loss functions from Kornblith al. (202T) (Figure top center), although cosine soft- max, which performed best for the zero-shot setting, is also among the best pest--erforming models here. Moreover, probing reduced the differences between different Siamese self-supervised learning S models (Figure bottom left), although Siamese models still performed substantially better than the non-Siamese models. Yet, as our analysis of zero-shot accuracy ($4.1). architecture (Figure top right) model size (Figure [acurre[/bttonn right) did not affeet odd-one-out accuracy. These findings hold across every dataset we have considered in our analyses (see Figure E.4).
Whereas the logits often achieve higher degree of alignment with human similarity judgments than the penultimate layer for zero-shot performance, alignment nearly always highest for the penultimate layer after applying W the transformation matrix learned during linear probing to a model's raw representation space and then performing the odd-one-out task or RSA (see Figure H.2
24

---

Published as a conference paper at ICLR 2023
(Hebart et el., 2020)
(Cichy et al., 2019)
(Kng al., 2019)
OPT 9401 K15
Al9 Perultinaze
Penuitimate
Figure E.2: Performance of the logits vs. penultimate layer for all models across all three datasets without (top row) and with (bottom row) applying the transformation matrix obtained from linear probing model's representation space. z-axis and g-axis represent odd-one-out accuracy for Hebart (2020) and Spearman's for Cichy et al.] (2019) and King et al. (2019). Networks are colored by their loss function. "softmax+" indicates softmax cross-entropy with additional regularization or normalization. Dashed lines indicate y = r.
Varping objectice
Verying architertree
Probing odd- onn OuE 30OUSAY
325
633
0.62
Wetural proples
355
635
Scadian
0.53
252
5A2
So
0.55
355
0.50
355
6.2
0.43
3.42
645
0.49
2.43
040
0.32
335
035
0.30
3.30
0.72
6.71
0.30
2:12
872
8.74
0.75
3.75
0.22
ImageNet
ImageNet
ImageNat accuraty
Sellsupervvsed
models
Probing odtd ootd-cne--ot Qut accurac
0.75
Deriw Tala
Sec.l
0.70
6.7
0.25
0.5
VSS
0.40-
Wattel
046
0.55
0.55
033
675
6.70
0.75
6.6
750
ImageNet
ImageNes
#Parameters (Million)
Figure E.3: Probing odd-one-out accuracy as function ImageNet accuracy number model parameters. Top Probing odd-one-out accuracies the embedding layer all models considered our analysis. Top center: Models have the same architecture (ResNet-50) hut were trained with a different objective func- tion (Kornblith et al.
202 T) Top right: Models were trained with the same objective function but vary in architecture (Korblith al 2019b). Bottom left: Performance for different SSL models. Bottom center: Different image/text models with their ImageNet accuracies. Bottom right subset ImageNet models cluding their number of parameters. Dashed diagonal lines indicate a least-squares fit. Dashed horizontal lines reflect chance-leve or cciling accuracy respectively.
25

---

Published as a conference paper at ICLR 2023
A
(Gichy 2019)
Verying abjection
Dreucto
subdens
Spormnns
0A-
0.72
2.25
0.:0
075
oda
Imagelken accurscy
ImageNer
ImageNec socuracy
Self supersipedd models
Image/teet models
Modal
Fytes
Sparrman 5p
wegelss 2010
2.4
01
055
ImageNet acouracy
Image
#Patameters (Million)
(King et al., 2013)
Varying architecture
Desuiee Mnbile
Spcaman's
Farter
3.5
Imagohes accuracy
652
0.5g
0.6g
0.70
0.50
0.12
Imagehiet accuracy
ImageNNet sccurary
1.5
Sell-oupesteed
Imager/eext
Model
Cliseister
VDG
Spearman'S
06
0.4
bugeres 2000
C2-
320
130
ImagsNes accuracy
ImageMet accuracy
#Parameters (Million)
Figure E.4: Spearman rank correlation after applying W to neural net's representation space is weakly correlated with ImageNet accuracy and varies with training objective but not with model architecture number of parameters for both human similarity judgment datasets from Cichy et al.] (2019) and King et all (0119 respectively, Diagonal lines indicate a least-squares fit.
26

---

Published as a conference paper at ICLR 2023
F LINEAR REGRESSION
In this section, we elaborate upon the results that we presented in $4.4.2 in more detail. For each of the 45 representation dimensions j from VICE, we minimize the following least-squares objective
arg min Ag..b 1=1
E (Ajs-(A,.00 (A,. + b,]) 2 +a, A
(3)
MSE where Yn is the value of the 5th VICE dimension for image i, æ; is the neural net representation of image i, and (ij O is regularization hyperparam- eter. We optimize each dimension separately, select- ing 0j via cross validation (see Appendix [A.3), and assess the fit in two ways. First, we directly mea- sure the R2 for held-out images. Second, we evalu- ate odd-one-out accuracy of the transformed neural
Embedding
Regression codd-on--ous accuracy
0.00
0.50
3.45
0.45
Superitedd (JTT3K)
Iroge kot
0.43
Supar vovd Brnogeter
Supariteed (Ecesem
net representations using similarity matrix S with
Sig == (Ax;+b) b) (Aay+tba +b), with A and b obtained from Fq. (i.e., regression odd-one-ow accuracy).
0.20
0.77
sha 252
Probing odd-one-out accuracy
Figure F.I: Regression as a function prob- ing odd-one-out accuracies for all models in Ta- ble [B.]
In Figure F.1 we compare odd-one-out accuracies after linear probing and regression respectively. The two performance measures are highly correlated for the embedding (r 0.982) and logit (r 0.972; see Figure F.3 layers We provide R" values for individual concepts Appendix We observe that the leading VICE dimensions, which are the most important for explaining human triplet responses, could be fitted with an R² score of >> 0.7 for most of the models the quality of the regression fit generally declines with the importance of dimension (see Figure F.4
We compared zero-shot with regression odd-one-out accuracies (as defined in $4.4.2 for the em- bedding layer of all models in Table [B.]] and observe strong positive relationship (r 0.795; Figure [.2). In addition, we contrasted regression odd-one-out accuracy between logit and penul- timate layers for ImageNet models. The results consistent with the findings obtained from the linear probing experiments, shown in Figure 4 Moreover, we observe that probing and regression odd-one-out are highly correlated, thus applying similar transformations to a neural net's represen- tation space. This is demonstrated in Figure E.1 for the embedding layer of all models in Table B.] and in Figure F33 for the logit layers of the ImageNet models. Figure F.4 shows R² scores of the linear regression fit from embedding-laye representations of all models in Table B.1 to the same subset of VICE dimensions that we leveraged for the linear probing experiments in Figure S
Embedding
Regression odd-one-out accuracy
regression odd-one-one accuracy
0.62
0.51
0.57
0.57
Soweeeetttooe Sinee Senewcedd tnateeaeeeeee
0.53
0.53
Logits
0.45
Supervised (JFT 3B)
0.45-
Image/Teat
Supervised Net
Supercised (Ecove)]
0.37-
Supervised Image N30-
0.57
Self-Supervise
0.35
0.37
o.ks
0.39
0.53
0.77
aka
0.33
0.77
0.11
0.
0.49
Zero-shot odd ont accuracy
Embedding
Figure F.2: Left: Zero-shot and regression odd-one-out accuracies for the embedding layer of all neural nets. Right: Regression odd-one-out accuracy for the emhedding and logits layer for all supervised models trained on ImageNet-1 K or ImageNet-21KK
We hypothesize that this is because VICE is trained on all images, and thus the transformation matrix learned in linear regression has indirect access to the images it is evaluated on, whereas the linear probe has no access to these images (see Appendix A.I))
2 Note that odd-one-out accuracies are slightly higher for linear regression (Figure F.I
27

---

Published as a conference paper at ICLR 2023
Logits
Regression odd-one-oul accuracy
0.41
Supervised lmageNett113)
0.37
Supervised (Ecoset)
linageNet-2100
0.38
0.33
0.37
0.41
0.53
057
0.b1
Probing odd-one-out accuracy
Figure F:3: Regression function of probing odd-one-out accuracies the logits layers of ImageNet
models Table B.]
Dimension 4 Animal-related
Dimension 10 Vehicles
Dimension 17 Sports
1.0
0.8
0.6
0.4
0.2
0.0
Baok Algo
Reater
Vco Durtot Matslattor
thepttion
Alattel Effic leittee
Cai Beaoe topy Align you Retter
Wi teptton haster
Dertt Resteet alastee efficie efficientied
RTTP Bluse CUP Allo my
Beriethe Anstted Mo Alestier
Figure F4: R² scores for models Table B.1 after fitting & regeulrare linear regression to predict individual VICE dimensions from the embedding-lay representation of the images in THINGS. Color-coding was determined by training data/objective. Violet: Image/Text. Green: Self superviedd. Orange: Supervised (ImageNet-1K). Brown: Supervised (Ecoset). Cyan: Supervised (ImageNet-2IK) Black: Supervised (JFT-
3B).
G ENTROPY
Let A = (),,)), ki, kj} be the set of all combinations of pairs in a triplet. The entropy of triplet then be written
k() =- E
(a,
where p({a, {i. is derived from the VICE model and is defined precisely in Equation 6 in Muttenthaler et al. (2022).
To understand how aleatoric uncertainty of odd-one-out predictions varies across different models, we calculated the entropy of each triplet THINGS for every model Table B.T and subsequently computed the Pearson correlation coefficient of these per-triplet entropies for every pair of models.
Although models with the same architecture often correlated strongly with respect their aleatoric uncertainty across triplets, not single model achieves a strong positive correlation to VICE (see Figure and G. Interestingly, the choice of the objective function appeared to play crucial role for the entropy-alignmen of neural net with other neural nets or with VICE.
G.1 HUMAN UNCERTAINTY DETERMINES ODD-ONE-OUT ERRORS
Since VICE provides probabilistic model of humans' responses on the odd-one-out task, we can use the entropy of a given triplet's probability distribution to infer humans' uncertainty regarding the odd-one-out. In this section, we evaluate model's odd-one-out classification error function of a triplet's 's entropy. For this analysis, we partitioned the original triplet dataset into sets D Di,..., D11 corresponding to 11 bins. A triplet belongs subset D; if the entropy of the VICE
28

---

Published as a conference paper at ICLR 2023
Logits: Correlation of Entropy
07
ens Rodd
vosus
Was Wnerr
Wes BARTO
KAND Fer
FICE C3440
VITEO2 1810
VISEOS 1210
1020
VTH424 120
VIAT2
Figure G.I: The correlation coefficient of entropy of the output probabilities for each triplet in THINGS.
output distribution for that triplet falls in between the 1)th and 7th bins' boundaries. We define triplet's 's entropy the entropy over the three possible odd-one-out responses, estimated using 50 Monte Carlo samples from VICE (see details Appendix G Note that the entropy for a discrete probability distribution with 3 outcomes lies in [0, log(3).
Unsurprisingly, all models in Table B.I yield a high zero-shot odd-one-out classification eroor for triplets that have high entropy, and all models' error rates increase monotonically human triplet entropies increase. However, most models make a substantial number of errors even for triplets where entropy is low and thus humans and ViTs trained on ImageNet- IK or ImageNet-21IK and SSL models show a similarly high zero- shot odd-one-out error, between 0.1 and 0.3, for triplets with low entropy, whereas ALIGN and in particular CLIP-RN, CLIP-ViL BASIC-L and ViT-G/14 JFT achieve a near-zero zero-shot odd-one-
very certain. We find that VGGs, ResNets, EfficientNets.
error for the same set of triplets (see Figure G%
29

---

Published as a conference paper at ICLR 2023
Embedding: Correlation of Entropy
Make
vo513
net par
2553 Cheg
WIRDN 100
4T2 110
Figure G.2: The correlation coefficient of entropy of the output probabilities for each triplet in THINGS.
Alashlee
VGD+
EllddentWWis
VTT
Zeroenn oxt-ore-onnno
02-
854
05-
0.5
02
Selfauperiseed modela
0.7
0.7
9
Entropy
Figure G.3: Zero-shot odd-one-out prediction errors as function of a triplet's entropy differ across model classes. Top: Logits layer of ImageNet supervised models. Bottom:
Embedding layer of SSL., Image/Text models and ViT-G/14 JFT. Since models with the same architecture, trained the same data (e.g., ImageNet- IK) with the same objective function, perform very similarly in their odd-one-out choices, we aggregated their predictions and report the average. To isolate architecture and training data from any other potentially con- founding variables, we excluded models from Kornblith al. (202 when aggregating the ResNet predictions.
30

---

Published as a conference paper at ICLR 2023
(a) VICE
(b) CLIP-RN
(c) CIT
(d) AlexNet
(e) VGG-16
(f) ResNet-18
(g) 152
(h) VIT-B/32
ii) EfficientNet B3
(i) Re (SimCLR)
(k) ResNet-50 (BarlowTwins)
ResNet-50 (Jigsaw)
Figure H.I: I-SNE visualizations for the embedding layer of subset of the models in Table B. Data points
are labeled according to higher-level categories provided in the THINGS database (Hebart et al. 2019.
H DIFFERENT CONCEPTS ARE DIFFERENTLY DISENTANGLED IN DIFFERENT REPRESENTATION SPACES
In this section, we show t-SNE (Van der Maaten & Hinton 2008 visualizations of the embedding layers of a subset of the models in B.1 To demonstrate the difference in disentanglement of higeer-level concepts- - which are provided in the THINGS database (Hebart et al. 2019)- different representation spaces, we have chosen a representative model for a subset of the architectures and training objectives. Figure H.I shows that the animal concept is substantially more disentangled from other high-level categories in the representation space of image/text models such as CLIP- RN/CLIP- ViT (Radford et al. 202T) than is for ImageNet models. The bottom row of Figure H.I shows that higher-level concepts appear to more distributed and poorly disentangled in Jigsaw, non-Siamese self-supervised model, compared to SimCLR and Barlow contrastive and non- contrastive self-supervised models respectively.
31

---

Published as a conference paper at ICLR 2023
I ODD-ONE-OUT AGREEMENTS REFLECT REPRESENTATIONAL SIMILARITY
To understand whether odd-one-out choice agreements between different models reflect similarity of their representation spaces, we compared the agreement in odd-one-out choices between pairs of models with their linear Centered Kernel Alignment (CKA), a widely adopted similarity metric for neural network representations (Kornblith et al., 2019a; Raghu et al. 2021). Let Rmxxpt and Y gmxp2 be representations of the same examples obtained from two neural networks with and p2 neurons respectively. Assuming that the column (neuron) means have been subtracted from each representation (i.e., centered representations), linear CKA is defined as
vec( (Xx EXT) veo(YY XXX IF DIYYYII |F
CKAlinear X, Y)
Intuitively, the representational similarity (Gram) matrices X and YY measure the similarity between representations of different examples according to the representations contained in X and Y. Linear CKA measures the cosine similarity between these representational similarity matrices after flattening them to vectors.
In Figure I.T we show heatmaps for both zero-shot odd-one-out agreements and CKA for the same pairs of models. The regression plot shows zero-shot odd-one-out choice agreement between all pairs of models in Table B.I as a function of their CKA. We observe a high correlation between odd-one-our choice agreements and CKA for almost every model pair. That is, odd-one-out choice agreements appear to reflect similarities of neural network representations.
Odd-one- out choice agreement
Centered Kernel Alignment
0.
Zero-shot odd one agreementt
07
Centered Kernel Alignment
Figure 1.1: Left: Heatmaps for zero-shot odd-one-out choice agreements and CKA for the same pairs of models. For better readability. we omitted labeling the names of the individual models. Right: Kero-shor odd-one oe--on--ut out choice agreements between all pairs of models a function of CKA plus linear regression fit.
J HUMAN ALIGNMENT CONCEPT SPECIFIC
In Figure J.I and Figure J.2 we compare zero-shot odd-one-out accuracy and R² scores respectively between the best-aligned ImageNet model and the best-aligned overall model. Although VIT-G/14 JFT achieves better alignment with human similarity judgments for most VICE dimensions, the best ImageNet model outperformed ViT-G/14 JFT for small subset of the concepts, e.g., tools s, technol paper, liquids ds. That is, there appear to be some human concepts which Ima- geNet models can represent fairly well without additionally learned linear transformation matrix, even better than a model trained on a larger, more diverse dataset. However, the R² scores show different pattern. In linear regression, representations of ViT-G/14 JFT could clearly be fitted better to the VICE dimensions for every concept compared to the best ImageNet model.
In Figure J33 we show per-concept zero-shot and probing odd-one-out accuracies for all models in Table B. for all 45 VICE dimensions. Whereas zero-shot odd-one-out performances did not show a consistent pattern across the individual dimensions, probing odd-one-out performances clearly demonstrated that image/text models and VIT-G/14 JFT are better aligned with human similarity judgments for almost every concept. However, there are some concepts (e.g., Mindoors-related ob- jects/dimension 8; powdery/dimension 22) for which these models worse aligned than ImageNet models. The difference in human alignment between image/text models plus ViT-G/14 JFT and Im- ageNet models is largest for royal and sports-related objects i.e, dimension and 17 respectively
32

---

Published as a conference paper at ICLR 2023
where image/text models and ViT-G/14 JFT outperformed ImageNet models by large margin. The bottom panel of the figure shows R² scores of the regression fit for each VICE dimension using the embedding layer representations of all models in Table [B. We observe that more important concepts are generally easier to recover. Recall that dimensions are numbered according to their importance (Muttenthaler al. 2022).
In addition, Figure J.4 Figure J.7 show both zero-shot and probing odd-one-out accuracies for all models in Table B. for larger set of human concepts as we do in the main text, including the most important THINGS images/categories for the dimensions themselves.
Zero-sho shot odd-one-o accuracy
0.80
0.75
0.70
0.86
0.80
0.55
VIT-G/14 224
RN50 (Cosine softmax)
Metal (1
Playtre Pleptrraatte corrrreted (2)
Artinal- reraatedd (e)
(2)
fondirre (5)
Ouidoors fosted (9)
Cleiting (0)
Buily Meniiies (1))
(8)
Tool' 2i
(+)
Preteenss Hhy
(a)
Sportss Papper (ee)
(e)
(P)
Lipauts (18)
(20)
Powrde (2))
Cit)
Hisapon' (d4)
(20)
gentieg City
Sky rerreleedd (2T
Long (20)
(26)
White (2)
Mositel inntunnent (a))
Sphetie (S CD
(lD)
Pattoried che)
Fin rerrretted (80)
Blgg. diy
String roldted (de)
haga Elorgahie Blorgated clo)
(gniy (T))
Home Top--nittend (42)
(lo)
Mediche envenaatedd (la)
(41
Yallow (4))
TraWWinter (45)
Colortt
Top
Dimension
Figure J.1: Comparison of shot odd-one-ou accuracy between the best ImageNet and the best overall model for all VICE dimensions.
RN50 (Cosine softmax)
VIT-G/14 224
0.6
R²
0.4
Metal
Plant Mant-relared (4)
Animal rollaaed
Food (2)
Fumitura
outdoars- pelared
Clothing (6)
Bedy part (a)
(7)
Veriictes (10)
(a)
Word (11)
Technology (14)
Toal (12)
Cabrribl
Parterna Circular (e))
(15)
Sports (17)
Liquids (19)
(9)
Sec (20)
Red (1t)
Higght Horeee (24)
Hes graiii
(23)
sky tegfii (28)
Black Bltered (27)
(d6)
Sppherii Sprerical ED
Musical Instrument rnd (aa)
(LO)
(d))
Grien (32)
Fire- stre-flltt Chint (37 Gel
(33)
Armala peppetaiii (40)
(io)
Hom Toy ralated (d))
(do)
Medicine CeENiieer (46)
(41)
String deneeeeeeee
Figure J.2: Comparison of the regression R² -ssoress between the best ImageNet and the best overall model for
all 45 VICE dimensions.
33

---

Published as a conference paper at ICLR 2023
1.0
ImageeTeet
Zero Zero-shot shot odd-one-ous accuracy
0.9
Superissd Brragehis- 1KI
Stiperr dood (Iragehiee2 21)
Stuperr rosed (1FT-33)
0.8
Seefh Superited
0.7
0.1
Matol (1)
mndeon (s)
(1)
motlii (1)
(7)
(o) to (o))
oi (75)
(1t)
Le
horter (oy
porr Toy retseee 1413
(l0)
Medicii WerWer (46)
Ut chdy
Fin coeeen
Bert
Outdioons
Armil te
Inaxgalive
Supervioo draagchic
Probing odd- aall-ne--on odd-one-on out accuracy
Oragghhss 21K)
Euper
0.7
0.6
0.5
0.4
0.3
Piert catate (C)
Food (o)
(o)
to)
Podted to
Be Priat (oo)
(O)
souce (4 (o)
cal (2))
T (
Core and (S)
Eiiing velieel do
Ammic rolnted ()
(S))
(S0)
Toy poprelsted (4))
Medicing La))
Vancoe (l.)
CetWereer (45)
Malaar
Fire vevtieee Sinnt
Quidourt
Horre
Musical
Sup sepperviied (JFT:3B)
Image/Text
Sup ervised (ImageNet 1K)
Sup Supervised (Ecoset)
Sup ervised (ImageNet 21K)
0.6
Self- Supervised
R2
0.4
0.2
Metal(1)
Animal Emmmal-eiaa deleeaaatted (4)
Food (1)
Firri (5)
Outdoors relatedd (a)
Clotting (o)
Bbefy peee (o)
Bechi dotteen (o
(01
sty Sonng Cly
Sogiil Grean I
Strryy (3 Got
Toy letedd vellow ()
Medicine- Meniieee (85)
Por
Ver Wi
Reid
Musical atAee
Fire. vel
Strine rolitidd
Home felated
Elengated
Figure J.3: Top: Zero-shot odd-one-out accuracies all models in Table B. for all 45 VICE dimensions, Middle: Probing odd-one-out accuracies of all models in Table B.1 for all 45 VICE dimensions. Bottom: R² scores for all models in Table B.I after fitting an fz-regularized linear regression to predict individual VICE dimensions/human concepts.
34

---

Published as a conference paper at ICLR 2023
Dimension Metal
Dimension 2/ Food
Dimension 3/ Plant-related
Zoro-dhot 005 one out a000b0))
0.9
0.
0.0
0.6
Fidling ofe sccurse
0.8
Dimension 4/ Animall-ellttdd
Dimension 5 Furniture
Dimension 6/ Clothing
2:00 Shot 000 one oet boordde
0.9
0.8
Fiaiig old 010 out accurscr
0.0
0.8
0.3
0.2
Figure J.4: Zero-shot and probing accuracy for triplets discriminated by VICE dimensions 1-6, following the approach described in $4.4 Color-coding determined by training data/objective. Violet: Image/Text. Green: Self-supervised. Orange: Supervised (ImageNet- IK). Cyan: Supervised (ImageNet-216) Black: Supervised (JFT-3B).
35

---

Published as a conference paper at ICLR 2023
Dimension Royal
Dimension Outdoors- lated
Dimension Body part
2000 thu 200 one oen a00330)
06
0.6
Fidling Podtrgood ole sterri
0.0
0.8
Dimension 10 Vehicles
Dimension 11 Wood
Dimension 12 Tools
Z000- odd wwtoodoo-ee-ooooooo out b0cur001
0.9
0.8
0.7
0.0
Fiaiigg old 0r0 out accurscr
0.8
0.2
Clos
to Spartg
Figure J.5: Zero-shot and probing accuracy for triplets discriminated by VICE dimensions 7-12, following the approach described in $4.4 Color-coding determined by training data/objective. Violet: Image/Text. Green: Self-supervised. Orange: Supervised (ImageNet- IK). Cyan: Supervised (ImageNet-211 Black: Supervised (JFT-3B).
36

---

Published as a conference paper at ICLR 2023
Dimension 13 Technology
Dimension 14/ Colorful
Dimension 15 Pattems
uniti
2000 shut 200 one 000030)
0.8
0.6
0.0
Fidting Podirggodooo ou
0.8
077
Dimension 16 Circular
Dimension 17/ Sports
Dimension 18 Paper
2:00 ald 000 o0 oet board0y
0.9
0.8
Fiaiig 050 010 out ac0ur0
0.6
0.3
02
Figure J.6: Zero-shot and probing accuracy for triplets discriminated by VICE dimensions 13-18, following the approach described in Color--ocn determined by training data/objective. Violet: Image/Text. Green: Self-supervised. Orange: Supervised (ImageNet-1K3 Cyan: Supervised (ImageNet-21) Black: Supervised (JFT-3B).
37

---

Published as a conference paper at ICLR 2023
Dimension Liquids
Dimension / Sea
Dimension 21
m
2000 thut 200 one a00r30)
0.
0.6
Fidting Podtrg ole su
0.0
0.8
Dimension 22 Powdery
Dimension 23/ Hygiene
Dimension 24 Weapons
2:00 Shot 0000 ore oe boardd
0.9
0.8
Frabing oo0 010 out acour3e
0.0
Figure J.7: Zero-shot and probing accuracy for triplets discriminated by VICE dimensions 19-24, following the approach described in Color--oon determined by training dats/objective. Violet: Image/Text. Green: Self-supervised. Orange: Supervised (ImageNet-1K3 IK). Cyan: Supervised (ImageNet-22 21)). Black: Supervised (JFT-3B).
38


Reviews:
[{"review_id": "xiTKlQDLM1", "reviewer": "Reviewer_hcki", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper explores the how human's representation of visual images is aligned with network representations. \nFor this goal, this paper utilized an \"odd-one-out\" test setting. This paper built a sparse Bayesian model to represent human's responses. And utilizes multiple methods such as zero-shot, linear probing and regression. By comparing the choices from the machine features under these distance metrics to human's choices. This paper identified factors in deep models that affect the alignments.  \n", "summary_of_the_review": "I fully respect the effort in this paper and I think all the experiments seem solid.\nHowever, I strongly disagree with the method utilized by this paper to measure the alignment between the two representations. \nI believe one acceptable way would be measuring both representations based on separate signal sources (human representation from EEG etc), then analyze  the correlation. And due to the complexity, human representation should not be only based on one task.\nWe are still not able to build up a real human representation. The models proposed by the paper are based on representations of human behavior (decisions). This representation, however, is only based on observations, not based on cog signals. And even for the representation of human decisions, there can be many different representations, Why is the odd-on-out method the right way?\nIn addition, as also pointed out by the paper in the discussion section, the conclusion on machine/human feature similarity is heavily dependent on the metrics used. With a slight different metric, there are can be totally different conclusions.  \n", "strength_and_weaknesses": "This paper is in good flow so that I can easily follow. \nThe direction of research is inherently interesting and could be of great importance for many researches such as network interpretability. \nThe experiment design is comprehensive. I think this would be a very solid paper if I had agreed with the methodologies provided by the paper (see the reason below). \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "A4nC0BQJIH", "reviewer": "Reviewer_fmAV", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies the degree representations of computer vision models are aligned with human judgements of similarity, and find that changes in model size and architecture have little effect on the degree of a model's alignment, while training data does. The authors show this result via a thorough set of experiments comparing many models and settings, measuring human aligning via annotations from an odd-one-out task. ", "summary_of_the_review": "I think the paper is a great contribution to the ICLR community, both from its thoroughness in experiments and its timeliness as we seek to understand the full benefits of large, pre-trained models and the role of pre-training data. My primary concern is the assumption that the triplet task is fully representative of \"human alignment\" as a general property, which can be addressed with more discussion, and perhaps examples from the annotation task, with which I'm happy to increase my score. ", "strength_and_weaknesses": "S1: Paper is very clear and easy to follow. \n\nS2. The paper conducts a very systematic and thorough investigation across a variety of different models architectures and settings, thus strongly supporting their claim that scale does not result in improved human alignment. \n\n\nW1: Why is the odd-on-out method the right way to measure human alignment of representations (versus pair similarities as in Peterson (2018) or other approaches)? Given that the paper's primary goal is to make a thorough claim on what properties (e.g. scale, objective) affect the learned representations alignment with humans, there is currently not much discussion on why the triple task is the best way to measure alignment. \n\nW2 (minor): The reference triplet task for CIFAR-100 appears quite different in spirit from the triplet task with human responses on the THINGS dataset, so I'm not sure how useful it is as a baseline to compare with. The process itself is explicitly \"coarse\" class driven, unlike the human judgements which aren't necessarily driven by class/object. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "8ZxLRnMNIqP", "reviewer": "Reviewer_u84n", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors investigated the alignment between DNNs' representations of objects in the THINGS dataset and humans' judgments of those same images. They found that the newest and largest-scale models trained on larger-than-imagenet datasets like the JIT-300 or trained with caption embeddings (e.g., CLIP) performed the best. They complement this finding with several surveys to identify the architectural/training choices that drive performance on THINGS triplet predictions, and analyze the concepts captured by DNN representations. \n\n\nUpdated:\n\nThe additional experiments done over the rebuttal period strengthen this paper and make it a good and rigorous contribution for AI and the brain sciences.", "summary_of_the_review": "I think the paper is either a preliminary report pointing towards bigger questions about understanding human vision and how DNNs can be improved as models of human vision or, in its current form, a solid workshop paper. There is nothing here that changes my thinking about human or machine vision and so I do not feel strongly about supporting its acceptance into ICLR.", "strength_and_weaknesses": "Strengths\n- Awesome introduction.\n- I am all for evaluations that assess how well today's models are explaining human perception, so I appreciate that contribution.\n\n\nWeaknesses\n- There's no finding here that moves the field of cog/vis sci or AI. The authors look at odd-one-out accuracy on the things dataset. They test a bunch of models. Some do better than others.\n- The potentially most interesting finding is that \"scaling ImageNet\nmodels does not lead to better alignment of their representations with human similarity judgments.\" However, the authors also found that pretraining models on bigger-than-imagenet datasets (like JIT-300) does improve alignment. Of course those large-data models are also bigger, so the deleterious effect of scale that they found can be easily counteracted by pursuing the scaling laws that have taken hold of the field today — there's no novel prescription that falls out of this work.\n- The gap that the authors note between the optimal score of 67% and models' scores of 57-58% is numerically small but potentially important. However, unlike similar work like Geirhos et al., 2021 \"Partial success in closing the gap...\" which focused on OOD generalization for humans and models, it's a stretch to find a similar utility for the things dataset triplet scores.\n\n- The analysis on what model specs affect odd-one-out accuracy could be refined. \"Varying objective\" is a combination of varying losses, augmentations, and regularizations. \"Varying architecture\" similarly has a lot of stuff going on at once. \"Model size\" is not just model size, but rather confounded with the many different bells-and-whistles in ResNet vs. VIT vs. VGG. I think this analysis should be redone. Choose one model architecture, like ResNet, and in separate plots, look at how different losses affect performance vs. different regularizations vs. different augmentations vs. varying depth vs. varying width vs. varying self supervision objective. Right now there's too many hidden differences between the models that go into each of the subplots in Fig. 3 to interpret those results.\n- The authors describe odd-one-out accuracy as being higher in the final vs. penultimate layer of the network. It would be helpful to understand why this finding is significant. Do we learn anything about human vision based on which layer the accuracy is the highest?\n\nMinor\n- The results are written to sound like extended figure captions (\"The plot in the bottom left corner of Figure 3 compares...\"). It would help readability if the results and findings were woven into the overall narrative, and figures were referenced parenthetically.\n- I think it would help interpretability if you plotted scores normalized by the optimal performance. In other words, models achieve X% of the optimal score \n- \"Prior work has investigated this question indirectly, by measuring models’ error consistency with humans (Geirhos et al., 2018; Rajalingham et al., 2018; Geirhos et al., 2021) and the ability of their representations to predict neural activity in primate brains (Yamins et al., 2014; Guc¸l ¨ u &¨ van Gerven, 2015; Schrimpf et al., 2020), with mixed results.\" Can you add a sentence clarifying what you mean by mixed results.\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster