Improving neural network representations using human similarity judgments
Authors: Lukas Muttenthaler
Lorenz Linhardt
Jonas Dippel
Robert A. Vandermeulen
Katherine Hermann
Andrew Kyle Lampinen
Simon Kornblith
Venue: NeurIPS 2023 poster (2023)
Keywords: representational alignment; human similarity judgments; neural networks; representation learning; few-shot learning; anomaly detection
TL;DR: We improve the transferability of neural network representations to downstream tasks by aligning them with human similarity judgments and ensuring that local similarity structure is preserved.

Abstract: Deep neural networks have reached human-level performance on many computer vision tasks. However, the objectives used to train these networks enforce only that similar images are embedded at similar locations in the representation space, and do not directly constrain the global structure of the resulting space. Here, we explore the impact of supervising this global structure by linearly aligning it with human similarity judgments. We find that a naive approach leads to large changes in local representational structure that harm downstream performance. Thus, we propose a novel method that aligns the global structure of representations while preserving their local structure. This global-local transform considerably improves accuracy across a variety of few-shot learning and anomaly detection tasks. Our results indicate that human visual representations are globally organized in a way that facilitates learning from few examples, and incorporating this global structure into neural network representations improves performance on downstream tasks.

Improving neural network representations using human similarity judgments
Lukas Muttenthaler'
Lorenz Linhardt Machine Learning Group Technische Universität Berlin
Jonas Dippel
Google DeepMind
Machine Learning Group Technische Universität Berlin
Machine Learning Group
Technische Universität Berlin
BIFOLD
BIFOLD Berlin, Germany
BIFOLD
Berlin, Germany
Berlin, Germany
Robert A. Vandermeulen
Katherine Hermann Google DeepMind Mountain View, CA. USA
Andrew K. Lampinen Google DeepMind London, UK
Machine Learning Group
Technische Universität Berlin
BIFOLD Berlin, Germany
Simon Kornblith
Google DeepMind Toronto, Canada
Abstract
Deep neural networks have reached human- performance many computer vision tasks. However, the objectives used train these networks enforce only that similar images are embedded at similar locations in the representation space, and do not directly constrain the global structure of the resulting space. Here, we explore the impact of supervising this global structure by linearly aligning it with human similarity judgments. We find that a naive approach leads to large changes in local representational structure that harm downstream performance. Thus, we propose a novel method that aligns the global structure representations while preserving their local structure. This global-local transform considerably improves accuracy across a variety of few-shot learning and anomaly detection tasks. Our results indicate that human visual representations that facilitates learning from few examples, and incorporating this global structure into neural network representations improves performance downstream tasks.
globally organized in a way
Introduction
Representation learning usually involves pretraining network on large, diverse dataset using one handful of different types of supervision. In computer vision, early successes came from training networks to predict class labels [4
85
28 More recent work has demonstrated the power
70 8. Self-supervised contrastive models learn a space where images lie close other augmentations of the same image [9. whereas image/text contrastive models learn space where images lie close to embeddings of corresponding text [70] 39 68).
of contrastive representation learning 9
29
WWok done part of the Google Research Collabs Programme. Berlin Institute for the Foundations of Learning and Data, Berlin, Germany.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Although these representation learning strategies are effective, their pretraining objectives do not directly constrain the global structure of the learned representations. To classify ImageNet images into 1000 classes, networks' penultimate layers must learn representations that allow examples of given class be linearly separated from representations of other classes, but the classes themselves could be arranged in any fashion. The objective itself does not directly encourage images of tabby cats to be closer to images other breeds of cats than to images of raccoons. Similarly, contrastive objectives force related embeddings to be close, and unrelated embeddings to be far, but if cluster of data points is sufficiently far from all other data points, then the location of this cluster in the embedding space has minimal impact the value of the loss 662 35 91.
Even without explicit global supervision, however, networks implicitly learn to organize high- level concepts somewhat coherently. For example, ImageNet models' representations roughly cluster according to superclasses [1]], and the similarity structure of neural network representation spaces is non-trivially similar to similarity structures inferred from brain data or human judgments The structure that is learned likely reflects a combination of visual similarity
40
92
66
80 49
between images from related classes and networks' inductive biases. However, there is little reason to believe that this implicitlyleaamed global structure should be optimal.
Sagiaal reprssertadtss Dicuuaat Ioa, poor grean
0.72
[ 0.96
Few- shot learning
Model
Anomaly detection
CLIP RN50 IWIT!
CLPV 4/44
0.57
CUIVIIFUU4 (LAVON-DDOM)
Granffores (prserecd but reproved (Mbdi SIAE
ELINVIT-L/4 (LOIN-28)
0.93
0.52
Transform
0.47
original
nave
0.42-
glocal
0.9
0.52
0.57
Dlotron
Human alignment
Human alignment
(a) Conceptual cartoon.
(b) Downstream task performance vs. human alignment.
representation space, which improves overall performance. (ap The original representations capture local structure, such as that different trees are similar, but have poor global structure. The glocal transform preserves local structure, while integrating global information from human knowledge; e-g., unifying superordinate categories, organizing by "animacy", or connecting semantically-related categories like "food" and "drink". (b) The glocal transforms improve both human alignment and downstream task performance compared to original and naively aligned representations for image/text models. We report mean accuracies on anomaly detection and (5-shot) few-shot learning tasks.
Figure 1: Global- (gLocal) transforms yield a best-of-botte
Human and machine vision differ in many ways. Some differences relate to sensitivity to distortions and distiiiiibttion generalization [2 22 25). For example, ImageNet models are more biased by texture than humans in their decision process 22 However, interventions that reduce neural networks' sensitivity to image distortions are not enough to make their representations as robust and transferable as those of humans (23). Experiments inferring humans' object representations from similarity judgments suggest that humans use a complex combination of semantics, texture, shape, and color-related concepts for performing similarity judgments [3 [60]
Various strategies have been proposed to improve representational alignment between neural nets and humans [e.g., (66 67 2 [1]. to improve robustness against image distortions [e.g.. 34 21 or for obtaining models that make more buman-like errors [c.g.. [2 26). Muttenthaler et al. [6]] previously
found that a linear transformation learned to maximize alignment on one dataset of human similarity judgments generalized to different datasets of similarity judgments. However, it remains unclear whether better representational alignment can improve networks" generalization vision tasks.
Here, we study the impact of aligning representations' global structure with human similarity judgments on downstream tasks. These similarity judgments, collected by asking subjects to choose the odd-one-out in a triplet of images
(32) provide explicit supervision for relationships among disparate concepts. Although the number of images we use for alignment is three to six orders of magnitude smaller than the number of images in pretraining, we observe significant improvements. Specifically, our contributions are as follows:
2

---

We introduce the gLocal transform, a linear transformation that minimizes combination of global alignment loss, which aligns the representation with human similarity judgments, and a local contrastive loss that maintains the local structure of the original representation space.
We show that the gLocal transform preserves the local structure of the original space but captures the same global structure as a naive transform that minimizes only the global alignment loss.
We demonstrate that the gLocal transform substantially increases performance on a variety of few- shot learning and anomaly detection tasks. By contrast, the naive transform impairs performance.
We compare the human alignment of gLocal and naively transformed representations on four human similarity judgment datasets, finding that the gLocal transform yields only marginally worse alignment than the naive transform.
2 Related work
How can we build models which learn representations that support performance on variable down- stream tasks? This question has been core theme of computer vision research [9] 27 588, but the impact of pretraining on feature representations is complex, and better performance does not necessarily yield more transferable representations [e.g.. 451 For example, some pretraining leads to shortcut learning 23 50 22
34 3 91I. Because the relationship between training methods and representations is complicated, it is useful to study how datasets and training shape representations [33]. Standard training objectives do not explicitly constrain the global structure of representations; nevertheless, these objectives yield representations that capture some aspects of the higher-order category structure [e.g.. 40) and neural unit activity [e.g.. 92, 80) of human and animal representations of the same images. Some models progressively differentiate hierarchical structure over the course of learning [5] in similar way to how humans learn semantic features [73 [8] 78| 5 [79.. Even so, learned representations still fail to capture important aspects of the structure that humans learn [6). Human representations capture both perceptual and semantic features 6,6 including many levels of semantic hierarchy (e.g. higher-level: "animate" [11. superordinate: "mammal", basic: "dog", subordinate: "daschund"), with bias towards the basic level [74 55 38).
as well as cross-cutting semantic features 84 56].
While models may implicitly learn to represent some of this structure, these implicit representations may have shortcomings. For example, Huh et al. (37 suggest that ImageNet models capture only higher-level categories where the sub-categories are visually similar. Similarly, Peterson et al. [66] show that model representations do not natively fully capture the structure of human similarity judgments, though they can be transformed to align better 166 2 In some cases, language provides more accurate estimates of human similarity judgments than image representations [54), and image- text models can have more human-aligned representations 61]. How does human alignment affect downstream performance? Sucholutsky & Griffiths [87] show that models which are more human- aligned (but not specifically optimized for alignment) are more robust on few-shot learning tasks. Other work shows benefits of incorporating higher-level category structure [51] [33. Here, we ask whether transforming model representations align with human knowledge can improve transfer.
3 Methods
Data. For measuring the degree of alignment between human and neural network similarity spaces, we use the THINGS dataset, which is a large behavioral dataset of 4.70 million unique triplet responses crowdsourced from 12,340 human participants for 1854 natural object images 32] Images used for collecting human responses in the triplet odd-one-out task were taken from the THINGS object concept and image database [30] which is a collection of natural object images.
Odd-one-out accuracy. The triplet odd-one-out task is a commonly used task in the cognitive sciences to measure human notions of object similarity without biasing a participant into a specific direction 20 72 31 60). To measure the degree of alignment between human and neural network similarity judgments in the THINGS triplet task, examine the extent to which the odd-one-out can be identified directly from the similarities between images in models' representation spaces. Given representations #1, Ø22 and 223 of the three images in triplet, first construct similarity matrix S E 3x3 where Si,j := x. æ ([|æ;]] || æ., l12). the cosine similarity between a pair of representations We identify the closest pair of images in the triplet as arg max,,>i S.jj with the
'We use cosine similarity rather than the dot product because Muttenthaler et al. [6 have shown that it nearly always yields similar or better zero-shot odd-one-ou accuracies.
3

---

remaining image being the odd-one-out. We define odd-one-ou accuracy as the proportion of triplets where the odd-one-out matches the human odd-one-out choice.
Alignment loss. Given an image similarity matrix S and a triplet {i.j, k} (here, images are indexed by natural numbers), the likelihood of a particular pair, {a, b} [i,j, being most similar, and hence the remaining image being the odd-one-out, is modeled by the softmax of the object similarities,
pl {a, Hi j. k). S) = exp(S..t) (exp(S + eep((S,. + exp(S,,)))
(1)
For triplet responses we use the following negarive log-likelihodd, precisely defined in [60]
Gyloball (S) ==
Σ
k: S).
odd-one-our prodiction
Since the triplets in [32 consist of randomly selected images, the concepts that humans use for their similarity judgments in the THINGS triplet odd-one-out task primarily reflect superordinate categories rather than fine- ane-grained object features [3 60), the above alignment loss can be viewed as a loss function whose objective is to transform the representations into a glloball-restructtuee human similarity space where superordinate categories are emphasized over subordinate categories.
Naive transform. We first investigate a linear transformation that naively maximizes alignment between neural network representations and human similarity judgments with L2 regularization. This transformation consists of square matrix W obtained as the solution to
arg min Cglobal (S) + \|| lp:: W.b
where Si₃- (Wx; + b) (Wa, + b). We call this transformation the naive transform because the regularization term helps prevent overfitting the training set, but does encourage the transformed representation space to resemble the original space. Muttenthaler et al. [61] previously investigated similar transformation. We determine A via grid-scarch using k-fold cross-validation (CV). To obtain a minimally biased estimate of the odd-one-out accuracy of the transform, we partition the 1854 objects in THINGS into two disjoint sets, following the procedure of Muttenthaler et 66..
Global transform. The naive transform does not preserve representational structure that is irrelevant to the odd-one-out task. The global transform instead shrinks W toward a scaled identity matrix by penalizing min I az I thus regularizing the transformed representation toward the original. The global transform solves the following minimization problem,
arg min Cglabal + À min | - WWoTllf arg min felabbal (S) + W SES-Wail W.b
W.b
We derive the above equality in Appx. Again, we select X via grid-search with k-fold CV.
goca transform. In preliminary experiments, we observed trade-off between alignment and the transferability of a neural network's 's human-aligned representation space to downstream tasks. Maximizing alignment appears to slightly worsen downstream task performance, whereas using a large value of À in Eq. 4 leads to a representation that closely resembles the original (since lim W ol). Thus, we add additional regularization term to the objective with the goal of preserving the local structure of the network's 's original representation space. This loss term can be seen as an additional constraint on the transformation matrix W.
We call this loss term local loss and the transform that optimizes this full objective the gLocal transform, where global and local representations structures are jointly optimized. For this loss function, we embed all images of the ImageNet train and validation sets [17] in a neural network's p-dimensional penultimate layer space or image encoder space of image/text models. Let Y RRx be a neural network's 's feature matrix for all m images in the ImageNet train set. Let S be the cosine similarity matrix using untransformed representations where S (y) ys) / 3 (22111),))) and let St be the cosine similarity matrix of the transformed representations where
st, = ((wy: y: + b)" (Wy, + b)) /(I y: + b|12| Wy, + b112). Let be a softmax function that transforms a similarity matrix into probability distribution, exp(S.//7)
a (S,T) Σd 1 144s)) exp (Sa/T)

---

where T is temperature and o(S, E (0, 1). We can then define the local loss as the following contrastive objective between untransformed and transformed neural network similarity spaces,
Sieaal (W,
1 ( -)iy log (o(S))))a)
(5)
m²
To avoid distributions that excessively emphasize the self-similarity of objects for small т, we exclude elements the diagonal of the similarity matrices. The final gLocal transform is then,
(Ei W W3s/D) 1|p
arg min (1 a)) Gglobal WWW b) + Goocall (,, b, W.B
W
(6)
alignment
locality preserving
where a is a hyperparameter that balances the trade-off' between human alignment and preserving the local structure of a neural network's original representation space. We select values of and À that give the lowest alignment loss via grid search; see details in Appx. A.2
3.1 Downstream tasks
Few-shot learning. In general, few-shot learning (FS) is used to measure the transferability of neural network representations to different downstream tasks [66] Here, we use FS to investigate whether the gLocal transform, as defined in $3 can improve downstream task performance and, hence, a network's 's transferabilityy, compared to the original representation spaces. Specifically, we perform FS with and without applying the gLocal transforms. For all few-shot experiments, we use multinomial logistic regression, which has previously been shown to achieve near-optimal performance when paired with good representation by fo-d cross-validation, with n2 being the number of shots per class (more details in Appx. A.3).
[88]. The regularization parameter is selected
Anomaly detection. Anomaly detection (AD) is task where one has collection of data considered "nominal" and would like to detect if test sample is different from nominal data. For semantic AD tasks, e.g. nominal images contain cat, it has been observed that simple AD methods using a pretrained neural network perform best 53 16]. In this work, we apply k-nearest neighbor AD to representations from a neural network, a method which has been found to work well [4 53 71]. We use the standard one-vs.-rest AD benchmark on classification datasets, where model is trained using "one" training class as nominal data and performs inference with the full test set with the "rest" classes being anomalous 775
4 Experimental results
In this section, we report experimental results for different FS and AD tasks. In addition, we analyze the effect of the Local transforms on both local and global similarity structures and report changes in representational alignment of image/text models for four human similarity judgment datasets. We start this section by introducing the different tasks and datasets and continue with the analyses.
4.1 Experimental setup
CIFAR-100 coarse The 100 classes in CIFAR- 100 can be grouped into 20 semantically meaningful superclasses. Here, we use these superclasses as targets for which there exist 100 test images each.
CIFAR-100 shift simulates a distribution shift between the normal distribution of the training and testing images CIFAR-100. For each the 20 superclasses, there exist five subclasses. We use the first three subclasses for training and the last two subclasses for testing.
Entity-13 and Entity-30 are datasets derived from ImageNet [1 76). They have been defined as part of the BREEDS dataset for subpopulation shift (77). In BREEDS, ImageNet classes are grouped into superclasses based on a modified version of the WordNet hierarchy. Specifically, one starts the Entiry node of that hierarchy and traverses the tree in a breadth- -arrttth-irttttt manner until the desired level of granularity (three steps from the root for Entity-13 13 and four for Entity 30). The classes residing at that level are considered the new coarse class labels and fixed number of subclasses are sampled from the trees rooted at these superclass nodes - twenty for Entity- and eight for Entity-30. Through this procedure, Entity-13 results in more coarse-graines labels than Entity-30. The subclasses of each superclass are partitioned into source (training) and target (test) classes, introducing a subpopulation shift. For testing, use all 50 validation images for each subclass.
5

---

Nearest Neighbor Preservation
100
Full Representation
Largest PCs
Remaining PCs
1..
80
Untrans.
Untrans.
Untrans.
0.8
Recall
60
Glocal
Glocal
Glocal
0.6
Global
Global
Global
0.4
40
gLocal -naive global
Nave
Nalve
Naive
0.2
20
Untrans
Glocal
Global
Naive
Untrans
Glocal
Global
Naive
Untrans.
Glocal
Global
Naive
0.0
0
10
20
Figure gLocal and global but not naive transforms preserve nearest neighbors in CLIP VIT-L represen- tations. y-axis indicates the percent- age of ImageNet validation images for which the closest image the untransformed space is among the k closest after transformation.
Figure The top principal components (PCs) of gLocal and global representations of ViT-L on the ImageNet validation set resemble those of the naive transformed representation, indicating that they share sim- ilar global structure. The remaining PCs more closely match the un- transformed representation. Left: CKA between full representations. Middle: CKA after setting singular values of each representation to zero for all hut the largest 10 PCs. Right: CKA after setting singular values to zero for the largest 10 PCs, but retaining smaller PCs.
4.2 Impact of transforms on global and local structure
Our goal is to use human similarity judgments to correct networks' global representational structure without affecting local representational structure. Here, we study the extent to which our method succeeds at this goal. To quantify distortion of local structure, we first find the nearest neighbor of each ImageNet validation image among the remaining validation 49,999 images in the original representation space. We then measure the proportion of images for which the nearest neighbor in the untransformed representation space is among the closest images in the transformed representation space. As shown Fig. 2 both the gLocal and global transforms generally preserve nearest neighbors, although the gLocal transform is slightly more effective, preserving the closest neighbor of 76.3% of images vs. 73.7% for the global transform. By contrast, the naive, unregularized transform preserves the closest neighbor in only 12.2% of images. For further intuition, we show the neighbors of anchor images in the untransformed, gLocal, and naively transformed representation spaces in Appx. C
Whereas the local structure of goocal and global representations closely resembles that of the original representation, the global structure instead more closely resembles the naive transformed representation. We quantify global representational similarity using linear centered kernel alignment
LCKA can be thought of as measuring the similarity between principal components (PCs) of two representations weighted by the amount of variance they explain; see further discussion in Appx. G It thus primarily reflects similarity between the large PCs that define global representational structure. As shown in Fig. 3 (left), LCKA indicates that the gLocal/global representations are more similar to the naive transformed representation than to the untransformed representation, suggesting that the gLocal, global, and naive transforms induce similar changes in global structure. We further measure LCKA between representations obtained by setting all singular values to zero except for those corresponding to either largest 10 PCs or the remaining 758 PCs. LCKA between representations retaining the largest 10 PCs resembles LCKA between the full representations (Fig. 3 middle). However, when retaining only the remaining PCs, the gl Local/global representations are more similar to the untransformed representation than the naive-transformed representation (Fig. right).
(LCKA) 43 [4]
PCA (global)
t-SNE (local)
Untransformed
gLocal
Naive
Untransformed
gLocal
Naive
Figure 4: The goocal transformed representation captures global structure of the naive transformed representation, as shown by PCA, but local structure of the untransformed representation, as shown by tSNE with perplexity 10. Visualizations reflect embeddings of 10 images from each of the 260 Entity- 13 ImageNet subclasses obtained from CLIP VÝT-L and are colored by the superclasses. 2D embeddings are rotated to align with the goocal representation using orthogonal Procrustes. The t-SNE fitting process initialized using PCA; we pick the embedding with the lowest loss from 10 runs.

---

In Fig. 4 we visualize the effects of different transformations using PCA, which preserves global structure, and t-SNE, which preserves local structure. As described by Van der Maaten & Hinton [8)] PCA "focus[es] on keeping the low-dimensional representations of dissimilar datapoints far apart," whereas t-SNE tries to "keep the low-dimensional representations of very similar datapoints close together." In line with the results above, the global structure of the glocal representation revealed by PCA closely resembles that of the naive transformed representation, whereas the local structure of the gLocal representation revealed by t-SNE closely resembles that of the untransformed representation.
To complement these analyses, in Appx. B we explore what the alignment transforms alter about the global category structure of the model representations. Generally, categories within a single superordinate category move more closely together, while different superordinate categories move apart, but with sensible exceptions - c.g. food and drink move closer together.
4.3 Few-shot learning
Table 1: 5-shot FS results using the original or transformed representations. t indicates the highest accuracy for each dataset. Results are averaged over 5 runs
Eniity13
Emtity-3
CIFAR1OOCoosSe
CIFAR1O
SUN397
DXTDD
Model Transform
original
gloceal|
original
glocal
original
glaeal
original
glacol
original
placal
original
g ocall
CLIP (WIT)
63.99
67.96
57.86
59,80
64,27
47:43
38.77
39.59
57.21
58.79
50.32
54.95
CLIP-WIT-J14 (WIT)
65.34
71.94
69.977
66.97
68.48
72.22
73.03
69.87
71.13
62.84
63.27
CLIP-VIT-L/4 (1 4000)
65.33
69.02
62:93
65.99
68.88
69.58
73.57
72.98
70.25
71.08
67.81
66.71
CLIP-VITL/14 (LARON-2B)
65.98
73.48
79.01
78.48
71.62
72.621
69.09
In this section, we examine the impact of the gLocal transform on few-shot classification accuracy on downstream datasets. We consider a standard fine-grained few-shot learning setup on CIFAR- 100 SUN397 [90], and the Describable Textures Dataset (DTD, 12], as well as a coarse- coarse-grained setup on Entity- (13,30) of BREEDS 777
In coarse-grained FS, classes are grouped into semantically meaningful superclasses. This is more challenging setting than the standard fine-grained scenario, for which there does not exist a superordinate grouping. In fine-grained FS, training examples are uniformly drawn from all (sub- Jelasses. For coarse-grained FS, we classify according to superclasses rather than fine-grained classes, and choose training examples uniformly at random for each superclass. This implies that not every subclass is contained in the train set if the number of training samples is smaller than the number of subclasses. On Entity- ( [3,30), superclasses between train and test sets are guaranteed to be disjoint due to a subpopulation shift. To achieve high accuracy, models must consider examples from unseen members of superclass similar to the few examples it has seen. Task performance is dependent on how well the partial information contained in the few examples of a superclass can be exploited by model to characterize the entire superclass with its subclasses. Hence, global similarity structure is more crucial than local similarity structure to perform well on this task. We calculate classification accuracy across all available test images of all subclasses, using coarse labels.
We find that transforming the representations via gLocal transforms substantially improves perfor- mance over the untransformed representations across both gearse-grained and fine-grained FS tasks for all image/text models considered (Tab. D. For CLIP models trained on LAION, however, we do not observe improvements for CIFAR-100 and DTT, which are the most fine- fine-grained datasets. For ImageNet models, the gLocal transform improves performance on Entity-(13.30) but has almost no impact on the performance for the remaining datasets (see Appx. D.I. D.I..
4.4 Anomaly detection
Here, we evaluate the performance of representations on k-nearest neighbor AD with and without using the gLocal transform. AD methods typically return an anomaly score for which a detection threshold has to be chosen. Our method is evaluated using each training class the nominal class and we report the average AUROC. Following $3.1 we compute the representations for each normal sample of the training set and then evaluate the model with representations from the test set. We set == 5 for our experiments but found that performance is fairly insensitive to the choice of k (see Appx. D.2). For measuring the distance between representations, we cosine similarity.
In Tab. 2 we show that the gLocal transform substantially improves AD performance across all datasets considered in our analyses. for almost every image/text model. However, as in the fhw-shot setting, we observe no improvements over the untransformed representation space for ImageNet models (see Appx. D.2). In Tab. 3 we further investigate performance on distribution shift datasets,
7

---

where improvements are particularly striking. Here, global similarity structure appears to be crucial for generalizing between the superclasses. Tab. aadditionlly reports current state-of-the-ee (SOTA) results on the standard benchmarks; SOTA results are not available for the distribution shift datasets. SOTA approaches generally use additional data relevant to the AD task, such as outlier exposure data or textual supervision for the normal class [53] B3 36), whereas we use only human similarity judgments, Our transformation also works well in non non-standard AD settings (see Appx. D.2)
Table 2: One-vs-rest nearest neighbor based AD results; with and without transformation. i indicates the highest accuracy for each datasct.
CIFARIO
CTEAR
CIFARIOO-Camse
ImaggNNe30
DTTD
Model Transform
original
glocal
orgiiul
glooal
original
glocel
original
glocal
original
glaneal
CLIP-RNSI (WIT)
89.44
91.19
92.92
86.47
59.29
98. 31
99.15
901.7
92.29
CLIP-VIT- /14 (WIT)
95.14
98.16
91.41
97.19
88.5
95.83
98.91
99.75'
92.00
94,9
CLIP-VIT-L/44 (III))
98.8
38.39
98.66
98.53
97.86
97.77
19.51
99.69
95.54
96.44
98.97
99.117
98.76
98.97'
98.05
98.51
99.70
99.74
94.87
96.78
SOTA
97.34
94.6
Table 3: One-vs-rest AD with a class distribution shift between train and test sets; with and without transformation. i indicates the highest accuracy for each dataset.
Butity-13 original
Eatity30 ociginal
Living-17 original
Nooliving-26 original
CI0ar100-abit
Model's \ Transform
glocal
gloooll|
gLocal
gLocal
origanal
gLocal
CI IP-RNSO (WIT)
90.22
92.03
91.64
93.71
94.62
93.28
87.49
91.09
76.27
901.3
CLIP-VIT WT-L// (WIT)
58.54
93.23
91.31
95.53
97.31
97.71
84.73
92:53
73.69
87.11
CI WTT-----//1A (LAION-400M)
96.56
CLIL-WTE-/14 (LAION-28)
90,33
93,00
92.1
95.45
96.96
97.37
88.82
93.54
39.73
93.94
4.5 Representational alignment
We have shown above that the gLocal transform provides performance gains on FS and AD tasks. Here, we examine whether these performance gains come at the cost of alignment with human similarity judgments as compared to the naive transform, which does not preserve local structure. Thus, we examine the impact of the gLocal transform on human alignment, using the same human similarity judgment datasets evaluated in Muttenthaler et al. [6]] plus an additional dataset H Specif- ically, we perform representational similarity analysis [RSA; 46) between representations of two image/text models - CLIP RN50 and CLIP VIT-L/14 and human behavior for four different human similarity judgment datasets B1 66
RSA is a method for comparing neural network representations to representations obtained from human behavior [46]. In RSA, one first obtains representational similarity matrices (RSMs) for the human behavioral judgments and for the neural network representations (more details in Appx. D These RSMs measure the similarity between pairs of examples according to each source. As in previous work 10, 41 611, we use the Spearman rank correlation coefficient to quantify the similarity of these RSMs. We find that there is almost no trade-off in representational aligument for the gLocal transform compared to the naively transformed representations (see Tab. 4.. Hence, the gLocal transform improve representational alignment while preserving local similarity structure.
10 41
In F we further demonstrate how the representational changes introduced by the linear and gocal transforms lead to greater alignment with human similarity judgments by visualizing the RSMs on each dataset. The global similarity structure captured by the RSMs is qualitatively identical between naive and gooaal transforms, and both of these transforms lead to RSMs that closely resemble human RSMs (see Fig. 5). Human similarity judgments for data from Hebart et al. B1 were collected in the form of triplet odd-one-out choices. Therefore, we used VICE [60] an approximate Bayesian method for inferring mental representations of object concepts from human behavior to obtain an RSM for those judgments. Human RSMs are sorted into five different concept clusters using k-means for datasets from King al. [41] and Cichy al. 10 and using the THINGS concept hierarchy (30) for RSMs from Hebart et al. [31] A visualization for RSMs obtained from CLIP RN50 representations and more detailed discussion on RSA can be found in Appx. E
'Human similarity judgments were collected by either asking participants to arrange natural object images from different categories computer screen [66 10 41 or in the form of triplet odd-one-out choices (31..
8

---

Table 4: The gLocal transform yields both high degree of alignment with datasets of human similarity judgments and good performance FS/AD tasks. Performance on human similarity datasets is measured as odd-one-out accuracy on a held-out test set for THINGS or Spearman's s p for the other three datasets, using either the original representations, the naively transformed representations, or representations transformed via the gLocal transform. For 5-shot FS and AD, we report the average performance across all tasks in Tab.
CLIP-RN5O (WTT)
CLI--VIT-/14 (WIT)
CLIP-VT-L/14 (LATON-25)
Dataset Transform
original
naive
gLocal
original
naive
gooal
original
naive
gLocal
Haman similarity datevets:
47.50%
King al. [
0,386
0.650
0.645
0.355
0.638
0.637
0.292
0.620
0.613
Cichy et 10
0.557
0.721
0.716
0.363
0.732
0.732
0.395
0.735
0.718
Petenson al. 665 66
0.364
0.701
0.705
0.260
0.688
0.688
0.314
0.689
0.660
Dovarstream fasks;
5-shot FS (avg.)
53.05%
41.20%
55.01% %
67.71%
47.62%
69.91
71.10%
50,71%
72.27%
Anomaly detection (avg.)
89.65%
90.75%
91.52%
90.16%
92.79%
95.19%
94.79%
93.09%
96.65% %
Discussion
Although neural networks achieve near-human-level performance on a variety of computer vision
tasks, they may not optimally capture global object similarities. By contrast, humans represent concepts using rich semantic features
including superordinate categories and other global constraints for per- forming object similarity judgments 666
Human
Original
Hebait 00200
41
10
31 60) These repre-
sentations may contribute to humans' strong generalization capabilities [21]
[24]. Here, investigated the impact of aligning neural network represen- tations with human similarity judg- ments on different FS and AD tasks.
12018)
We find that naively aligning neu- ral network representations, without regularizing the learned transforma- tions to preserve structure in the orig- inal representation space, impair downstream task performance. How- ever, our goocal transform, which combines an alignment loss that op-
arg.aal.20000 (2039)
timizes for representational alignment with a local loss that preserves the nearest neighbor structure from the original representation, can substan- tially improve downstream task per- formance while increasing represen- tational alignment. The transformed representation space transfers surpris- ingly well across different human sim- ilarity judgment datasets and achieves almost cqually strong alignment as the naive approach [61], indicating that it captures human notions of object similarity. In addition, it substantially improves downstream task performance compared to both original and naively aligned representations across variety of FS and AD tasks. The gLocal transform yields state-of-the art (SOTA) performance on the CIFAR-100 coarse AD task, and approaches SOTA on other AD benchmarks.
Figure 5: RSMs for human behavior and CLIP ViT-L/14 [WTT]
70
for four different human similarity judgment datasets 31 66
T0
4
We contrast RSMs obtained from the network's original rep- resentations (second column), the naively aligned representations 61 (third column), and the representations + gLocal transform (rightmost column) against RDMs directly constructed from human similarity judgments (leftmost column). Yellower colors indicate greater similarity; bluer colors indicate greater dissimilarity.
blations. In addition, we show that the goccal transform can readily be used combination with other approaches that are specifically designed for few-shot learning with pretrained representations of image/text models - such as Tip-Adapter [93]. We observe considerable improvements compared to using Tip-Adapter without applying the gLocal transform (see Appx. D.I. This indicates that

---

methods designed to improve the transferability of pretrained representations without incorporating knowledge about human global similarity structure can additionally benefit from the gLocal transform. Using a object that is different from the human choice, for optimizing the gloccll transform either yields no change in model's representation space and thus no difference in downstream task performance or changes a model's representation space in a way that substantially deteriorates its downstream task performance (see Appx. D.4). These observations corroborate our findings and suggest that the benefits the gLocal transform most likely stem from including an inductive bias about global object similarity.
tripler dataset, where each odd-one-out choice is determined to be an
Limitations. Our work has some limitations. First, as we show in Appx. D the gLocal transform fails to consistently improve downstream task performance on ImageNet. We conjecture that the gLocal transform can succeed only if representations capture the concepts by which human representations are organized, and ImageNet representations may not. Second, human similarity judgments are more expensive to acquire than other forms of supervision, and there may important human concepts that are captured neither by the 1854 images we use for alignment nor by the pretrained representations.
Conclusion. Our results imply that even with hundreds of millions of image/text pairs, image/text contrastive learning does not learn representation space with human-like global organization. How- ever, since the gLocal transform successfully aligns contrastive representations' global structure using only small number of images, these representations do seem to have a pre-existing representation of the concepts by which human representations are globally organized. Why does this happen? One possibility is that image/text pairs do not provide adequate global supervision, and thus contrastive representation learning is (near- joptimal given the data. Alternatively, contrastive learning may not incorporate signals that exist in the data into the learned representation because it imposes only local constraints. Previous work has shown that t-SNE and UMAP visualizations reflect global structure only if they are carefully initialized 42). Given the similarity between contrastive representation learning and I-SNE/UMAP [5] and the known sensitivity of contrastive representations to initial- ization [52, it is plausible contrastive representations also inherit their global structure from their initialization. Although our gLocal transform provides a way to perform post-hoc alignment of representations from image/text models using human similarity judgments, there may be alternative initialization strategies or objectives that can provide the same benefits during training, using only image/text pairs.
Acknowledgements
LM, LL, JD. and RV acknowledge funding from the German Federal Ministry of Education and Research (BMBF) for the grants BIFOLD22B and BIFOLD23B. LM acknowledges support through the Google Research Collabs Programme. We thank Pieter-Jan Kindermans for helpful comments an earlier version of the manuscript.
References
[1] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprint arXiv:1907.028900 2019.
[2] Maria Attarian, Brett D) Roads, and Michael C Mozer. Transforming neural network visual representations to predict human judgments of similarity. In NeurIPS 2020 Workshop SVRHM, 2020.
[3] Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European conference on computer vision (ECCV), pp. 456-473, 2018.
[4] Liron Bergman, Niv Cohen, and Yedid Hoshen. Deep nearest neighbor anomaly detection. CoRR, abs/2002. 10445, 2020.
[5] Alsallakh Bilal, Amin Jourabloo, Mao Ye, Xiaoming Liu, and Liu Ren. Do convolutional neural networks leam class hierarchy? IEEE transactions visualization and computer graphics, 24 (16,152-162, 2017.
[6] Jeffrey Bowers, Gaurav Malhotra, Marin Dujmović, Milton Llera Montero, Christian Tsvetkov, Valerio Biscione, Guillermo Puebla, Federico Adolfi, John E Hummel, Rachel F Heaton, et al.
10

---

Deep problems with neural network models of human vision. Behavioral and Brain Sciences, pp. 1-74, 2022.
[7] Stefania Bracci and Hans Op de Beeck. Dissociations and associations between shape and category representations in the two visual pathways. Journal of Neuroscience, 36(2):432 444, 2016.
[8] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.
October
2021.
[9] Ting Chen, Simon Korbllith, Mohammad Norouzi, and Geoffrey Hinton. A simple frame- work for contrastive learning of visual representations. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 1597-1607. PMLR, 13-18 Jul 2020.
[10] Radoslaw M. Cichy, Nikolaus Kriegeskorte, Kamila M. Jozwik, Jasper J.F. van den Bosch, and Ian Charest. The spatiotemporal neural dynamics underlying perceived similarity for weal-world objects. Neurolmage, 194:12-24 24, 2019. ISSN 1053-8119, doi: https://doi.org/10. 101 wwwrj.eerrrimaaee22199555
[11] Radoslaw Martin Cichy, Dimitrios Pantazis, and Aude Oliva. Resolving human object recogni- tion in space and time. Nature neuroscience, 17(3)4555462, 2014.
[12] M. Cimpoi, S. Maji, I. Kokkinos, S. Moohamed,, and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2014.
[13] Matan Jacob Cohen and Shai Avidan. Transformaly-two (feature spaces) are better than one. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 4060 2022.
[14] Corinna Cortes, Mchryar Mohri, and Afshin Rostamizadch. Algorithms for learning kernels based on centered alignment. The Journal of Machine Learning Research, 13(1):795-88) 828, 2012.
[15] Sebastian Damrich, Niklas Böhm, Fred A Hamprecht, and Dmitry Kobak. From I-sne umap with contrastive learning. In The Eleventh International Conference on Learning Representa- tions, 2022.
[16] Lucas Deecke, Lukas Ruff, Robert A. Vandermeulen, and Hakan Bilen. Transfer-based semantic anomaly detection. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th Inter- national Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 2556-2555 PMLR, 18-24 Jul 2021.
[17] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large- scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Patlern Recognition, pp. 248-255, 2009. doi: 10. N0.1109/CVPR PW..PORR.00006688
[18] Jia Deng, Alexander C Berg, Kai Li, and Fei-Fei. What does classifying more than 10,000 image categories tell us? In Computer Vision-ECCV 2010: 11th European Conference Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part 11, pp. 71-84. Springer, 2010.
[19] Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21 21-26 June 2014, volume 32 of JMLR Workshop and Conference Proceedings, pp. 647-655, JMg..org. 2014.
[20] Kazuyoshi Fukuzawa, Motonobu Itoh, Sumiko Sasanuma, Tsutomu Suzuki, Yoko Fukusako, and Tohru Masui. Internal representations and the conceptual operation of color in pure alexia with color naming defects. Brain and Language, 34(1))98-122 1988. ISSN 0093-934X. doi: https://doi.org/10.. /ttp:////..////00/0030/118911010

---

[21] Robert Geirhos, Carlos R. M. Temme, Jonas Rauber, Heiko H. Schütt, Matthias Bethge, and Felix A. Wichmann. Generalisation in humans and deep neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gareet (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[22] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves aceuracy and robustness. In 7th International Conference on Learning Representations,
ICLR 2019. OpenReview.net, 2019.
[23] Robert Geirhos, Born-Herrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 1):665-6775 -733, November 2020. doi: 02.1038/442256-00--0022777
[24] Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Matthias Bethge, Felix A. Wich- mann, and Wieland Brendel. On the surprising similarities between supervised and self- supervised models. In NeurIPS 2020 Workshop SVRHM, 2020.
[25] Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, Curran Associates, Inc., 2021.
pp.
[26] Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 23885-23899, Curran Associates, Inc., 2021.
[27] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33:21271-21284 2020.
(28) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
[29] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[30] Martin N. Hebart, Adam H. Dickter, Alexis Kidder, Wan Y. Kwok, Anna Corriveau, Caitlin Van Wicklin, and Chris I. Baker. THINGS: database 1,854 object concepts and more than 26,000 naturalistic object images. PLOS ONE, 14(10):1-22) 2019. doi: 10.137 137773jjumall.oo 0223792.
[31] Martin N. Hebart, Charles Y. Zheng, Francisco Percira, and Chris I. Baker. Revealing the multi- dimensional mental representations of natural objects underlying human similarity judgements. Nature Human Behaviour, 4(11):1173-1188 October 2020. doi: 10.1038//44152200000009911
[32] Martin N Hebart, Oliver Contier, Lina Teichmann, Adam H Rockter, Charles Y Zheng, Alexis Kidder, Anna Corriveau, Maryam Vaziri-Pashle and Chris I Baker. Things-data, multimodal collection large-scale datasets for investigating object representations in human brain and behavior. eLife, 12:882580, feb 2023. ISSN 2050-084X. doi: 10.7554/eLife.8255 82580.
[33] Katherine Hermann and Andrew Lampinen. What shapes feature representations? exploring datasets, architecture,, and training. Advances Neural Information Processing Systems, 33: 9995-10006. 2020.
12

---

[34] Katherine Hermann, Ting Chen, and Simon Korblith. The origins and prevalence of texture bias in convolutional neural networks. In H. Larochelle, M. Ranzato, Hadsell, M.F. Balcan, and H. Lin (eds.), ) Advances in Neural Information Processing Systems, volume 33, pp. 19000-19015. Curran Associates, Inc., 2020.
[35] Geoffrey E. Hinton and Sam T. Roweis. Stochastic neighbor embedding. In Suzanna Becker, Sebastian Thrun, and Klaus Obermayer (eds.), Advances in Neural Information Processing Sys- tems 15 [Neural Information Processing Systems, NIPS 2002, December 9-14, 2002, Vancouver, British Columbia, Canada), pp. 833-840. MIT Press, 2002.
[36] Shell Xu Hu, Da Li, Jan Stühmer, Minyoung Kim, and Timothy M. Hospedales. Pushing the limits of simple pipelines for few-shot learning: External data and fine-tuning make a difference. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9058-9067, 2022. doi: 10.1109/CVP552688 2w0.:80..222000000 2022.00886.
[37] Mi-Young Huh, Pulkit Agrawal, and Alexei A. Efros. What makes ImageNet good for transfer learning? CoRR, abs/ abs/1608.088 608.08614. 2016.
[38] Marius Cätälin lordan, Michelle R Greene, Diane M Beck, and Li Fei-Fei. Basic level cat- egory structure emerges gradually across human ventral visual cortex. Journal of cognitive neuroscience, 27(7):1427-1446 2015.
[39] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 4904-4916. PMLR, 18-24 Jul 2021.
[40] Seyed-Mahdi Khaligh- Razavi and Nikolaus Kriegeskorte. Deep supervised, but not unsuper- vised, models may explain it cortical representation. PLoS computational biology, 10(11): e1003915, 2014.
[41] Marcie L. King, Iris L.A. Groen, Adam Steel, Dwight J. Kravitz, and Chris Baker. Similarity judgments and cortical visual responses reflect different properties of object and scene categories in naturalistic images. Neurolmage, 197:368-380 382, 2019. ISSN 1053-8119 8119. doi: https://doi.org/
[42] Dmitry Kobak and George C Linderman. Initialization is critical for preserving global data structure in both t-sne and umap. Nature biotechnology, 39(2):156-157, 2021.
[43] Simon Korbliihh, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton, Similarity of neural network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICMI. 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 3519 -529. PMLR, 2019.
[44] Simon Kornblith, Jonathon Shlens, and Quoc V. Le. Do Better ImageNet Models Transfer Better? In IEEE Conference Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 2661-2671. Computer Vision Foundation / IEEE, 2019. doi: 10.1 10111099/...209.0000 9.00277.
[45] Simon Kornblith, Ting Chen, Honglak Lee, and Mohammad Norouzi. Why Do Better Loss Functions Lead Less Transferable Features? In Mare Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14 14, 2021, virtual, volume 34, pp. 2021.
[46] Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini. Representational similarity analysis connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 2008 ISSN 1662-5137. doi: 10.3389/neuro.06 wtw.....0000000200....
13

---

[47] Alex Krizhevsky. One weird trick for parallelizing convolutional neural networks. arXiv e-prints, art. arXiv:1404.5997, April 2014.
[48] Alex Krizhevsky and Geoffrey Hinton. Technical Report 0, University of Toronto, Toronto, Ontario, 2009.
Learning multiple layers of features from tiny images.
[49] Manoj Kumar, Neil Houlsby, Nal Kalchbrenner, and Ekin D Cubuk. Do better ImageNet classifiers assess perceptual similarity better? Transactions on Machine Learning Research, 2022.
[50] Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Wojeiech Samek, and Klaus-Robert Müller. Unmasking clever hans predictors and assessing what machines really learn. Nature communications, 10(1): 1006, 2019.
[51] Aoxue Li, Tiange Luo, Zhiwu Lu, Tao Xiang, and Liwei Wang. Large-scale few-shot learning: Knowledge transfer with class hierarchy. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pp. 7212-7220, 2019.
[52] Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y Zou. Mind the gap: Understanding the modality gap multi-modal contrastive representation learning. Advances in Neural Information Processing Systems, 1512712527, 2022.
[53] Philipp Liznerski, Lukas Ruff, Robert A. Vandermeulen, Billy Joe Franks, Klaus-Robert Müller, and Marius Kloft. Exposing outlier exposure: What can be learned from few, one and zero outlier images. CoRR, abs/2205. 11474, 2022. doi: 10.485500arXxv.2205..11
[54] Raja Marjich, Pol van Rijn, Ilia Sucholutsky, Theodore R. Sumers, Harin Lee. Thomas L. Griffiths, and Nori Jacoby. Words are all you need? Capturing human sensory similarity with textual descriptors, 2022.
[55] Ellen M Markman. Categorization and naming in children: Problems of induction. mit Press, 1989.
[56] JL McClelland, Z Sadeghi, and AM Saxe. A critique of pure hierarchy: Uncovering cross- cutting structure in a natural dataset. In Neurocomputational Models of Cognitive Development and Processing: Proceedings of the 14th Neural Computation and Psychology Workshop, pp. 51-68. World Scientific, 2017.
[57] Thomas McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. arXiv preprint arXiiv:992.00I 2019.
[58] Jovana Mitrovic, Brian McWilliams, Jacob C Walker, Lars Holger Buesing, and Charles Blundell. Representation learning via invariant causal mechanisms. In International Conference on Learning Representations, 2021.
[59] Lukas Muttenthaler and Martin N. Hebart. Thingsvision: A python toolbox for streamlining the extraction of activations from deep neural networks. Frontiers in Neuroinformatics, 15:45, 2021. ISSN 1662-5196. doi: w0..389/nnnff22221.... 6779888.
[60] Lukas Muttenthaler, Charles Y Zheng, Patrick McClure, Robert A Vandermeulen, Martin N Hebart, and Francisco Pereira. VICE: Variational Interpretable Concept Embeddings. Advances in Neural Information Processing Systems, 35:33661-33675, 2022.
[61] Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A Vandermeulen, and Simon Korn- blith. Human alignment of neural network representations. In 11th International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, Mai 01-05, 2023. OpenReview.net, 2023.
[62] A. Paccanaro and G.E. Hinton. Learning distributed representations of concepts using linear relational embedding. JEFE Transactions on Knowledge and Data Engineering, 13(2):232-244 2001. doi: 10. 109/699997....
14

---

[63] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zachary De Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelz- imer, Florence d' Alché Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Anmual Conference Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 8024-8035, 2019.
[64] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit- learn: Machine learning in python. Journal of machine learning research, 12(Oct):2825-283) 2011.
[65] Joshua C. Peterson, Joshua T. Abbott, and Thomas L. Griffiths. Adapting deep network features to capture psychological representations. In Anna Papafragou, Daniel Grodner, Daniel Mirman, and John C. Trueswell (eds.), Proceedings of the 38th Annual Meeting of the Cognitive Science Society, Recogbizing and Representing Events. CogSci 2016, Philadelphia. PA, USA, August 10-13, 2016. cognitvesciencessoiity.ogg 2016.
[66] Joshua C. Peterson, Joshua T. Abbott, and Thomas L. Griffiths. Evaluating (and improving) the correspondence between Deep Neural Networks and Human Representations. Cogn. Sci., 42(8): 2648-2669, 2018. doi: 10. 1 //coo....6600
[67] Joshua C. Peterson, Ruairidh M. Battleday, Thomas L. Griffiths, and Olga Russakovsky. Human uncertainty makes classification more robust. In 2019 IEEE/CVF International Conference on Computer Vision. ICCV 2019, Seoul, Korea (South), October 27 -Nvember 2, 2019, pp. 9616-9625. JEEE, 2019. doi: D0.100//CCV.2219.000999
[68] Hieu Pham, Zihang Dai, Golnaz Ghiasi, Kenji Kawaguchi, Hanxiao Liu, Adams Wei Yu, Jiahui Yu, Yi-Ting Chen, Minh-Thang Luong, Yonghui Wu, Mingxing Tan, and Quoe V. Le. Combined scaling for open-vocabulary image classification. arXiv e-prints, art. arX1v:111..10000, 2022.
[69] Daria Proklova, Daniel Kaiser, and Marius V Peelen. Disentangling representations of object shape and object category in human visual cortex: The of cognitive neuroscience, 2865)668 2016.
distinction. Journal
[70] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 8748-8763. PMLR, 18-24 Jul 2021.
[71] Tal Reiss, Niv Cohen, Liron Bergman, and Yedid Hoshen. Panda: Adapting pretrained features for anomaly detection and segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 2806-2814, June 2021.
[72] Rocco Robilotto and Qasim Zaidi. Limits of lightness identification for real objects under natural viewing conditions. Journal of Vision, 4(9):9-9, 09 2004. ISSN 1564-7362. doi:
10.1167/4.999
[73] Timothy T Rogers, James L McClelland, al. Semantic cognition: A parallel distributed processing approach. MIT press, 2004.
[74] Eleanor Rosch, Carolyn B Mervis, Wayne D Gray, David M Johnson, and Penny Boyes-Braemt Basic objects in natural categories. Cognitive psychology, 8(3):382-43, 1976.
[75] Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International conference machine learning, pp. 4393-4402. PMLR, 2018.
15

---

[76] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Fei-Fei Li. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis., 115(3)211122) 11-222, 2015. doi: 10. 1007/000/ 1263-015-0886--
[77] Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. Breeds: Benchmarks for subpopu- lation shift. arXiv preprint arXiv: 04859, 2020.
[78] Andrew M Saxe, James L McClellans, and Surya Ganguli. Learning hierarchical categories in deep neural networks. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 35, 2013.
[79] Andrew M Saxe, James L McClelland, and Surya Ganguli. A mathematical theory of semantic development in deep neural networks. Proceedings of the National Academy of Sciences, 116 (23):11537-11546, 2019.
[80] Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib J. Majaj, Rishi Rajalingham, Elias B. Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy, Franziska Geiger, Kailyn Schmidt, Daniel L. K. Yamins, and James J. DiCarlo. Brain-score: Which artificial neural network for object recognition is most brain-like" bioRxiv, 2020. doi: 10.1101/407007.
[81] Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarezyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. LAION-400M: open dataset of clip-filtered 400 million image-text pairs. CoRR, abs/2111.022 14, 2021.
[82] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. LAION-5B: open large-scale dataset for training next generation image-text models. In NewrlPS, 2022.
[83] Eli Schwartz, Leonid Karlinsky, Rogerio Feris, Raja Giryes, and Alex Bronstein. Baby steps towards few-shot learning with multiple semantics. Pattern Recognition Letters, 160:142-147, 2022.
[84] Patrick Shafto, Charles Kemp, Vikash Mansinghka, Matthew Gordon, and Joshua B Tenenbaum. Learning cross-cutting systems of categories. In Proceedings of the 28th annual conference of the Cognitive Science Society, volume 2146, pp. 2151, 2006.
[85] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for acrge-scale image recognition. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
[86] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In
I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
[87] Ilia Sucholutsky and Thomas L Griffiths. Aligmment with human representations supports robust few-shot learning, arXiv preprint arX1v:2301.11990, 2023.
[88] Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B Tenenbaum, and Phillip Isola. Rethinking few-shot image classification: good embedding is all you need? In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV Springer, 2020.
16, pp.
[89] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 911),, 2008.
[90] Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3485-3492. IEEE, 2010.
16

---

[91] Kai Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry, Noise or signal: The role of image backgrounds object recognition. arXiv preprint arXiv:2006.0990 09994, 2020.
[92] Daniel K. Yamins, Ha Hong, Charles F. Cadieu, Ethan A. Solomon, Darren Seibert, and James J. DiCarlo. Performance-optimizn hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 1 (11(2))8619-8666 2014.
doi: 10.1073/pnas.1400 14031 12111.
[93] Renrui Zhang, Wei Zhang, Rongyao Fang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free adaption of clip for few-shot classification. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision - ECCV 2022, pp. 493-510, Cham, 2022. Springer Nature Switzerland. ISBN 978-3-031-19833-5.
17

---

A Experimental details
A.1 Model features
We extract penultimate layer features of four different ImageNet-models AlexNet 477, VGG-16
85]. ResNet-18 18, and ResNet-50 [28] and image encoder features of four different image/text
models CLIP RN50 and CLIP ViT-L/14 trained on WIT [70]: CLIP ViT-L/14 trained on Laion-
400M 81 and Laion-2B [22 respectively. For extracting the model features, we use the Python
library thingsvision [59]
A.2 gLocal probing
To optimize the gLocal transforms, we use standard SGD with momentum and perform cross- validation according the procedure proposed in Muttenthaler et [61]. For finding the optimal gLocal transform, we perform an extensive grid search over four different hyperparameter values the learning rate, 17, the strength of the regularization term X, the global-local trade-off parameter a, and the temperature parameter, т, used in the softmax expression for the local contrastive loss term (see Eq. [5. Specifically, we perform an extensive grid search over the Cartesian product the following sets of hyperparameters:
7/ E (0.0001, 0.001, 0.01, 0.1),
À E {0.01, 0.1, 1.0, 10.0}.
a E {0.05, 0.1, 0.25, 0.5, 1.0).
THE 00,,,,.2500.5. 1.0).
We use the same 7 and grids for global probing. We use PyTorch [63] for implementing the probes and PyTorch lightning to accelerate training. We choose the gLocal transform that achieves the lowest alignment loss (see alignment term in Eq. 6 Among the values in the above grid, we find that combination of (a 0.1, 0.1, 77 0.001) yields the lowest alignment loss/highest probing odd-one-out accuracy for both CLIP RN50 (WIT) and CLIP ViT-L/14 (WIT) (see Fig. ( A combination of (a 0.25, 0.1, 17 0.001) gives the second lowest alignment loss/highest probing odd-one-out accuracy for CLIP RN50 (WTT) and CLIP ViT-L/14 (WIT).
one-out accuracy
Odd-one out accuracy
0.60
0.05
0.59
0.0
0.59
0.38
0.58
-057
-0.66
0.53
0.55
0.01
0.1
1.0
10.0
0.01
0.1
1.0
10.0
A
CLIP RNSO (WTT)
(b) CLIP VIT-L/14 (WTT)
Figure 6: Among all hyperparameter combinations considered in our grid search, a combination of (a 0.1, 0,1, 0.001) for Fq. in $ yields the hest odd-one-out accuracy on held-out test set for both CLIP RNS0 and CL.P VYT-L/14 (WIT).
For each (a, A) combination we select that combination with the best probing odd-one-out accuracy on a held-out test among the set possible learning rate, 7), and temperature value, 7, combinations determined by the above grid. We observe that = 0.001 generally gives the best results across the different (a, A) combinations, whereas performance is fairly insensitive to the value of т. Since
neither (a 0.1, À 0.1) nor (a 0.25, 0.1) are values at the edges of the hyperparameter
18

---

grid, is plausible to assume that both the contrastive local loss and the regularization term in Eq. 6 in aare are necessary to obtain a transformation that leads a sett-of-both-worle representation.
Although our goal has been to find a transform that induces both increased representational alignment and improved downstream task performance, task performance can potentially be improved by excluding the alignment loss. Note that a 1.0 causes the optimization process to ignore the alignment loss. Unsurprisingly we did not find that be the case, We remark that minimizing both the local contrastive loss and the regularization preserves the local similarity structure of the original representation space but does not inject any additional information into the representations. Moreover, it is non-trivial to choose transform that works well across all downstream tasks without including the alignment loss. Therefore, we exclude a 1.0 in Fig. O
considered
1.0 to examine whether downstream
Compute. We used a compute time of approximately 400 hours a single Nvidia A100 GPU with 40GB VRAM for all linear probing experiments - including the hyperparameter sweep. The computations were performed on standard, large-scale academic SLURM cluster.
A.3 Few-shot learning
Here, we use folf cross-validation for finding the optimal fz-regularization parameter, where ng refers to the number of shots per class. We select the parameter from the following set of values, {le+6, 6, 1, 1e+5, 1e-44, 1e++3, 1, 1e+2, 1e+1, 1, 1e- 1e-3, 1e-4]. We use the scikit-learn [64] implementation of (multinomial) logistic regression and refit the regression after selecting the optimal regularization parameter.
Compute. We used compute time of approximately 5600 CPU-hours of 2.90GHz Intel Xeon Gold 6326 CPUs for all few-shot experiments. Computations were performed on a standard, large-scale academic SLURM cluster.
A.4 Anomaly Detection
In this section, we outline our anomaly detection experimental setting in more detail. In the anomaly detection settings that we consider in our analyses normal/anomally classes are determined via the original classes in the data. Here, each of the original classes is once selected as a normal class with the remaining classes being anomalous and, vice versa, each class in the data is once selected as an anomalous class with the other classes being normal. After embedding the training images from either the normal or the anomalous class in a model's representation space, at inference time a model must classify whether a new image belongs to the normal data or whether it deviates from and is thus considered an anomaly. For each example in the test set, a model yields an anomaly score where higher scores indicate more probability of an example being anomalous. Using the binary anomaly labels and the anomaly scores for each of the examples, we can then compute the theeder-ree--reeie----eeeee oew.aatiinnnaaaaaeeiiiiiiiiiiiie characteristic-cun (AUROC) to quantify the performance of the model.
One-vs-rest. Given dataset (e.g., CIFAR-10) with C' classes, one class (e.g.. "airplane"" is chosen to be the normal class and the remaining of the C classes is once selected as a normal class and the AUROC is averaged across the classes.
classes of the dataset are considered anomalies. Each
Leave-one-out (LOO). In contrast to the "one-vs-rest" setting, in LOO we define one class of the dataset as an anomaly and the remaining classes as normal. Similarly to the "one-vs-rest" setting, this results in C evaluations for dataset with C classes.
In both "one-vs-rest" and LOO AD settings, we evaluate model representations in the following way: First, we compute the representations Xarai of the normal samples in the train set. Then, we compute the representations of all test set examples Xiea- For each test set representation, we compute the cosine similarity to all normal train representations, X and select the k nearest neighbor samples that have the highest cosine similarity.
The anomaly score of test set representation is then defined as the average cosine distance to the k nearest train representations. A is a hyperparameter that determines the number of nearest neighbors over which the anomaly score is computed. We choose h = 5 for our experiments but show that performance is fairly insensitive to the value of l (see Tab. 12).
19

---

Compute. For all AD experiments, we used compute time of approximately 20 hours on a single Nvidia A100 GPU with 40GB VRAM. Computations were performed on standard, large-scale academic SLURM cluster.
B
What changes in the global structure of the representations after alignment?
In this section, we attempt to build some intuitions for how the global structure of the representations changes after aligmment. To do so, we analyze the movements of the representations of items and superordinate categories in the THINGS dataset. Specifically, we compute cosine differences between the
representations of each pair of items in THINGS and then compute how these distances change under the transforms.
We show the pairs of items that change the most in distance in Table 5 Items that are semantically related, like "curry" and "scrambled egg", tend move closer together, and therefore have trans- formed distances that are smaller than their original distance. By contrast, items like "handcuff" and "stethoscope", which are semantically unrelated but perhaps have some slight visual similarity, tend to move farther apart. The distance changes under the gLocal transforms are correlated with, though generally less varied than, those under the naively-aligned transform.
To more broadly analyze the change in global structure, we then look at how distances between pairs of items change within and across superordinate categories (the top-down categories from THINGS). We show the results in Fig. [ Under the naively aligned transform, the items within each superordinate category tend move slightly closer together the diagonal is slightly blue while the items from different categories tend to move substantially farther apart - the off-diagonal is mostly red. That is, the representations broadly moving in way that reflects the overall human semantic organization of the categories.
There few notable standouts: the categories of drink, food, plant, and animal change particularly much, and in particularly interesting ways. These categories each move much farther relative to all other categories (such tool musical instrument) than those other categories move relative to each other. This perhaps reflects the particular semantic salience food, drink, plants, and animals from human perspective. Furthermore, food and drink are of the few pairs of superordinate categories between which distances actually decrease after the transform, presumably reflecting the strong semantic ties between these categories. Similarly, animals move less far from plants than from any other category, perhaps reflecting the fact that the animate/inanimate distinction is one of the strongest features in human semantic representations 73
Under the gLocal transform, the pattern of changes is strongly correlated with the naively aligned transform (r 0.96, p < 10 10-16). However, in keeping with the regularization, the magnitude of the changes varies less.
Table 5: Distances between pairs of individual items from THINGS, ranked by the relative change in cosine distance from before to after naive alignment (normalized by original distance). The top items move much closer together under naive alignment, while the bottom ones move much farther apart. (All results are from CLIP-ViT-LE VIT-L/14.)
Item
Item 2
original dist.
naively aligned dist.
Local dist.
curry
scrambled egg
0.303120
0.005276
0.401019
otter
warthog
0.305242
0,005530
0,382150
parfait otter
spaghetti
0.457553
0.009115
0.540346
rhinoceros
0.327497
0.006530
0.456641
stethoscope grass cat
wheat wallet
0.263908
1,284535
0,935891
0.277866
1.347424
1.056572
traffic light
0.285151
1.378671
0.981944
handcuff
sugar cube
0.272936
1.308337
0.904380
20

---

transform nalvely aligned
transform glocal
badly part
animal
0.8
0.8
clething
clothing
cantainer drink
0.6
0.6
clectronic
Top down category
food
furniture
hame decor
agpliance
kitchen tool
medi cal equipment
musical instrument
none above
of car
plant
sports equ pment
vahicle
weapon
animaa
part
distning
dottingg acccesson
contained
drink electronic
Toed
fun home de-col
kitchen agpliat
kitchen tool
medical equlpmennt
musica a instrumen
nore office supph
part of car
plant
sports equipme
tool
toy
wehidde wesppnn
-0.2
snimma
body dothing
accessop
containge
drink
device
Tood
home uumuure
Wicher apillance
medic Witchen wol
music souttnn pore of the sbeve affice suppl part of car
ment oo
wehit
-0.2
electronic
dothina
sports
Top-down 2ategery
Top-down category 2
Figure 7: How does the global structure of the representations change after alignment? Here, analyze the movements of representations of pairs items from different superordinate categories from the THINGS dataset. The squares on the diagonal indicate the change in distance between items within a superordinate category, while the squares off the diagonal indicate changes between pairs of items from the corresponding pair of superordinate categories. A red color indicates the items from the categories each other after alignment, blue indicates moving closer together. Generally, items within superordinate category move slightly closer together under naive alignment, while those different categories move farther apart. A similar overall pattern is reflected in both the naively-aligned transform (left) and gLocal (right) ones, though under gLocal alignment there is greater overall spreading of the representations. (All results are from VIT--1.T-))
farther apart from
C Visualization of neighboring images
To provide further insight into the difference between the effects of the naive and gLocal transforms, in Fig. [8] we visualize the neighbors of nine anchor images. In order to show a diverse set of images, we pick the nearest neighbors in the CLIP VIT-L/14 (WTT) embedding space subject to the constraint that each neighbor comes from different class from the original images and the nearer neighbors. In accordance with the results $4.2 we find that the neighbors in the untransformed and gocal spaces are generally similar, whereas neighbors in the naive representation space are frequently different. The naive transform appears to discard all non-semantic properties images, whereas the untransformed and gLocal representation spaces are sensitive to pose, color, and numerosity. In cases where the closest neighbor differs between the naive and gLocal representations (third and fourth row), the neighbors in the gLocal representation are arguably better matches to the anchor.
21

---

Untransformed
gLocal
Naive
Untransformed
gLocal
Naive
Untransformed
gLocal
Naive
Untransformed
gLocal
Naive
Untransformed
gLocal
Naive
Untransformed
gLocal
Naive
Untransformed
gLocal
Naive
Untransformed
gLocal
Naive
Figure 8: Comparison of neighbors in the ImageNet validation for representations with different transforms. We visualize the 10 closest images subject to the constraint that comes from unique class. The anchor images are shown in the leftmost column. The three rows corresponding to each anchor image show their nearest neighbors in the untransformed, gLocal transformed, and naively transformed representations.
22

---

D Additional results on downstream tasks
In this section, we provide additional sho-shot learning and anomaly detection results for all ImageNet and image/text models that we considered in analyses (see $(A.)). We start this section by
demonstrating strong relationship between the performances of the different downstream tasks.
1.0
Anomaly Detection (AUROC)
0.95
0.9
x
Training
0.85
CUP
ImageNet
Transfor
0.8
original
nalve
glocal
0.75
0.2
0.2
0.3
0.4
0.5
Few-Shot Classification (Accuracy)
Figure 9: Here, we show anomaly detection AUROC averaged across all tasks reported in Tab. 2 as function of the average 5-shot classification performance for all ImageNet and CLIP models (see $A. using either the original representations the representations transformed via the naive or gLocal transformations.
Downstream task relationship. We observe strong positive relationship (r 0.8524, < 10-6) between the average few-shot learning and the average anomaly detection performance for all ImageNet and image/text models that we considered in our analyses (see Fig. 9 This observation holds for both the original representation space and the representations transformed via the naive or ocal transformations. This indicates that both downstream tasks require similar representations for similarly strong performance.
D.1 Few-shot learning
In the following section, we show additional few-shot learning results. Specifically, we report 5-shot performance of ImageNet models and show few-shot results as function of the number of samples used during fitting.
Results for ImageNet Net models. In Tab, 6 we report additional 5-shot results for ImageNet models. The gLocal transforms improve few-shot accuracy on Entity- 13,30) from BREEDS but the impact on few-shot performance is either inconsistent or negative for CIFAR-100 coarse, CIFAR-100. SUN397, and DTD of which the latter three are more mne-ggained datasets than the other three.
Table 6: 5-shot FS results using the original transformed representations.
Emtity-33 original
Emiit-30
CIFAR100-Cause glocal
CIFAR100
SUN397
Model \ Transform
glocal
original
glocal
original
original
glocal
original
g/ocal
original
glocal
AlexNer
37.49
42.13
27.73
26.93
32.33
31,44
28.70
23.29
26.43
19.12
37.05
30.54
R8sNet-18
35.92
37.26
ResNei-50
49.27
59.181
50.16
54.11
50,991
47.99
44.75
47.17
44.26
54.02
51.49
VGG-16
51.78
56.49
44.69
45.97
39.08
37.03
34.54
28.08
37.12
29.89
45.39
38.01
investigate whether the gLocal transforms improve downstream task performance of image/text models in few-shot learning settings in combination with techniques that were specifically designed for this task, we evaluate CLIP models combined with Tip- Tip-Adapter 993 Tip-Adapter is designed to produce effective few-show classifiers from pretrained image/text models by linearly combining the outputs of two modules:
Results for Tip-Adapter [93].
a zero-shot classifier, where the similarity of the input embedding space with the embedded textual description of cach class determines the output
23

---

(ii) key-value cache model that computes its output by summing the one-hot labels of all stored sho- data, weighted by the similarities of the associated image embeddings the embedded imput sample
For the evaluation of glocal transforms, we transform both the text embedding and any image embedding involved in the inference part. The myper-parameterd that which have been identified as optimal for ImageNet classification by Zhang et al. 193.
are 1 and [ 5.5
We observe that even in conjunction with method specifically designed for few-shot learning, gLocal transforms improve few-shot accuracy in most scenarios (see Tab. 7. in particular on Entity- (0ttty-(13,30) (13,30) from BREEDS. Furthermore, the performance increase appears to be stronger for the CLIP models trained on the OpenAI WIT datasets compared LAION (see Fig. TÖp. For the smaller of the two LAION datasets [LAION-400M: 8I), the effect of the gLocal transform on CLIP with Tip-Adapter is negligible compared to effect on models trained on the larger LAION-2B 82 or OpenAl's WIT dataset (70)
Table 7: S-shot FS results using the original or transformed representations in combination with Tip- Tip-Adapter.
Entity133
Emilty-3 original
CIFAR100
CIFAR100
SUN397
L711D
Model Transform
original
gLocal
gLocal
original
gLocal
odiginal
gLocal
original
gLocal
original
glocal
CLIP (WIT)
66.29
69.95
59.24
61.76
31.41
33.69
38.53
44.82
53,42
55.75
47.98
47,91
CLIP-VIT-/IR (WIT)
68.17
77.15
66.73
45.79
64.51
64.77
69.80
57.96
CLIP-VIT-L/IA (LAION- 100M)
72,87
72.04
69.60
67.21
75.07
71.38
69.65
68.27
71.24
70.30
68.96
65.89
CLIP VILL/14 (B)1ON-2))
73.19
75.54
70.30
71.79
77.45
75.81
71.87
73.35
70.24
71.17
66.10
64.43
Entity-13
Entity:30
CIFARIDO
0.8
0.7
5-Shot Accuracy: gLocal + Tip-Adapter
0.5
0.3
0.85
0.7
0.15
0.6
0.7
0.5
0.3
0.4
0.5
CIFARL00-Coare
SUN397
DTD
0.0
0.0
CUP-IITUA SAION 281
cur-MNS INTT
CUP.VITUAA NMT.
0.7
CUPMITTUU 2,AOON
0.7
0.6
0.6
0.5
9.4
0.5
0.6
0.7
9.6
0.7
0.8
05
0.6
0.8
5-Shot Accuracy: original Tip-Adapter
Figure 10: 5-shot FS results using the original transformed representations in combination with Tip-Adapter.
Effect of transforms for different numbers of training samples. When varying the number of training samples for the seo-shot experiments described in $4.3 we observe consistent improvements of the gooa transforms across shots. Excluding the high-variance setting of 2-shot learning, we either find stable improvements in accuracy for image/text models, or a downward trend for ImageNet models on some tasks. This corroborates our findings from $4.3) Results appear to be robust to changes the training set size, in particular for the CLIP models. Yet, we observe the most substantial benefits in low data regimes. See Fig. I1 for more details.
24

---

Entity-13
En
CIFAR100-CoaD
Chunge Thanstom
12.
Akalec
20.8
PMT P44 (ANIM21)
0.8
1.88
10
12
10
12
10
12
CIFARIOO
SUN397
DTD
Abstet
VOS
14
CL CLEVVODO SATT CL PLEVITUCE (MT)
Accurayy Chamge
PLPIIT-04 (144244)
Figure 11: Change in average accuracy for different numbers of training samples per super-class (top row) or (sub-)elass (bottom row) used for few-shot learning. Error bands depict 95% Confidence Intervals (CIs), computed over 5 different runs.
Few-shot learning ImageNet-IK, In addition the few-shot learning results that presented in $4.3 here we report few-shot classification results on ImageNet-1K 1K [77. For both the original and tran---rrnnforrmee CLIP representations, we trained linear classifier with different numbers of shots, k. i.c. the number of training examples class. We find that the gLocal traeall-trasffomee repre- sentation achieves better ImageNet validation accuracy compared to the original representation for both CLIP ViT-L/14 (WIT) and CLIP ViT-L/14 (LAION-2B) in small training data settings where 5. We don't observe a signifieant difference between the two representation spaces for CLIP RN50 (WIT) and CLIP ViT-L/14 (LAION-400 M) which generally perform worse than the other two models across all training settings (see Fig.
CLP--VT-L/14 (WIT)
CLIP-VI-LL14 (LAION-400M)
85
ImageNet accuracy
75
65-
65-
55-
45
35
35
10
15
20
10
15
20
CLIP-VIT-L/14 (LAION-2B)
CLIP- CLIP-RNSO (WIT)
ImageNet accuracy
85
60
75
50-
40
55
original goocal
45
15
Number of shots
Number of shots
Figure 12: ImageNett validation set accuracy function of the number of training examples per class used for training a linear classifier on the original and gLocal-transformed representatinss For all CLIP models, we report the mean validating set accuracy across three different soods.
D.2 Anomaly detection
In addition to the results of image/text models for the "one-vs-rest" anomaly detection (AD) setting that we presented in $4.4. here we show "one-vs-rest AD performance of ImageNet models. While the gLocal transform considerably improves AD performance over the untransformed representations
25

---

across the different datasets for image/text models (see Tab. 2 2 for ImageNet models we do not observe any improvements over the original representation space (see Tab. [
Furthermore, we present results for the non-standard Leave-one-out (LOO) setting and for "CIFARIO 10- D-CCIFARIOO" for all image/tett and Imagenet models that considered. In the "CIFARTO-VE CIFARI00 AD task, all data of CIFARIO is considered to be the normal class, and each sample from the CIFAR 100 dataset considered an anomaly. Similarly to the previously reported AD results, the gLocal transform substantially improves AD performance compared to the original representations image/text models across all datasets but does not appear to have considerable impact on the performance of ImageNet models (see Tab. [10
Table 8: One-vs-rest nearest neighbor hased AD results; with and without transformation. ImageNet30 results for ImageNet models omitted due overlap with train data.
CIFARI0
CIFAR100
CIEAR Coareee
ImageNet30 original
DTD
Model \ Transform
original
glocal
original
glocal
original
olaal
glocal
original
glacal
AlexNet
89.43
83.63
92.34
88.53
87.53
82.75
86.33
79.51
ResNer-18
92.19
84,96
95.06
92.16
84.89
94.38
89.55
ResNer-50
94.74
89.74
96.46
93.76
94.3
91.2
94.47
91.52
VGG-16
90.33
88.0
93.56
91.97
89.78
88.16
91.15
85.5
Table 9: One-vs-rest AD with a class distribution shift between train and test sets; with and without transforma- tion.
Entity-13
Entity-30
Living-17
Nonliving-22 26
Cifar100-shif
Model | Transform
original
glocal
original
gLocal
original
gLocal
original
gloccal
original
gLocal
AlexNer
83.84
81.45
85.38
83,71
87.04
79.09
81.45
78.84
80.21
76.37
ResNet-18
91.84
88.01
93.18
90.32
96.82
89.11
90.97
88.82
81.83
76.84
ResNet-50
89.59
87.24
93.51
89.63
98.27
89.87
90.61
88.65
84.73
84.45
VGG-16
89.78
88.87
90.7
91.56
94.72
89.98
89.78
89.32
83.42
81,91
Table 10: LOO nearest neighbor based AD results and "CIFAR- 10 vs. CIFAR-108 AD results; with and without using the gLocal transform.
CIFARIO
CIFAR
Cifar Lffar10-Coaaree original
Cifarl0 Cifar 100 original
Model \ Transform
original
gLocal
original
gLocal
gLocal
gLocal
AlexNet
67.64
62.7
58.94
55.83
63,33
59.37
69.87
68.01
ResNet-18
72.35
63.47
64.86
58.53
71.38
62.89
81.42
73.71
ResNet-50
76.62
68.27
66.91
61,26
74.78
68.63
84,27
80.17
VGG-16
68.45
64.31
59.92
57.89
65.81
64.28
73.55
74.7
CLIP-RN50
70.32
72.46
59.91
61.43
65.63
68.07
72.55
76.78
CLIP-VIT-L/14 (WIT)
84.91
91.3
67.08
72.24
73.48
80.7
85,24
92,78
CLIP 1/T (LAION 400M)
93.0
92.37
74.05
74.15
82.13
82.88
94.44
94.68
CLIP-VIT-LI4 (LAION-2B)
93.55
95.23
76.88
77.46
84.67
85.78
93.18
95.26
Table 11: LOO nearest neighbor based AD results; with and without using the gLocal transform.
Entity-13
Entity-30
Living-17
Non-Living-22
Model
Tranffomm
original
gLocal
original
gLocal
original
gLocal
original
gLocal
AlexNet
62.05
59.73
58.2
56.23
61.02
$6.07
56,27
54.93
ResNet-18
74.26
68 88
70.6
64.7
76.48
70.79
66.61
63,82
ResNet-50
72.46
73.67
73.37
72.49
83.5
82.45
68,16
68.29
VGG-16
70.26
68.03
66.38
63.24
73.31
65.99
64.87
62.95
CLIP-RN5O
69.99
71.12
63.49
63.72
72.52
70.04
63.4
CLIP-VITH L/T-L/14 (WIT)
71.68
74,88
68.96
70.58
82.9
82.72
63,94
67.33
VIT- 400M)
69 98
71.44
65.68
66,26
77.35
77.4
65.19
66.27
CLIP-VIT-/14 (LAION-2B)
70.77
72.49
67.68
80.07
80.09
67.99
The nearest neighbor hyperparameter k. From the results reported Tab. 12 can be inferred that the nearest neighbor hyperparameter k does not have a considerable impact on AD task performance across the different datasets. Here, we report the impact of on the performance of CLIP ViT-L/14 (WTT) but the observation holds across all image/text models.
26

---

Table 12: Nearest Neighbor AD performance of CLIP ViT-L/14 for different A:
Dataset
\ TTrnnform
original
gLocal
original
gLocal
original
gloccal
original
gLocal
CIFAR-IO
95.37
98.16
95.14
98. 16
94.86
98.11
94.50
98.03
CIFAR-100
91.90
97.22
91.41
97.19
90.93
97.08
90.39
96.92
CIFAR-100-coar
89.28
88.50
86.81
CIFAR-I00-shh
74.48
87.06
73.69
87.11
73.00
87.01
72.29
86.85
ImageNet30
98.95
99.74
98.91
99,75
98.85
99.77
98.78
99.77
Entity-13
88.37
92.5
88,54
93.23
88.45
93.63
88.28
93.95
Entity 30
91.26
95.11
91.31
93.53
91.22
95.74
91.03
95.91
D.3 Global versus gLocal transform
Aside from the AD performance CLIP RN50 and CLIP ViT-L/14 (WIT), the gLocal transform leads to more substantial improvements downstream tasks than the global transform. In Tab 13 we report the average few-shot and anomaly detection performances using the global or gLocal transforms. For FS, we average performance over all results reported in Tab. and for AD average performance across all results reported in Tab. 2
Table 13: Comparison of the average downstream task performance global and gLocal transforms.
AD
FS
Model Transform
global
gLocal
global
gLocal
AlexNet ResNet-18
81,16
81.76
28.43
28.91
84.62
86.91
43.42
44.61
ResNet-50
93.19
89.56
VGG-16
87.32
88.36
37.23
39.25
CLIP R.IP-RN5O (WIT)
92.12
91.52
54.63
55,00
VIT--V (WTT)
95.49
95.19
69.92
69.91
CLIP-VIT-LI4 (LAION-400M)
95.72
96.08
68.
CLIP-VIT-LI4 (LAION-2B)
96.33
96.65
72.00
72.27
D.4 Human-adversarial experiments
To rule out confounding variables other than global similarity structure for improved downstream task performance of the gLocal transform, we created new triplet dataset where for each triplet in the data we choose an object that is different from the original human choice to be the new odd-one-out choice. Note that this the set of two objects not chosen by human participant, i.c., auuman-adversarian choices (albeit random and deterministic). Using this human-ade auman-adversariall triplet dataset, we use the same gLocal optimization as we do for the original task (see Eq. 6). We minimize bounded cross- cross-entropy error and determine convergence on held-out, auman-adversarial validation set as we have done for the original task. This allows us to use the same hyperparameter setting that we found is optimal for the Local transform (see Appx. App for further details). Across the different few-shot learning and anomaly detection tasks, we find the adversarial transform performing slightly worse than the original transform and substantially worse than the gLocal transform for all CLIP models (see Fig. 13
a random choice over all objects but random choice over
E Representational alignment
E.1 Human similarity judgments and RSMs
Multi-arrangement task. Human similarity judgments for King et al. 41 and 10 were obtained by using a multi mulli-arrangement task. In a aratt-arrangemmnt task, participants are presented with a computer screen showing images of several different objects. The participants images into semantically meaningful clusters, given the instruction that images of objects that lie close together considered more similar. From this arrangement, one can infer pairwise ddis-)similarities of the objects and average those across all participants to obtain a representative (dis- siimilrttyl matrix.
asked to arrange the
pairwise similarity judgments were obtained by asking human participants to rate the similarity of pairs of objects on an ordinal scale that ranges from 0 (onot similar all")) to 10 ("very similar"). The pairwise similarity ratings can averaged across the different participants which in turn yields matrix of similarities between pairs of objects.
Ordinal scale. Peterson et al. 65 666,
27

---

0.72
0.95
0.67
Few-shot learning
CLIP-RNSD (WIT)
Anomaly detection
0.92
0.62
CLIP-ITT-/44 (WIT)
CLIP-VITL/14 (LAION-400M)
0.89
0.57
CUP-VIT-L24 (LAION-28)
0.86
0.52
Transform
original
0.47
naive
0.83
gLocal
0.42
adversarial
0.8
0.35
0.4
0.45
0.5 0.55 0.6
0.35
0.4
0.45 0.5
0.55 0.6
Human alignment =
Human alignment =
Figure Fow-shot learning and anomaly detection performances a function of the degree alignment with huraan similarity judgments the original, naive, gLocal, and human-adversariia transforms.
Triplet odd-one-out choices. The triplet odd-one-out task is a commonly used task in the cognitive
sciences infer pairwise object similarity ratings [72 31 60). The triplet odd-one-out task is purre--atermatiiefooreedddd
task where participants are presented with three objects and have to select the one that does not fit. In contrast to the multi-an arttiarrrrneementt task or an ordinal scale, the triplet odd-onc-out task does not naturally yield a similarity matrix. A similarity matrix can be obtained, however, by learning representations for the objects being used in the task from the human responses. Variational Interpretable Concept Embeddings (VICE) - an approximate Bayesian method for inferring mental representations of object concepts from triplet odd- out-one-out choices method that was specifically developed for that purpose. VICE uses variational inference to learn representations for the objects in the triplets by fitting the human responses via stochastic gradient descent. The method minimizes Lgiohal with additional non-negativity and sparsity constraints on the representations. More details about the optimization can found Muttenthaler et al. [60]. From the VICE solution, one can easily compute a representational similarity matrix (RSM). Specifically, given learned object representations V E R xx one first computes the dot-product similarity matrix Sp := VV and then exponentiate this matrix elementwise, S% exp( (S.) One then apply the softmax function defined in Eq. to every combination of triplets in the exponentiated similarity matrix which yields the final RSM for triplet odd-one-out choices from Hebart et al. [32) The last step is performed to guarantee that the pairwise similarities are modeled according to the triplet odd-one-out objective function that was used to learn the human object representations V (see Eq.
E.2 Neural network representations and RSMs
Neural network representations. RSMs for neural network representations are obtained by first embedding the same set of images that were presented to the human participants in the p-dimensional latent space of a neural net. The latent space could be any layer of a neural network. Here we use the penultimate layer space for ImageNet models and the image encoder space for image/text models. We do this because previous work has shown that the penultimate layer space of ImageNet models and the image encoder space of image/text models respectively yield the highest similarity human behavior [66, 67, 61]. After embedding the images into the neural net's latent space, one obtains a representation matrix X E R for the n images in the data. Instead of simply computing the dot-produc similarity matrix S = XXT in RSA one typically uses either cosine similarity or Pearson correlation kernel to compute the affinity matrix,
x x
(x) xT,) 6( x,) [x xt,) |1all (x, 2., l12
(x; W;:)
cos(x,, z3))
|
where the cosine similarity kernel function cos( 2;)) or the Pearson correlation kernel function ((x;, 2 is applied to every (æ;, 2) vector pair of the matrix X for obtaining the final representa- tional similarity matrix S' " Here, we use the Pearson correlation kernel function O( æ;) to obtain a neural net's RSM. Pearson correlation is the centered version of cosine similarity and the ranking of the obtained similarities does not differ between the two kernel functions but Pearson correlation first centers the vectors to have zero mean and is therefore a more robust measure. For obtaining RSMs with transformed representations, the transforms are first applied to X before computing S'.
28

---

E.3 Representational Similarity Analysis (RSA)
Additional RSMs. To corroborate our findings from 4.5 here additionally show RSMs for CLIP RNSO and CLIP ViT-L/14 (Laion 2B). In accordance with the different RSMs obtained from the representation space of CLIP ViT-1/144 (WTT), there does not appear be qualitative difference in the global similarity structure between the RSMs obtained from applying either the naive or the gLocal transforms CLIP RN50 or CLIP ViT-L/14 (Laion 2B) (see Fig. 14). Hence, the gLocal transform improves representational alignment while preserving the local similarity structure of the original representation equally well for the different CLIP models, as we show Tab,
Human
Driginal
Humman
Dr Oraginl
harve
NMtraon 120211 Indetti CEOD
City
Brg (2233)
(a) CLIP RN50
(b) CLIP ViT-L/14 (Laion 2B)
Figure Here, we show representational similarity matrices (RSMs) for human behavior and RNSO [WTT: 70] and CLIP ViT-L/14 [Laion 2B; 82) for four different human similarity judgment datasets [31 66 10 41) We contrast RSMs obtained from the network's 's original representation space (second column), the naively transformed representation space 6l] (third column), and the representation space obtained by using the goocal transform (rightmost column) against RSMs directly constructed from human similarity judgments (leftmost column).
F
Global transform derivation
Here derive that
min W a
W
EEe Wa/p) T
First, observe that
min
IW o/ 1
min ΣΣ
Wij al |=))) 2
iml jml
min
Σ
W
WWi a)²
1= j=
k=
P
W
min
Wit kk a)2
29

---

The minimizer of mina Ei=1 (Wi+ - a)² is attained with a Ei= Ww/p. Substituting this back into the last equality and reversing the steps from before have
W
min
WWk
a)²
W +
Wksk
Σ
i=
j=1,j
i=
k=
t=1
Wij
WHiP
[(==s]
1=1
W
Chi Ww/p)
which finishes our derivation.
G Properties of LCKA
Kornblith et al. 4 previously validated linear centered kernel alignment (L.CA)) as way to measure similarity between neural network representations. Given representations X RRxx and Y E Roxp2 containing embeddings of the same n images stacked row-wise, I.CKA is:
ICKA(X,Y))
XX
I
X
where X and Y are equal to X and Y with column means subtracted. (Formally, X = HX and Y = HY and H = I 11 is the centering matrix, which is a matrix representation of the linear operator that subtracts column means.)
As Kornblith et al. 43 note, linear CKA can be thought of as measuring the cosine similarity between all pairs of principal components (PCs) of and Y. weighted by the products of the proportions of variance these PCs explain in each representation. Formally, X UEVT T and Y U' and Y. The left-singular vectors u, U., are the (unit-norm) PCs of X, and the squared singular values Σ are the amount of variance that those PCs explain (up to a factor of 1/n). Given these singular value decompositions, linear CKA is:
be the singular value decompositions
Σ -
P2
X
u u())
LCKA(X, Y)
(8)
X
1 X²2
30


Reviews:
[{"review_id": "KFC2nojNY2", "reviewer": "Reviewer_2yQv", "summary": "This paper proposes a new alignment strategy (post-hoc) that manages to \"re-align\"/transform the learned feature space of modern neural networks into a new feature space that is correlated with that of humans. Authors find a way to do this by getting actual human judgment behavioural data and incorporate this in the loss function, and with only a relative low number of examples, can \"correct\" for the machines learned representations into a more accurate one. This paper's contributions is to show that this is: 1) actually possible with few data points ;2) show increase in performance and many computational tasks that can boost performance post-alignment correction.", "questions": "* What is meant by downstream tasks?\n* Is there a control for the post-hoc alignment? For example, what would have happened if you somehow maximized the global loss at alignment as a control and kept the local relations intact. Would the most still had achieved a higher performance?\n* It would have been cool to do qualitative assessments of alignment. As an example: What would have happened in terms of the difference of running an adversarial attack (like PGD) on the networks pre and post alignment? Do the perturbed stimuli now fool the human? I think this would be a pretty cool finding to show such as what can be done via adversarial training as shown in Santurakar et al. (NeurIPS, 2019 with ResNets) & Berrios & Deza (BSW, 2022 with Transformers), Harrington & Deza (ICLR 2022).\n* It is not obvious to me on what is the difference between the “naive” condition and “untransformed”. Both sound like the same, but I feel like I am wrong and missing something that hasn’t been really spelled out in the paper.", "limitations": "See questions. I overall think this is a good paper, I still do think that authors could have added more references in the NeuroAI space that do address the question of representational alignment through different training strategies (works of DiCarlo, Konkle, Krigeskorte, Yamins, etc... -- some of these works are cited, but not the most modern and relevant ones). In addition, it is worth mentioning that adversarial training in particular plays a *big* role as shown in works of Santurkar, Harrington & Deza.\n\nI'm overall very happy/lenient to increase my score if the questions + limitations are addressed!", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is on a very important topic such as representational alignment, and very likely one of the very few works that articulate why this is important (through their empirical experiments)\n* Authors provide a very clear mathematical reasoning on why their alignment procedure works (the optimization process to minimize the global distance while preserving the local distance too)\n* Authors provide many experiments showing the benefit  -- across multiple datasets -- that the alignment procedure works.", "weaknesses": "* There is a lack of qualitative assessment, aside from the RDM visualizastion. I suggest in the question part of this review that authors run adversarial attacks to see how the perturbed stimuli look like on both the aligned and un-aligned networks.\n* Somehow I feel like another control could have been used, such as an \"ortho-normal\" alignment e.g. preserve or maximize global distance but minimize local distance: do the results change?\n* Experiments of robustness such as testing the performance of the system on style-vs-texture image fusions to test for shape bias, and/or on the ImageNet-C dataset would be really interesting. If the model is truely aligned to human perception, then the o.o.d. performance should be exceedingly higher than non-aligned networks.\n*Authors focus is on accuracy. I think this is a missed opportunity to also focus more on robustness instead.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Responses**\n\n> There is a lack of qualitative assessment, aside from the RDM visualizastion.\n\nAlongside the RDM visualizations which demonstrate differences in global similarity structure between human similarity judgments and neural net representations (Figure 5 and Figure E.1), we did a thorough analysis of the local and global structure of the neural net representations for each of the conditions (see Section 4.2 *Impact of transforms on global and local structure*) where we perform PCA (*global*), t-SNE (*local*), and analyses on the nearest neighbor preservation (see Figures 2, 3, and 4). To corroborate our findings from Section 4.2, where we quantify the nearest neighbor preservation, we additionally visualize the nearest neighbor images in Appendix C in the Supplementary Material (see Figure C.1) which we think yields interesting insights into how local similarity structure is distorted by the naive transform but preserved (and sometimes even more intuitive) by the gLocal transform.\n\nMoreover, we dedicate an entire section (Appendix B in the Supplementary Material) to an analysis of the *global structure of the representations after alignment* where we investigate the movements of the representations of pairs of items from different superordinate categories from the THINGS dataset (see Figure B.1).\n\n> What is meant by downstream tasks?\n\nIn a *downstream task* one typically evaluates the representations of a pretrained model on a problem different from the pretraining task. For instance, to evaluate the downstream task performance of a model that was trained on image/text pairs with a contrastive learning objective such as CLIP, one has to test the representations on a task that is different from this pretraining task.\n\nNote that this can essentially be any task one cares about (image classification, predicting fMRI signals, etc.) as long as the goal of that task is clearly defined — often with targets. In our work, the downstream tasks we cared about were all concerned with image classification or subsets thereof. Specifically, we focussed on few-shot learning and anomaly detection. We think that these are useful tasks to evaluate the representations of a pretrained model because the prior (or *initialization* of a model) matters most in low data settings.\n\n> Is there a control for the post-hoc alignment? For example, what would have happened if you somehow maximized the global loss at alignment as a control and kept the local relations intact. Would the most still had achieved a higher performance?\n\nBecause adversarially training on the target task (human alignment) rather than on the auxiliary task (preserving local structure) causes problems with the optimization — there exists no lower bound on the cross-entropy error anymore because for adversarial optimization $\\mathcal{L}_{\\text{global}}({\\bf{W}}, {\\bf{b}}) \\in [-\\log2,-\\infty)$ —,  we decided to perform a task that’s highly similar to what you’ve suggested but better for optimization: Instead of training adversarially on the original THINGS triplet dataset, we created a new triplet dataset where for each triplet in the data we choose an object that is different from the original human choice to be the new odd-one-out. Note that this is not a random choice over all objects but a random choice over the set of two objects that were not chosen by a human participant. Using this new adversarial triplet dataset, we use the same gLocal optimization as we’ve done for the original task (see Equation 6).\n\n**Observations**: For FS, we see that the adversarial transform is slightly worse than the original transform and substantially worse than the gLocal transform. For AD, the adversarial probe is worse than both the naive transform and the original representation and substantially worse than the gLocal transform.\n\nThanks for suggesting to perform this experiment! See **Figure 1** in the PDF attached to the general response for a plot including adversarial probing results.\n\n> It is not obvious to me on what is the difference between the “naive” condition and “untransformed”. Both sound like the same, but I feel like I am wrong and missing something that hasn’t been really spelled out in the paper.\n\nAs explained in the methods section of our submission (see Equation 3), the **naive transform** is a linear transformation that is learned by solely minimizing the alignment loss plus some amount of $\\ell_{2}$-regularization on the transformation matrix $\\bf{W}$. That is, for learning the **naive transform** we solve Equation 3. After convergence, we can use this transformation for applying it to the neural network representations before evaluating downstream task performance.\n\nIn contrast to the **naive transform**, the **untransformed** (or **original** if you will) representation does not include any transformation. You can think of the transform in the *untransformed condition* as the identity matrix where $S_{ij} = (I {\\bf{x}}_i)^{T} (I{\\bf{x}}_j) = {\\bf{x}}_i^{T}{\\bf{x}}_j$. That is, we take the neural network representations as is.\n\n> [...], I still do think that authors could have added more references in the NeuroAI space [...]\n\nWe agree that tying stronger connections to the NeuroAI field is useful and thus we are happy to include more recent work from the NeuroAI space (from the labs that you’ve mentioned and beyond) in the introduction and the related works section. However, since none of our experiments directly involve neuro data we didn't want to dedicate too much space to neuro literature review in the initial submission (due to space constraints), but we'll use part of the additional page in the camera ready to expand the related work section. We will also mention the works of Santurkar, Harrington & Deza which seems to be pretty interesting. Thanks for the pointer!"}, {"review_id": "qy6Jy2pxdd", "reviewer": "Reviewer_eWR8", "summary": "This paper proposes to learn a linear transformation on top of pre-trained CLIP representations to improve alignment with human odd-one-out judgments from the things dataset. To do this, a similarity matrix is constructed based on pairwise similarities between the three images, and the representation is optimized so that the most similar pair matches the human decision. The proposed gLocal transformation combines the global loss, which encourages alignment with human annotations, with a local loss, which encourages the transformed representations to be similar to the untransformed representations. Evaluations are performed on downstream few-shot and anomaly detection tasks. ", "questions": "- I'm not clear on why the second part of table 4 is included with comparisons on other human similarity datasets, and not with the respective tables 1 and 2? It seemed to be an average of the different tasks presented in the earlier tables.\n- The experiments in the paper focus mostly on few-shot and anomaly detection settings. How does using the gLocal transformation perform on ImageNet linear probing or other transfer learning tasks?\n", "limitations": "Limitations are adequately addressed. The collected images span a subset of human similarity dimensions, and these may not necessarily align with ImageNet representations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The idea is clear and intuitively makes sense overall.\n- There are extensive experiments that evaluate a number of datasets for few-shot and anomaly detection benchmarks, and various model backbones.\n- The learned representations are analyzed using PCA, t-SNE, and CKA visualization. I found these analyses interesting to understand the changes that the representation undergoes during the training process.\n- Alignment to human preferences is demonstrated across multiple human annotated datasets.\n", "weaknesses": "- The results seem to vary by dataset and by model - the benefits are mostly on the CLIP model, and ImageNet backbones are less consistent in their benefits when using the gLocal objective. I would be curious if similar trends would hold for OpenCLIP or DINO backbones.\n- In most cases, the gLocal objective is compared against the naive objective (for example, Fig 1b, table 4). It would be helpful to also compare to the global objective (Eqn 4), or perhaps have a separate table that evaluates each component of the final loss function on a fixed set of tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Responses**\n\n\n> The results seem to vary by dataset and by model - the benefits are mostly on the CLIP model, and ImageNet backbones are less consistent in their benefits when using the gLocal objective. I would be curious if similar trends would hold for OpenCLIP or DINO backbones.\n\nWe report results for OpenCLIP models in our submission; these are the LAION-400M and LAION-2B models in our tables. The other two CLIP models are CLIP models that are trained on WIT. Since all of them are trained using the same objective function (Contrastive Language-Image Pretraining), they are generally referred to as CLIP models. To emphasize the difference between the two sets of models we report the dataset that they were trained on in brackets (WIT vs. LAION-X). Note that the dataset is the main difference between CLIP and OpenCLIP models.\n\nThe results do not vary much across the different CLIP $\\cup$ OpenCLIP models (see Figure 1 and Table 4) nor do they vary among ImageNet models (see tables in Appendix D). However, they vary between CLIP $\\cup$ OpenCLIP and ImageNet models. While we observed that achieving a *best-of-both-worlds* representation space — i.e., improved human alignment + better downstream task performance — generally works well for all CLIP $\\cup$ OpenCLIP models that we considered (see Figure 1), we were not able to obtain such a representation space for ImageNet models (see Appendix D in the Supplementary Material). \n\nWe hypothesize that this is because the visual information encoded in the representations of CLIP $\\cup$ OpenCLIP is much richer (due to dataset diversity + multi-model pretraining) compared to the representations of ImageNet models (fewer data points + unimodal training). Therefore, global similarity structure can be easily decoded from the CLIP representations, and at the same time we can preserve local similarity structure. There’s recent research that has shown that objective function and pretraining dataset have a much larger impact on the degree of alignment with human perception than the specific architecture or the size (width + depth) of a model [1] which may in part explain the differences that we observe between CLIP $\\cup$ OpenCLIP and ImageNet models. We provide an explanation for our findings in the Discussion section.\n\n\n\n> In most cases, the gLocal objective is compared against the naive objective (for example, Fig 1b, table 4). It would be helpful to also compare to the global objective (Eqn 4), or perhaps have a separate table that evaluates each component of the final loss function on a fixed set of tasks.\n\nWe performed this ablation. We report results in the Supplementary Material in A2 and D3 (please see Figure A.1 and Table D.7).\n\n> I'm not clear on why the second part of table 4 is included with comparisons on other human similarity datasets, and not with the respective tables 1 and 2? It seemed to be an average of the different tasks presented in the earlier tables.\n\nWe include the averages to make it easy to contrast human similarity with the downstream task accuracy for each transform type and to be in line with the results reported in Figure 1. We'll add a note that states this explicitly to the figure caption. It’s mainly done for better readability. In addition, we will report the naive transform accuracy for each task separately in the Supplementary Material.\n\n> The experiments in the paper focus mostly on few-shot and anomaly detection settings. How does using the gLocal transformation perform on ImageNet linear probing or other transfer learning tasks?\n\nWe’ve looked into this and just performed linear probing on ImageNet where we varied the number of samples per class for training the linear probe. For few-shot probing ($k<10$) on ImageNet we observe that the gLocal transforms generally outperform the original representations by a substantial margin, whereas for larger $k$ ($k>10$), we find that the gLocal transform and the original representations perform similarly well. This is not too surprising given that, since a linear transformation of the representation changes the linear probe solution by changing the impact of (explicit and implicit) regularization. Regularization matters most in low data regimes and becomes less relevant as the number of data points increases. Taking $\\lim_{k \\to \\infty}$, the optimal model has no regularization and a linear transformation has no impact. Please see Figure 2 in the PDF attached to the general response for a plot that shows ImageNet linear probing performance as a function of different $k$, where $k$ refers to the number of samples per class used for training the probe. We’ve also evaluated the gLocal transform using the entire ImageNet training set and observed that it performs as well as the original CLIP representations for the full data setting (see the table at the bottom of the PDF document attached to the general response).\n\n\n**References**\n\n[1] Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A Vandermeulen, and Simon Kornblith. Human alignment of neural network representations. In *11th International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, Mai 01-05, 2023*. OpenReview.net, 2023.\n\n"}, {"review_id": "7KVC6C1pEn", "reviewer": "Reviewer_dfyY", "summary": "This paper proposes a linear transformation named gLocal transform to align the global structure of the representation space with human similarity judgments, while at the same time maintains the local structure of the original representation space. The method is shown to be superior than a naive transform that minimizes only the global alignment loss and increases performance on few-shot learning and anomaly detection tasks.", "questions": "See Weaknesses above.", "limitations": "The paper has discussed the limitations.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "It is interesting to investigate the global structure of representation space and the correlation with human judgements. The paper further shows its practical value in down-stream tasks. The idea is novel. The experimental design is also interesting that verifies the human alignment of gLocal and naively transformed representations using four human similarity judgment datasets.", "weaknesses": "In regards to the experiments on few-shot learning, it would be beneficial to not only compare the results with CLIP experiments but also include experiments with general baselines. This will provide a comprehensive evaluation and determine if the proposed approach consistently improves the results compared to existing methods.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Response**\n\n\n> In regards to the experiments on few-shot learning, it would be beneficial to not only compare the results with CLIP experiments but also include experiments with general baselines. This will provide a comprehensive evaluation and determine if the proposed approach consistently improves the results compared to existing methods.\n\n\nWe refer the reviewer to Section D in the Supplementary Material, where we provide numerous experimental results for general baselines such as supervised models trained on ImageNet (e.g., ResNet50, VGG16, Alexnet). We find that the gLocal transform does not work as well for ImageNet models as it does for CLIP models. The gLocal transform does not seem to find a *best-of-both-worlds* representation space for ImageNet models. We don’t think that this is too surprising given that the visual information encoded in the representations of CLIP models is much richer (due to dataset diversity + multi-model pretraining) compared to the representations of ImageNet models (fewer data points + unimodal pretraining). Therefore, global similarity structure can be easier decoded from the CLIP representations vs. ImageNet model representations, while at the same time preserving local similarity structure. We explain our findings in the Discussion section of the main text. See also Muttenthaler et al. (2023) [1] for evidence on the importance of training data + objective function (vs. architecture + model size) for alignment with human similarity judgments.\n\n[1] Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A Vandermeulen, and Simon Kornblith. Human alignment of neural network representations. In *11th International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, Mai 01-05, 2023*. OpenReview.net, 2023."}, {"review_id": "tsAhdbGZ4a", "reviewer": "Reviewer_BF6t", "summary": "The paper proposes a new objective function on the representation level that enforces a global structure similar to that of humans, while maintaining as much as possible the original local structure. The goal is to maintain a separability of classes (ie keep local structure) while placing similar classes close to each other (ie enforce a global structure similar to humans), eg, cats of different breads should be placed more closely to each other than raccoons (global), while they should still be separable (local).\nThree main losses are proposed: (1) naive transform, which only maximizes the alignment between a NNs representation and human similarity judgements, ie only enforces a new human-like global structure; (2) global transform, which regularizes the transformed representation toward the original one,  ; (3) gLocal transform, which minimizes a combination of a global alignment loss while also maintaining a local structure with the use of a local contrastive loss.\nThe authors show that gLocal transform preserves the local structure of the original representation space while capturing the same global structure as the naive transform. They demonstrate that the gLocal transform improves performance on few-shot learning and anomaly detection tasks. The naive transform approach on the other hand does not improve performance. Finally, they compare the alignment of the transformed representations with human similarity judgment datasets and find that the gLocal transform yields only slightly worse alignment than the naive transform.", "questions": "* Why is LCKA the chosen RSM? The motivation behind the use of this measure could be made more clear in the text.\n* Why was RSM chosen for the final analysis on alignment with human judgments? There are many other representation similarity measures, e.g., CKA [1].\n* Why not test the representations on transfer learning (not only few shot learning or anomaly detection)?\n* Is there a way to quantify the t-SNE and PCA plots (e.g., Pearson correlation between them)? Sometimes only looking at images might lead to misleading conclusions.\n\n[1] Kornblith, Simon, et al. \"Similarity of neural network representations revisited.\" International conference on machine learning. PMLR, 2019.", "limitations": "The authors have adequately addressed the clear limitations of the proposed method.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The paper is well-written, clear and has an easy-to-follow flow. The method also has originality in the way that the authors piece the components of the losses together. The experiments sound ", "weaknesses": "The results only have marginal improvements over baseline. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Responses**\n\n> Why is LCKA the chosen RSM? The motivation behind the use of this measure could be made more clear in the text\n\nThe appropriate representational similarity measure here ultimately relates to how one defines “global representational structure” mathematically. If one defines global representational structure as the dot products or Euclidean distances (since $2x^\\top y = ||x||^2 + ||y||^2 - ||x - y||^2$) between the representations of the individual examples, then LCKA is a natural way to measure the similarity of global representational structure. The RBF kernel and other kernel functions that decay to 0 with distance are not suitable because our representational similarity measure should care about whether distances between points match even when they are far. Regression and CCA-based similarity measures are not suitable as they rescale directions in one or both representation spaces, and after doing so, points that were far apart could be close together. Although excluding these families of measures still leaves many to choose from, LCKA is arguably the simplest. Furthermore, the decomposition of LCKA in Appendix G seems intuitively appropriate to our case: LCKA effectively measures the alignment between pairs of PCs, putting greater weight on PCs that explain more variance.\n\n> Why was RSM chosen for the final analysis on alignment w/ human judgments? There are many other representation similarity measures, e.g., CKA\n\nRSA is a commonly used technique for comparing neural network representations (or essentially any latent representation) with human similarity judgments. In a multi-arrangement task, human participants are asked to freely arrange objects on a computer screen according to their *perceived* similarity [1,2]. These pairwise distances are subsequently aggregated across the different participants. \n\nCKA is a specific flavor of RSA where the RSMs/kernel matrices are positive semi-definite (PSD), the means are subtracted from the rows and columns, and the similarity between the RSMs is measured using cosine similarity (see Appendices E and G in the Supplementary Material for details on RSA and CKA respectively). The aggregated pairwise distances from the multi-arrangement task cannot necessarily be converted to a PSD RSM; a relationship between distance matrices and PSD similarity matrices exists only when the distance is a semi-metric of negative type [3]. The human pairwise similarity data obtained by Peterson et al. (2016; 2018) likewise does not result in a PSD matrix. Rather than try to justify the use of non-PSD matrices with CKA, we decided to use the same flavor of RSA as the work that introduced these datasets. This amounts to constructing a similarity matrix for the neural net representations using Pearson correlation between embeddings and measuring similarity between the RSMs using Pearson correlation. Based on our past experiences, there is little qualitative difference between this flavor of RSA and CKA.\n\n> Why not test the representations on transfer learning (not only few shot learning or anomaly detection)?\n\n\nFS and AD are commonly used downstream tasks that evaluate the transferability of pretrained representations (see response to Reviewer 2yQv). In addition, we've performed linear probing on ImageNet (see PDF in general response).\n\n> Is there a way to quantify the t-SNE and PCA plots (e.g., Pearson correlation between them)? Sometimes only looking at images might lead to misleading conclusions.\n\nAlthough we could’ve stated this more explicitly, in Figures 2 and 3 of the submission, we directly quantify the underlying high-dimensional structure that the t-SNE and PCA plots attempt to represent in low-dimensional space.\n\nt-SNE attempts to reduce the dimensionality of the data such that neighborhood relationships are preserved. In the first paragraph of Section 4.2 and Figure 2, we directly quantify the extent to which neighborhood relationships are preserved, observing much better preservation for the gLocal/global probes, in line with the t-SNE plots.\n\nIf we took the dot products between all of the pairs of points in each PCA plot and measured the cosine similarity between those dot products, the result would correspond to CKA after setting the singular values of each representation to zero for all but the two largest PCs. We perform a similar quantification in Figure 3 using the 10 largest PC. Using the top 2 PCs yields qualitatively similar results to using the top 10 PCs.\n\nFor completeness, we have also measured the mean-squared Euclidean distances between the points in the plots after standardization to unit variance in the table below. The conclusions match both those in Figures 2 and 3 and the qualitative appearance of the plots: the gLocal PCA plot closely resembles the naive PCA plot, whereas the gLocal t-SNE plot closely resembles the untransformed t-SNE plot.\n\n|                        | PCA           | t-SNE         |\n|------------------------|---------------|---------------|\n| Untransformed vs. gLocal |  0.14  | 0.05 |\n| Untransformed vs. Naive  | 0.19 | 0.20 |\n| gLocal vs. Naive         | 0.02 | 0.18 |\n\n\n**References**\n\n[1] Radoslaw M. Cichy, Nikolaus Kriegeskorte, Kamila M. Jozwik, Jasper J.F. van den Bosch, and Ian Charest. The spatiotemporal neural dynamics underlying perceived similarity for real-world objects. *NeuroImage*, 194:12–24, 2019. ISSN 1053-8119. doi: \nhttps://doi.org/10.1016/j.neuroimage.2019.03.031.\n\n[2]  Marcie L. King, Iris I.A. Groen, Adam Steel, Dwight J. Kravitz, and Chris I. Baker. Similarity judgments and cortical visual responses reflect different properties of object and scene categories in naturalistic images. *NeuroImage*, 197:368–382, 2019. ISSN 1053-8119. doi: https://doi.org/ 493 10.1016/j.neuroimage.2019.04.079.\n\n[3] Sejdinovic, D., Sriperumbudur, B., Gretton, A., & Fukumizu, K. (2013). Equivalence of distance-based and RKHS-based statistics in hypothesis testing. *The Annals of Statistics*, 2263-2291."}]

Author rebuttal:
We thank the reviewers for the overall very positive feedback. Although the reviewers’ concerns do not overlap , we would like to provide a general response where we address a few points that could be of interest to all reviewers.

**Adversarial alignment**

Reviewer 2yQv suggested to run the gLocal minimization (see Equation 6) to reduce alignment with human similarity judgments while still preserving local similarity structure in order to rule out confounding variables other than global similarity structure. We think that this is an interesting experiment and would like to thank reviewer 2yQv for their suggestion! 

To perform such an analysis, we created a new triplet dataset where for each triplet in the data we choose an object that is different from the original human choice to be the new odd-one-out. Note that this is not a random choice over all objects but a random choice over the set of two objects that were not chosen by a human participant. Using this new adversarial triplet dataset, we use the same gLocal optimization as we’ve done for the original task (see Equation 6). We minimize a bounded cross-entropy error and determine convergence on a held-out validation set as we’ve done before. This allows us to use the same hyperparameter setting as we’ve used for the gLocal transform. Across the different downstream tasks, we see that the adversarial transform is slightly worse than the original transform and substantially worse than the gLocal transform for all CLIP $\cup$ OpenCLIP models. Please see Figure 1 in the attached PDF for a plot including these adversarial alignment results.


**ImageNet linear probing**

Reviewer eWR8 wanted us to evaluate the gLocal transform on ImageNet via linear probing. We performed linear probing on ImageNet where we varied the number of samples per class used for training the probe. We observe that for small sample sizes ($k<10$) the gLocal transform outperforms the original representation, but performs on par with the original representation for larger sample sizes ($k>10$). As we describe in greater detail in the response to Reviewer eWR8, given that our transformation is linear, it is expected that it becomes less relevant as the number of data points increases. Therefore most of our experiments are concerned with few-shot learning and anomaly detection (which is zero-shot). Please see Figure 2 in the attached PDF for a plot that shows ImageNet linear probing performance as a function of different $k$, where $k$ refers to the number of samples per class used during training. We also provide a table at the bottom of the PDF document (below Figure 2) including linear probing results for the ImageNet full data setting.

We address the remaining points in individual responses to the reviewers.


Meta-review: The submission introduces and empirically demonstrates a framework to tune large-scale model representations via using additional human behavioral data. Positive and negative results are demonstrated and discussed, which may inspire further study. This interdisciplinary work will be of interest to several subcommunities at the conference.
Decision: Accept (poster)