SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lukas Muttenthaler*
BIFOLD
Lorenz Linhardt
Jonas Dippel
Robert A. Vandermeulen
Katherine Hermann
Mountain View, CA, USA
Andrew K. Lampinen
Simon Kornblith
Deep neural networks have reached human-level performance on many computer vision tasks. However, the objectives used to train these networks enforce only that similar images are embedded at similar locations in the representation space, and do not directly constrain the global structure of the resulting space. Here, we explore the impact of supervising this global structure by linearly aligning it with human similarity judgments. We find that a naive approach leads to large changes in local representational structure that harm downstream performance. Thus, we propose a novel method that aligns the global structure of representations while preserving their local structure. This global-local transform considerably improves accuracy across a variety of few-shot learning and anomaly detection tasks. Our results indicate that human visual representations are globally organized in a way that facilitates learning from few examples, and incorporating this global structure into neural network representations improves performance on downstream tasks.
Representation learning usually involves pretraining a network on a large, diverse dataset using one of a handful of different types of supervision. In computer vision, early successes came from training networks to predict class labels [47, 85, 28]. More recent work has demonstrated the power of contrastive representation learning [9, 29, 70, 8]. Self-supervised contrastive models learn a space where images lie close to other augmentations of the same image [9], whereas image/text contrastive models learn a space where images lie close to embeddings of corresponding text [70, 39, 68].
*Work done as part of the Google Research Collabs Programme.
$^{\dagger}$  Berlin Institute for the Foundations of Learning and Data, Berlin, Germany.
Although these representation learning strategies are effective, their pretraining objectives do not directly constrain the global structure of the learned representations. To classify ImageNet images into 1000 classes, networks' penultimate layers must learn representations that allow examples of a given class to be linearly separated from representations of other classes, but the classes themselves could be arranged in any fashion. The objective itself does not directly encourage images of tabby cats to be closer to images of other breeds of cats than to images of raccoons. Similarly, contrastive objectives force related embeddings to be close, and unrelated embeddings to be far, but if a cluster of data points is sufficiently far from all other data points, then the location of this cluster in the embedding space has minimal impact on the value of the loss [62, 35, 9].
Even without explicit global supervision, however, networks implicitly learn to organize high-level concepts somewhat coherently. For example, ImageNet models' representations roughly cluster according to superclasses [19], and the similarity structure of neural network representation spaces is non-trivially similar to similarity structures inferred from brain data or human judgments [40, 92, 66, 80, 49]. The structure that is learned likely reflects a combination of visual similarity between images from related classes and networks' inductive biases. However, there is little reason to believe that this implicitly-learned global structure should be optimal.
(a) Conceptual cartoon.
Figure 1: Global-local (gLocal) transforms yield a best-of-both-worlds representation space, which improves overall performance. (a) The original representations capture local structure, such as that different trees are similar, but have poor global structure. The gLocal transform preserves local structure, while integrating global information from human knowledge; e.g., unifying superordinate categories, organizing by "animacy", or connecting semantically-related categories like "food" and "drink". (b) The gLocal transforms improve both human alignment and downstream task performance compared to original and naively aligned representations for image/text models. We report mean accuracies on anomaly detection and (5-shot) few-shot learning tasks.
(b) Downstream task performance vs. human alignment.
Human and machine vision differ in many ways. Some differences relate to sensitivity to distortions and out-of-distribution generalization [21, 22, 25]. For example, ImageNet models are more biased by texture than humans in their decision process [22, 34]. However, interventions that reduce neural networks' sensitivity to image distortions are not enough to make their representations as robust and transferable as those of humans [23]. Experiments inferring humans' object representations from similarity judgments suggest that humans use a complex combination of semantics, texture, shape, and color-related concepts for performing similarity judgments [31, 60].
Various strategies have been proposed to improve representational alignment between neural nets and humans [e.g., 66, 67, 2, 61], to improve robustness against image distortions [e.g., 34, 21], or for obtaining models that make more human-like errors [e.g., 22, 26]. Mutenthaler et al. [61] previously found that a linear transformation learned to maximize alignment on one dataset of human similarity judgments generalized to different datasets of similarity judgments. However, it remains unclear whether better representational alignment can improve networks' generalization on vision tasks.
Here, we study the impact of aligning representations' global structure with human similarity judgments on downstream tasks. These similarity judgments, collected by asking subjects to choose the odd-one-out in a triplet of images [32], provide explicit supervision for relationships among disparate concepts. Although the number of images we use for alignment is three to six orders of magnitude smaller than the number of images in pretraining, we observe significant improvements. Specifically, our contributions are as follows:
- We introduce the gLocal transform, a linear transformation that minimizes a combination of a global alignment loss, which aligns the representation with human similarity judgments, and a local contrastive loss that maintains the local structure of the original representation space.
- We show that the gLocal transform preserves the local structure of the original space but captures the same global structure as a naive transform that minimizes only the global alignment loss.
- We demonstrate that the gLocal transform substantially increases performance on a variety of few-shot learning and anomaly detection tasks. By contrast, the naive transform impairs performance.
- We compare the human alignment of gLocal and naively transformed representations on four human similarity judgment datasets, finding that the gLocal transform yields only marginally worse alignment than the naive transform.
How can we build models which learn representations that support performance on variable downstream tasks? This question has been a core theme of computer vision research [9, 27, 58], but the impact of pretraining on feature representations is complex, and better performance does not necessarily yield more transferable representations [e.g., 44, 45]. For example, some pretraining leads to shortcut learning [23, 50, 1, 57, 22, 34, 3, 91]. Because the relationship between training methods and representations is complicated, it is useful to study how datasets and training shape representations [33]. Standard training objectives do not explicitly constrain the global structure of representations; nevertheless, these objectives yield representations that capture some aspects of the higher-order category structure [e.g., 40] and neural unit activity [e.g., 92, 80] of human and animal representations of the same images. Some models progressively differentiate hierarchical structure over the course of learning [5] in a similar way to how humans learn semantic features [73, 18, 78, 5, 79]. Even so, learned representations still fail to capture important aspects of the structure that humans learn [6]. Human representations capture both perceptual and semantic features [7, 69], including many levels of semantic hierarchy (e.g. higher-level: "animate" [11], superordinate: "mammal", basic: "dog", subordinate: "daschund"), with a bias towards the basic level [74, 55, 38], as well as cross-cutting semantic features [84, 56].
While models may implicitly learn to represent some of this structure, these implicit representations may have shortcomings. For example, Huh et al. [37] suggest that ImageNet models capture only higher-level categories where the sub-categories are visually similar. Similarly, Peterson et al. [66] show that model representations do not natively fully capture the structure of human similarity judgments, though they can be transformed to align better [66, 2]. In some cases, language provides more accurate estimates of human similarity judgments than image representations [54], and imagedirect models can have more human-aligned representations [61]. How does human alignment affect downstream performance? Sucholutsky & Griffiths [87] show that models which are more human-aligned (but not specifically optimized for alignment) are more robust on few-shot learning tasks. Other work shows benefits of incorporating higher-level category structure [51, 83]. Here, we ask whether transforming model representations to align with human knowledge can improve transfer.
Data. For measuring the degree of alignment between human and neural network similarity spaces, we use the THINGS dataset, which is a large behavioral dataset of 4.70 million unique triplet responses crowdsourced from 12,340 human participants for 1854 natural object images [32]. Images used for collecting human responses in the triplet odd-one-out task were taken from the THINGS object concept and image database [30], which is a collection of natural object images.
Odd-one-out accuracy. The triplet odd-one-out task is a commonly used task in the cognitive sciences to measure human notions of object similarity without biasing a participant into a specific direction [20, 72, 31, 60]. To measure the degree of alignment between human and neural network similarity judgments in the THINGS triplet task, we examine the extent to which the odd-one-out can be identified directly from the similarities between images in models' representation spaces. Given representations  $\boldsymbol{x}_1, \boldsymbol{x}_2$ , and  $\boldsymbol{x}_3$  of the three images in a triplet, we first construct a similarity matrix  $S \in \mathbb{R}^{3 \times 3}$  where  $S_{i,j} \coloneqq \boldsymbol{x}_i^\top \boldsymbol{x}_j / (\| \boldsymbol{x}_i \|_2 \| \boldsymbol{x}_j \|_2)$ , the cosine similarity between a pair of representations. We identify the closest pair of images in the triplet as  $\arg \max_{i,j > i} S_{i,j}$  with the
<sup>3</sup>We use cosine similarity rather than the dot product because Mutenthaler et al. [61] have shown that it nearly always yields similar or better zero-shot odd-one-out accuracies.
remaining image being the odd-one-out. We define odd-one-out accuracy as the proportion of triplets where the odd-one-out matches the human odd-one-out choice.
Alignment loss. Given an image similarity matrix  $S$  and a triplet  $\{i,j,k\}$  (here, images are indexed by natural numbers), the likelihood of a particular pair,  $\{a,b\} \subset \{i,j,k\}$ , being most similar, and hence the remaining image being the odd-one-out, is modeled by the softmax of the object similarities,
$$
p (\{a, b \} | \{i, j, k \}, S) := \exp (S _ {a, b}) / \left(\exp (S _ {i, j}) + \exp (S _ {i, k}) + \exp (S _ {j, k})\right). \tag {1}
$$
For  $n$  triplet responses we use the following negative log-likelihood, precisely defined in [60],
$$
\mathcal {L} _ {\text {g l o b a l}} (\boldsymbol {S}) := - \frac {1}{n} \sum_ {s = 1} ^ {n} \log \underbrace {p \left(\left\{a _ {s} , b _ {s} \right\} \mid \left\{i _ {s} , j _ {s} , k _ {s} \right\} , \boldsymbol {S}\right)} _ {\text {o d d - o n e - o u t p r e d i c t i o n}}. \tag {2}
$$
Since the triplets in [32] consist of randomly selected images, the concepts that humans use for their similarity judgments in the THINGS triplet odd-one-out task primarily reflect superordinate categories rather than fine-grained object features [31, 60], the above alignment loss can be viewed as a loss function whose objective is to transform the representations into a globally-restructured human similarity space where superordinate categories are emphasized over subordinate categories.
Naive transform. We first investigate a linear transformation that naively maximizes alignment between neural network representations and human similarity judgments with  $L_{2}$  regularization. This transformation consists of a square matrix  $W$  obtained as the solution to
$$
\underset {\boldsymbol {W}, \boldsymbol {b}} {\arg \min } \mathcal {L} _ {\text {g l o b a l}} (\boldsymbol {S}) + \lambda \| \boldsymbol {W} \| _ {\mathrm {F}} ^ {2}, \tag {3}
$$
where  $S_{ij} = (\boldsymbol{W}\boldsymbol{x}_i + \boldsymbol{b})^\top (\boldsymbol{W}\boldsymbol{x}_j + \boldsymbol{b})$ . We call this transformation the naive transform because the regularization term helps prevent overfitting to the training set, but does not encourage the transformed representation space to resemble the original space. Muttenthaler et al. [61] previously investigated a similar transformation. We determine  $\lambda$  via grid-search using  $k$ -fold cross-validation (CV). To obtain a minimally biased estimate of the odd-one-out accuracy of the transform, we partition the 1854 objects in THINGS into two disjoint sets, following the procedure of Muttenthaler et al. [61].
Global transform. The naive transform does not preserve representational structure that is irrelevant to the odd-one-out task. The global transform instead shrinks  $\mathbf{W}$  toward a scaled identity matrix by penalizing  $\min_{\alpha} \| \mathbf{W} - \alpha I \|_{\mathrm{F}}^2$ , thus regularizing the transformed representation toward the original. The global transform solves the following minimization problem,
$$
\underset {\boldsymbol {W}, \boldsymbol {b}} {\arg \min } \mathcal {L} _ {\text {g l o b a l}} (\boldsymbol {S}) + \lambda \underset {\alpha} {\min } \| \boldsymbol {W} - \alpha I \| _ {\mathrm {F}} ^ {2} = \underset {\boldsymbol {W}, \boldsymbol {b}} {\arg \min } \mathcal {L} _ {\text {g l o b a l}} (\boldsymbol {S}) + \lambda \left\| \boldsymbol {W} - \left(\sum_ {j = 1} ^ {p} \boldsymbol {W} _ {j j} / p\right) I \right\| _ {\mathrm {F}} ^ {2}. \tag {4}
$$
We derive the above equality in Appx. F. Again, we select  $\lambda$  via grid-search with  $k$ -fold CV.
gLocal transform. In preliminary experiments, we observed a trade-off between alignment and the transferability of a neural network's human-aligned representation space to downstream tasks. Maximizing alignment appears to slightly worsen downstream task performance, whereas using a large value of  $\lambda$  in Eq. 4 leads to a representation that closely resembles the original (since  $\lim_{\lambda \to \infty}W = \sigma I$ ). Thus, we add an additional regularization term to the objective with the goal of preserving the local structure of the network's original representation space. This loss term can be seen as an additional constraint on the transformation matrix  $W$ .
We call this loss term local loss and the transform that optimizes this full objective the gLocal transform, where global and local representations structures are jointly optimized. For this loss function, we embed all images of the ImageNet train and validation sets [17] in a neural network's  $p$ -dimensional penultimate layer space or image encoder space of image/text models. Let  $\mathbf{Y} \in \mathbb{R}^{m \times p}$  be a neural network's feature matrix for all  $m$  images in the ImageNet train set. Let  $\mathbf{S}^*$  be the cosine similarity matrix using untransformed representations where  $S_{ij}^* = (\mathbf{y}_i^\top \mathbf{y}_j) / (||\mathbf{y}_i||_2||\mathbf{y}_j||_2)$  and let  $\mathbf{S}^\dagger$  be the cosine similarity matrix of the transformed representations where
$$
S _ {i j} ^ {\dagger} = \left(\left(\boldsymbol {W} \boldsymbol {y} _ {i} + \boldsymbol {b}\right) ^ {\top} \left(\boldsymbol {W} \boldsymbol {y} _ {j} + \boldsymbol {b}\right)\right) / \left( \right.\left\| \boldsymbol {W} \boldsymbol {y} _ {i} + \boldsymbol {b} \right\| _ {2} \left. \right\| \boldsymbol {W} \boldsymbol {y} _ {j} + \boldsymbol {b} \| _ {2}\left. \right).
$$
Let  $\sigma$  be a softmax function that transforms a similarity matrix into a probability distribution,
$$
\sigma (\boldsymbol {S}, \tau) _ {i j} := \frac {\exp (\boldsymbol {S} _ {i j} / \tau)}{\sum_ {k} ^ {m} \mathbb {1} _ {[ k \neq j ]} \exp (\boldsymbol {S} _ {i k} / \tau)},
$$
where  $\tau$  is a temperature and  $\sigma (S,\tau)_{ij}\in (0,1)$ . We can then define the local loss as the following contrastive objective between untransformed and transformed neural network similarity spaces,
$$
\mathcal {L} _ {\text {l o c a l}} (\boldsymbol {W}, \boldsymbol {b}, \tau) := - \frac {1}{m ^ {2} - m} \sum_ {i} ^ {m} \sum_ {j} ^ {m} \mathbb {1} _ {[ i \neq j ]} \sigma \left(\boldsymbol {S} ^ {*}, \tau\right) _ {i j} \log \left[ \sigma \left(\boldsymbol {S} ^ {\dagger}, \tau\right) _ {i j} \right]. \tag {5}
$$
To avoid distributions that excessively emphasize the self-similarity of objects for small  $\tau$ , we exclude elements on the diagonal of the similarity matrices. The final gLocal transform is then,
$$
\underset {\boldsymbol {W}, \boldsymbol {b}} {\arg \min } \underbrace {(1 - \alpha) \mathcal {L} _ {\text {g l o b a l}} (\boldsymbol {W} , \boldsymbol {b})} _ {\text {a l g i n m e n t}} + \underbrace {\alpha \mathcal {L} _ {\text {l o c a l}} (\boldsymbol {W} , \boldsymbol {b} , \tau)} _ {\text {l o c a l i t y - p r e s e r v i n g}} + \lambda \left\| \boldsymbol {W} - \left(\sum_ {j = 1} ^ {p} \boldsymbol {W} _ {j j} / p\right) I \right\| _ {\mathrm {F}} ^ {2}, \tag {6}
$$
where  $\alpha$  is a hyperparameter that balances the trade-off between human alignment and preserving the local structure of a neural network's original representation space. We select values of  $\alpha$  and  $\lambda$  that give the lowest alignment loss via grid search; see details in Appx. A.2.
3.1 Downstream tasks
Few-shot learning. In general, few-shot learning (FS) is used to measure the transferability of neural network representations to different downstream tasks [86]. Here, we use FS to investigate whether the gLocal transform, as defined in §3, can improve downstream task performance and, hence, a network's transferability, compared to the original representation spaces. Specifically, we perform FS with and without applying the gLocal transforms. For all few-shot experiments, we use multinomial logistic regression, which has previously been shown to achieve near-optimal performance when paired with a good representation [88]. The regularization parameter is selected by  $n_s$ -fold cross-validation, with  $n_s$  being the number of shots per class (more details in Appx. A.3).
Anomaly detection. Anomaly detection (AD) is a task where one has a collection of data considered "nominal" and would like to detect if a test sample is different from nominal data. For semantic AD tasks, e.g. nominal images contain a cat, it has been observed that simple AD methods using a pretrained neural network perform best [53, 4, 16]. In this work, we apply  $k$ -nearest neighbor AD to representations from a neural network, a method which has been found to work well [4, 53, 71]. We use the standard one-vs.-rest AD benchmark on classification datasets, where a model is trained using "one" training class as nominal data and performs inference with the full test set with the "rest" classes being anomalous [75].
In this section, we report experimental results for different FS and AD tasks. In addition, we analyze the effect of the gLocal transforms on both local and global similarity structures and report changes in representational alignment of image/text models for four human similarity judgment datasets. We start this section by introducing the different tasks and datasets and continue with the analyses.
CIFAR-100 coarse The 100 classes in CIFAR-100 can be grouped into 20 semantically meaningful superclasses. Here, we use these superclasses as targets for which there exist 100 test images each.
CIFAR-100 shift simulates a distribution shift between the normal distribution of the training and testing images in CIFAR-100. For each of the 20 superclasses, there exist five subclasses. We use the first three subclasses for training and the last two subclasses for testing.
Entity-13 and Entity-30 are datasets derived from ImageNet [17, 76]. They have been defined as part of the BREEDS dataset for subpopulation shift [77]. In BREEDS, ImageNet classes are grouped into superclasses based on a modified version of the WordNet hierarchy. Specifically, one starts at the Entity node of that hierarchy and traverses the tree in a breadth-first manner until the desired level of granularity (three steps from the root for Entity-13 and four for Entity 30). The classes residing at that level are considered the new coarse class labels and a fixed number of subclasses are sampled from the trees rooted at these superclass nodes — twenty for Entity-13 and eight for Entity-30. Through this procedure, Entity-13 results in more coarse-grained labels than Entity-30. The subclasses of each superclass are partitioned into source (training) and target (test) classes, introducing a subpopulation shift. For testing, we use all 50 validation images for each subclass.
Figure 2: gLocal and global but not naive transforms preserve nearest neighbors in CLIP ViT-L representations. y-axis indicates the percentage of ImageNet validation images for which the closest image in the untransformed space is among the  $k$  closest after transformation.
Figure 3: The top principal components (PCs) of gLocal and global representations of ViT-L on the ImageNet validation set resemble those of the naive transformed representation, indicating that they share similar global structure. The remaining PCs more closely match the untransformed representation. Left: CKA between full representations. Middle: CKA after setting singular values of each representation to zero for all but the largest 10 PCs. Right: CKA after setting singular values to zero for the largest 10 PCs, but retaining smaller PCs.
4.2 Impact of transforms on global and local structure
Our goal is to use human similarity judgments to correct networks' global representational structure without affecting local representational structure. Here, we study the extent to which our method succeeds at this goal. To quantify distortion of local structure, we first find the nearest neighbor of each ImageNet validation image among the remaining validation 49,999 images in the original representation space. We then measure the proportion of images for which the nearest neighbor in the untransformed representation space is among the closest  $k$  images in the transformed representation space. As shown in Fig. 2, both the gLocal and global transforms generally preserve nearest neighbors, although the gLocal transform is slightly more effective, preserving the closest neighbor of  $76.3\%$  of images vs.  $73.7\%$  for the global transform. By contrast, the naive, unregularized transform preserves the closest neighbor in only  $12.2\%$  of images. For further intuition, we show the neighbors of anchor images in the untransformed, gLocal, and naively transformed representation spaces in Appx. C.