Decoupling Semantic Similarity from Spatial Alignment for Neural Networks.
Authors: Tassilo Wald
Constantin Ulrich
Priyank Jaini
Gregor Koehler
David Zimmerer
Stefan Denner
Fabian Isensee
Michael Baumgartner
Klaus Maier-Hein
Venue: NeurIPS 2024 poster (2024)
Keywords: Representational Similarity
Representational Similarity Analysis
Computer Vision
Primary area: other
TL;DR: We make representational similarity matrices permutation invariance and show resulting improvements in retrieval.

Abstract: What representation do deep neural networks learn? How similar are images to each other for neural networks? Despite the overwhelming success of deep learning methods key questions about their internal workings still remain largely unanswered, due to their internal high dimensionality and complexity. To address this, one approach is to measure the similarity of activation responses to various inputs.
Representational Similarity Matrices (RSMs) distill this similarity into scalar values for each input pair.
These matrices encapsulate the entire similarity structure of a system, indicating which input lead to similar responses.
While the similarity between images is ambiguous, we argue that the spatial location of semantic objects does neither influence human perception nor deep learning classifiers. Thus this should be reflected in the definition of similarity between image responses for computer vision systems. Revisiting the established similarity calculations for RSMs we expose their sensitivity to spatial alignment. In this paper we propose to solve this through _semantic RSMs_, which are invariant to spatial permutation. We measure semantic similarity between input responses by formulating it as a set-matching problem. Further, we quantify the superiority of  _semantic_ RSMs over _spatio-semantic_ RSMs through image retrieval and by comparing the similarity between representations to the similarity between predicted class probabilities.

Decoupling Semantic Similarity from Spatial Alignment for Neural Networks
Tassilo Wald 1,2,3 Constantin Ulrich 1,4,7 Gregor Köhler 1,3
David Zimmerer 1.2 Stefan Denner 1.3 Michael Baumgartner 1.3 Fabian Isensee 1.2 Priyank Jaini Klaus H. M 1,1,2,3,4,6
Division of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg, Germany Helmholtz Imaging, DKFZ, Heidelberg, Germany Faculty of Mathematics and Computer Science,
University Heidelberg, Germany Medical Faculty Heidelberg, University of Heidelberg, Germany 5 Google Deepmind
Pattern Analysis and Learning Group, Department Radiation Oncology National Center for Tumor Diseases (NCT) Heidelberg, Germany
Abstract
Wh do deep neural networks learn? How similar are images to each other for neural networks? Despite the overwhelming success of deep learning methods key questions about their internal workings still remain largely unanswered, due to their internal high dimensionality and complexity. To address this, one approach is measure the similarity of activation responses to various inputs. Representational Similarity Matrices (RSMs) distill this similarity into scalar values for each input pair. These matrices encapsulate the entire similarity structure of system, indicating which input leads similar responses. While the similarity between images is ambiguous, we argue that the spatial location of semantic objects does neither influence human perception nor deep learning classifiers. Thus this should be reflected in the definition of similarity between image responses for computer vision systems. Revisiting the established similarity calculations for RSMs we expose their sensitivity to spatial alignment. In this paper, we propose to solve this through semantic RSMs, which are invariant to spatial permutation. We measure semantic similarity between input responses by formulating it as a set-matching problem. Further, we quantify the superiority of semantic RSMs over spatio-semantic RSMs through image retrieval and by comparing the similarity between representations the similarity between predicted class probabilities.
1 Introduction
Deep neural networks are trained to extract powerful feature representations for wide range of downstream tasks. Despite this, their inner workings are highly-complex, making understanding how networks solve tasks and what they learn challenging. To obtain better understanding of these fundamental questions, researchers in the fields of neuroscience,
"Shared last authorship. "Corresponding author: tassilo. tasill.waldddkf-hhdeeeerr...d
38th Conference Neural Information Processing Systems (NeurIPS 2024).

---

cognitive science, and machine learning independently developed various methods to interpret and relate representations [29].
In the realm of machine learning. a lot of prior-methods have been proposed to meaningfully
31 25 19 [2]. measure the similarity between intermediate representations of ANNs [1]
Klabunde et al. [10] provides comprehensive summary and categorizes them info measures based on a) Canonical Correlation Analysis (CCA) [9] b) Alignment measures
<) Representational Similarity Matrices (RSMs) d) Nearest Neighbors e) Topologies and f) Descriptive Statistics.
Of all these methods, Representational Similarity Matrix (RSM) [13 based measures have enjoyed the most attention over the last years. RSMs consist of sample-to-sampll comparison, measuring the similarity between the (intermediate) responses of the same network to two different samples. Based on many such comparisons, the RSM represents the similarity structure of what model considers similar. This representation of a system's behavior reduces the highly-dimensionss interest may possess, to N x N Matrix for N samples. It enables the comparison of the similarity structure of any system, as long as one can input the same samples and measure the similarity between the responses.
complex internal structure, that the model of
In the machine learning domain, this concept was introduced by Kornblith et al. [2 in conjunction with Centered Kernel Alignment (CKA) to compare the similarity between RSMs of different layers within a model or across models. CKA superseded previously popular Canonical Correlation Analysis (CCA) metrics 225] 19], since they need vast amount of samples to measure similarity. Consequently, CKA was used in various applications, to measure the similarity between Transformers and CNNs [66 or wide and deep networks [21 and to understand catastrophic forgetting 27 or transfer learning [20] to name few.
In this paper, we revisit the key component of the most used similarity measure in the field: Representational Similarity Matrices and how they are constructed in the vision domain. The key contributions of our work are summarized as follows:
We highlight that current RSMs are constructed in way that couples localization and semantic information, which constraints one only to measure similarity if, spatial and semantic information aligns between two samples.
To address this issue we propose semantic RSMs, which are invariant to spatial permutation and exclusively measure semantic similarity, by formulating it as set-matching problem.
We show that the inter-sample similarity of semantic RSMs leads to improved retrieval performance and better reflects the similarity between representations of classifiers and their predictive behavior.
Moreover, due to the computational complexity of the proposed algorithm, we introduce approximations that significantly reduce computation time.
The Code is available here
2 Representational Similarity
Formalization To establish the concept of representational similarity in the context of computer vision, we provide brief formalization of the problem. Let r E {zo..... ,NN denote the input samples for which we collect input responses Z, E [21.0..... & and Z2, which we call representations. The representations can take different shapes, with ZCNN E R NxCx H denoting the responses of a CNN with C' channels and spatial extent of W. H or ZVIT E NxNT denoting the responses of ViT with depth D and tokens T. For the purpose of simplification and without loss of generality, we unify the spatial dimensions W, Hl for CNNs and 7 for VITs into joint spatial dimension S, resulting in
2 E NxCx9 Given representations 7, we can construct RSMs K, 7. E NxN, with values K₂ = k( 21,)) and T., = 1(22.1, #2,)) measuring how similar the representation 2, is to 2j given the kernels k or I, The kernels define the measure of similarity between the

---

Features
Samples
RSMs
K
0
0 000
Representations
Spaka Eqeasor
0000000000 00
K
000000000 01
Sample-.a
Akgener
Figure 1: Current spatio-semantic RSMs couple semantic similarity with spatial alignment. Our proposal focuses solely on measuring semantic similarity. We achieve this by deter- mining the optimal permutation between two representations and introducing sample-wise permutation invariance.
representation vectors and hence play important role. We introduce exemplary kernel in Section and all kernels used in this paper with some properties in Appendix With the two RSMs K and L hand, is possible compare the similarity structure between the two models. As introduced in Kornblith et al. 12 one can use the Hilbert- Schmidt independence criterion (HSIC) (5 28 to calculate the level of independence through
Centered Kernel Alignment, providing a measure of similarity of the two representations Z and Z2 with H denoting the centering matrix.
HSIC(K,
tr (KHLH)
(1)
(n 1)2
HSIC(K, L)
CKA(K, L)
(2)
/ /HSIC(K, K ()HSIC L)
Alternatively, variety of different measures based RSMs are possible for which refer to Section 3.3 of Klabunde et al. [10]. As highlighted above, the similarity calculation based RSMs is two-step process with the first being the calculation of the RSMs and the latter being the comparison of the RSMs. In this paper, we focus on the first step, by quantifying the importance of disentangling semantic similarity from spatial alignment. While not the focus of this paper, we provide qualitative examples of the downstream effect on CKA measures in Appendix
3 The Semantic Representational Similarity Matrix
Representational Similarity Matrices (RSMs) are designed to reflect the system behavior of interest. The RSM K, originally introduced by Kriegeskorte et al. [3 represents the similarity structure of system given set of inputs z E (xp.....z .N }. Each value Kúj in K quantifies how similar the responses of two inputs 2; and 2y are each other. The definition of what symmetries between representations similarity measures should be invariant to is central point of debate. Previous work proposed permutation invariance [15], invariance to orthogonal transformations [12,, or invariances to invertible linear transformations 255 19. While arguments for any of these invariances are valid, we believe that an important aspect has been neglected in the calculation of RSMs: The spatial alignment between the representations!
The dependency on spatial alignment Revisiting the structure of representations of CNN, channels C correspond to semantic concepts while the spatial position corresponds to where the semantic concept is localized in the input image [34]. Consequently, one can reformulate the representation 2, of sample x; to be fully defined by set of semantic concept vectors v, one for each spatial location S: 2; = (vo...., v.) with v E RC. In the case of linear CKA [12], the RSMs are then calculated, between semantic concept vectors at the same spatial location. For instance, when employing the linear kernel Kij can be expressed as:
Kij = Σ (V (V Vzs.e)
(3)
3

---

This formulation emphasizes the coupling of semantic similarity and localization during similarity calculation, which we term as sputio-semantic RSMs. This coupling can lead to issues, e.g. when comparing an image to a translated version of itself. Due to the quasi translation-eqqivariini nature of ANNa@semanti vectors are translated similarly, changing the alignment of pairs of v, leading to a low perceived similarity despite highly similar semantic vectors. This issue is visualized in a small toy example in Fig.
3.1 Decoupling Localization and Semantic Content
As shown above, current RSMs compare different input samples without accounting for the lack of spatial alignment. Previous work of Williams et al. [32 recognized this and introduced translation invariance to RSMs by finding the optimal translation a, 6 of the representations zj- = (Vo+a.0tb..... Vw+a maximize similarity Kúj through circular shifts.
maXa,5 = (24 z/)
While this is an improvement to spatial alignment and emulates a CNN's inherent translation equivariance, we argue that the measure of representational similarity should not be constrained to what the underlying model is invariant to, but the similarity measure should be invariant to the possible spatial configurations of semantic features in the input image.
To motivate this, we propose a thought experiment:
Imagine we have trained classifier with an augmentation pipeline including rotations. Given an image and rotated version of the image, we extract representations z at layer i once for the normal and once for the rotated image ri, Due to the initial rotation, these representations may differ in earlier layers i, due to the network extracting different edges and corners. However, if the network successfully learned to become invariant to the augmentation, it may have learned to map it to the same semantic vector v at a later layer but at different spatial location. For such cases, we argue that the similarity between the two representations should be high. Should the model be sensitive to the rotation, no semantically similar representations may be expressed at later layer, which should lead to a low similarity.
This reasoning can be extended to all kinds of shifts, be they artificial augmentations like shearing or mirroring or natural variations of the input manifold. Subsequently, we argue that the similarity measure should be invariant to as many spatial shifts as possible. This alone allows one to measure the similarity of representations a model is invariant to, be these learned or designed invariances. Such variable shifts cannot be captured with simple translation operations.
3.1.1 Introducing permutation invariance
To impose minimal constraints spatial structure possible, we propose to make Kij invariant to all spatial permutations of the semantic concept vectors v. Formalizing this we demand that the similarity Kiş- = k(2,,2) 2,) = k(2,, P,yzz) with P. E S*8 being a unique permutation matrix for the pair of 2; and zy. To accomplish this, propose to find the optimal permutation matrix Pij that maximizes the similarity Kij-
Pi argmax, k(z,, Pojzz)
To find the optimal permutation matrix Pıj, we decide to use the linear kernel (..)) to maximize both the magnitude of activation and the direction of vectors, as both magnitude of activation and direction of the vectors matter [2]] This allows us to calculate an affinity matrix Aij E Sx5 measuring the similarity between all concept vectors:
Aij = V (v..... vy.s).
With this affinity matrix, bipartite set-matching algorithms, such Hungarian matching, can be employed to find the optimal permutation matrix Pi, that maximizes the inner product between 2, and z/.. Finding all P.y for all pairs i, and applying the chosen kernel k yields the semantic RSM. This semantic RSM is invariant to any arbitrary, unique spatial
"The patch embedding similarly translate semantics to different position in the sequence.

---

Translated TinylmageNiet
Spatio-Semantic Semantic RBF RSMS
Aggregated category
Shared Content
Sample
Cuntaation
Compantion
magoo
up-shitted
down-shifted
ResNet18
Cons-Bi-Teell
Basi Blokk
Bustd Blocker
Besic Blockes
Layer
up-cthited (lieevaliitdd)
Layer ld
2
6
8
Seelanty
Layer
Semantic RBF RSMs (ours)
Figure 2: Semantic RSMs capture similarity independent of spatial localization, in contrast current spatio-semantic RSMs. We utilize Tiny-ImageNet generate partially overlapping crops of the same sample (left) and calculate RSMs for trained ResNet18 model. The plot displays the original spatio-semane semantic RSMs (middle bottom) across various layers for single batch. Additionally, the distribution of similarity values over multiple batches is shown (right). The results indicate that spatio-semantic RSMs struggle to detect largely identical but translated images, while semantic RSMs exhibit an enhanced off-diagonal the RSMs and a significant gap between distributions. This demonstrates the capability of our method to detect the same semantics even when translated.
RSMs (middle top) and our proposed
permutation P.j for each pair of representations, and, depending on the choice of kernel k, invariant to orthogonal transformations U BCxC along the channel dimension. These semantic RSMs can be used as a drop-in replacement for any other RSM, e.g. for applications such calculating CKA(K, L.) to measure the similarity between systems.
Computational Complexity Finding the optimal permutation matrix Pij is NP-hard and needs to be repeated for each pair of representations 2z, With samples, this results in N-(N+1) unique permutations that need to be computed for semantic RSM. The overall complexity of bipartite matching algorithms grows with the spatial dimensions cubed, resulting in O(N²) x O(S*). The outer O(N2) complexity can be parallelized, or reduced by decreasing the batch size. However, the inner permutation can become time- consuming, especially with large spatial dimensionality. To address this, we provide various approximations to reduce the complexity, which are detailed in Section 4.4 For all later experiments, except the translational toy example, we use the Batch-Optimal approximation with windows size 512, with the batch referring batches of semantic concept vectors v and not samples. The pseudo-code for calculating semantic RSMs visualized in the Appendix under Algorithm
Experiments: Semantic vs Spatio-Semantic RSMs
Given the novel permutation-invariant similarity definition, we evaluate the utility of our semantic RSMs relative to spatio-semantic RSMs for various similarity kernels, architectures, and tasks. Across all experiments we compare the linear kernel, the radial basis function (RBF) kernel, and the cosine similarity kernel, see Appendix for details.

---

Similanty Metric
0.6
Cosine Kernal
Retrieval
(ours) Cosine
0.4
Linerr Kemel
(ours) Pl Linear Keme!
0.2
RBF Kernel
(ours) Pl RBF Kernel
Dinov2-Giant
CLIP (VIT-B32)
CLiPSeg (Rd54)
SAM (VIT-B)
Architecture
Figure 3: Relaxing the constraint of spatial alignment leads to better retrieval. We leverage general feature extractors to embed images of the EgoObjects dataset. We then compare these embeddings either with or without permutation invariance. PI: Permutation Invariant
4.1 Translation sensitivity
To illustrate the problems of coupling semantic content and localization toy dataset is created using 84 x 84 pixels large, downsampled images of ImageNet [2. For each image two 64 x 64 crops are extracted, one from the upper-left and one from the lower-left corner, resulting in two images that share 44 64 identical pixels (Fig. left). Ten upper-left and ten lower-left crops are then used to extract representations of ResNet18 [7], which are subsequently used to calculate spatio-semantic and semantic RSMs at different layers of the architecture (Fig. 2 middle). As kernel, we use the radial basis function, as it provides bounded similarity values allowing better visualization.
As expected, the spatio-semantic RSM measures low similarity between pairs of overlapping crops, due to the semantic concept vectors not aligning. Only in the last layer, after many pooling operations, the off-diagonal is slightly expressed. Conversely, our semantic RSM is capable of detecting the high semantic similarity of the partially overlapping crops throughout the entire depth of the architecture, evident by the highly similar off-diagonal.
Aggregating the similarity values between nartiall-oveellapping and between different images across multiple batches, allows to measure the distribution of similarity values between overlapping crops, and non-related image comparisons. Throughout the entire depth of the architecture, the similarity distributions show that our measure better separates overlapping images from different images. Notably, the similarity distribution in spatio- semantic RSMs shows significant overlap of the distributions of partially overlapping images and non-related images, making differentiation between them difficult (Fig. [ right). A similar toy experiment for ViT-B/16 16 4 is provided in Appendix B
4.2 Similarity-based retrieval
To test the impact of the semantic RSMs in real-world applications, we now investigate the common task of image retrieval. Each entry in an RSM quantifies a sample-to-sample similarity value, which can be directly used for retrieval. While not specifically designed for it, we argue that better retrieval performance reflects better inter-sample similarity. This allows us to quantify improvements in the RSM structure. To measure retrieval performance the EgoObjects dataset 35| used. It contains frames of video that capture the same scene from different viewing perspectives and lighting conditions. This results in object centers being distributed across the extent of the image.
By randomly sampling 2000 query images and 5000 database images from the test set and using general feature extractors to extract embeddings from them we construct RSMs that allow us to do retrieval. As feature extractors we use CLIP (VIT /B32) (24, CLIPSeg (Rd64) 16], Vinovat [22], SAM /VIT/B32) 9 and BIT-50 [1] and as kernels for similarity calculation we use the cosine similarity, RBF and the inner product.
For all RSMs, we retrieve the most similar image that is not part of the same video the same scene but different conditions are allowed. As multiple objects can be present in each scene,

---

Cos Sim
(our) Perm. Inv. Cos Sim
Cos Sim
(our) Perm. Inv. Cos Sim
Query Image
1st Neighbor
2nd
Neighbor
3rd
Neighbor
Figure Retrieving by permutation invariant similarity returns similar scenes of different spatial geometry. We visualize the top most similar images according to two exemplary query images for SAM /B3.
we quantify retrieval performance by the F1-Score, measuring the overlap of annotated objects between images. Due to the rather complex dataset, we elaborate this in more detail in Appendix D.1
Across all architectures and metrics, the inclusion of permutation invariance (PI) for the similarity calculation improves retrieval performance relative to the non-invariant similarity, in some cases with a dramatic difference in performance, see Fig, 3 For models designed for dense downstream tasks like SAM CLIPSeg, the retrieval performane changes particularly much, while models with more global reasoning, like BiT improve less, relatively.
Qualitative Similarity Aside from a quantitative comparison, we visualize the most similar retrieved images for two exemplary queries of SAM in Fig, I as case examples.
Left Query: The image displays various utensils scattered the permutation-invaaiann similarity metric two images of the same scene but a very different perspective are successfully retrieved as most similar. Retrieving with the non-invariant similarity metric fails to retrieve similar images, due to lack of spatial alignment of the semantic concepts. Instead, it retrieves images of whiteboard, possibly due its spatial alignment with the paper on the desk.
desk. When retrieving with
Right Query: The image features a blender on a counter. The retrieval based on non- permutation-invariant similarity fails to retrieve any of the semantically similar scenes and returns images with a light switch, likely due to the spatial alignment of the light-switch- looking object to the right of the blender. Contrary, the retrieval based permutation- invariant similarity correctly returns the blender all cases from different perspectives. Additional qualitative examples are provided in Appendix D.3
These experiments display clearly, that demanding spatial alignment can be significant shortcoming when semantically similar concepts are misaligned. In Fig, 4 the network learned to represent the objects very similarly, despite shift in perspective, but due to the same objects not aligning anymore, spatio-semantic similarity fails to recognize this. This effect should generalize other datasets where objects are not heavily centered. For datasets with heavy object-centric behavior, like ImageNet, this should be less pronounced.

---

Table 1: Similarity invariant to spatial permutations is better at predicting if the class probabilities will be similar. Pl: Permutation Invariant
Pearson Correlation p Inner Product
Architectures
Cosine Sim. (ours) Pl
RBF
(ours) Pl
(ours) PI
ResNet18
-0.276
-0.326
-0.259
-0.270
-0.176
-0.199
ResNet50
-0.248
-0.291
-0.243
-0.261
0.040
0.029
ResNet101
-0.192
-0.276
-0.174
-0.240
0.091
0.084
ConvNextVV2Base
-0.134
-0.098
-0.132
-0.171
0.117
0.090
ViT-B/16
-0.046
-0.100
-0.045
-0.026
-0.077
-0.122
VIT-L/32
-0.138
-0.188
-0.138
-0.144
-0.134
-0.166
D2naV2-
-0.012
-0.044
-0.013
-0.031
-0.008
-0.048
4.3 Output similarity vs Representational Similarity
While the retrieval experiments relate to rather human notion of similarity, one can raise the question if semantic RSMs are also better at measuring the similarity for classifiers. For each pair of samples, we can compare how similar the predicted class probabilities of model are and compare this to the representational similarity. A commonly used metric for this is the Jensen- Sensen-Shannon Divergence (JSD), which quantifies how dissimilar the two probability distributions are from one another. More details are provided in Appendix
Consequently, use various classifiers trained to predict ImageNet1k from Huggingface and compare the Pearson correlation p between their JSD and the representational similarity of their last hidden layer. We chose to the Pearson correlation, it allows observing direct linear behavior between representational similarity and predictive similarity. Again measure semantic similarity and spatio-semantic similarity with different kernels. Due to JSD measuring dissimilarity, we want the correlation to be as negative as possible. As models we use multiple ResNets [71, ViTs convnextv2 33 classifier.
fine-tuned D2 [2] classifier from and
The results, displayed in Table show that for almost all architectures and kernels tested, the permutation invariant similarities are better at capturing the notion of what a classifier deems similar. While better than the spatio-semantic similarity, overall correlations low, indicating that either, the similarity metric is confounded by irrelevant representation tions, or that the kernels should be improved. Moreover, the RBF kernel sometimes provides positive correlation indicating it is unsuitable to predict the similarity of output probabilities, whereas the Cosine Similarity and the Inner Product both are consistently negative for all architectures tested.
generally
4.4 Optimizing runtime
Since we find the best possible permutation matrix through linear sum assignment a that maximize the inner product of two samples, we guarantee that the K Kij.emannti 2 K.yVi,j. This provides us with an upper bound of similarity that can leveraged to measure how much of the maximally achievable semantic similarity was measured by the spatio-semantic similarity. Additionally, it can be used as a baseline to estimate the quality of permutation matrices P. provided by faster, approximative assignment algorithms.
Decreasing computational complexity Determining the optimal permutation between samples poses substantial computational challenge with a complexity of O($$) for each of the ((V2) pairs in the same mini-batch, particularly for early layers with large spatial resolution S. Although, in theory, the calculation of the K matrix needs to be conducted only once for the desired representations, applying the method to representations with larger spatial extents becomes impractical with the demands of optimal matching.
To mitigate runtime, two options are available: reducing the batch size N lessen the number of permutation calculations or decreasing the time spent on finding the permutation. Given that scenarios like image retrieval often desire larger batches, our focus is on minimizing the time required to obtain suitable assignments.

---

Figure 5: Approximative algorithms yield comparable matching quality to optimal algorithms. The ratio of similarity from various approximations relative to maximal semantic similarity is visualized across mul- tiple layers of ResNet18.
Figure 6: Relative similarity not not isotropic. When aligning semantic concepts concetss we observe that similarity changes heterogeneously, in- dicating that some pairs of samples have more spatially misaligned semantic concepts than others.
Solving the optimal bipartite matching between semantic concept vectors is equivalent to the well-known assignment problem [8] 1. We attempted find existing approximate a algorithms primarily focus on optimal solutions, and existing approximate algorithm implementations, such as those based on the auction algorithm [6], are not runtime-optimizee often taking longer than optimal algorithms in our experiments. To enhance computational efficiency nonetheless, we explored three tailored approximation algorithms:
algorithms for this purpose. Unfortunately, most established
A) A Greedy breadth-first matching (Greedy)
B) An optimal matching of the TopK values based on their Norm, followed by the Greedy algorithm for the remaining samples (TopK-Greedy)
C) Optimal matching of smaller batches, with samples batched by their Norm (Batch- Optimal)
For explicit details on the approximation algorithms, we refer to Appendix We conducted comprehensive comparison between the approximate algonthmms and the optimal algorithm. We compare their runtime per sample and the quality of matches, quantified by the average relative similarity Koptima!" The evaluation utilized representations
from a ResNet18 on TinylmageNet, as illustrated Fig.
It can be seen that the measured spatio-semantic similarity for TinyImageNet samples are, on average, 30% lower with layers of higher spatial resolution exceeding 40%. This suggests notable misalignment of semantic concept vectors. Notably, the Batch-Optimal approximation stands out as reliable approximation for optimal matching. The fastest of the Batch-Optimal approximation methods shows < 8% error while improving run-time x36 relative to the fastest optimal algorithm for spatial extent 4096, while no spatial alignment shows 42% deviation from the optimal matching. Moreover, we highlight the time vs accuracy trade-off of the different optimal and approximate algorithms in Appendix F.] Furthermore, it can be seen that the changes between spatio-semantic and semantic RSMs are anisotropic, as highlighted in Fig. 6 indicating scale invariant downstream applications may be influenced.
Discussion, Limitations, and Conclusion
The concept of Representational Similarity Matrices (RSMs) is powerful tool to represent the similarity structure of complex systems. In this paper we revisit the construction of such RSMs for neural networks of the vision domain, question the current state, and propose semantic RSMs, warranting discussion.

---

Spatio-semantic coupling Being aware that current, spatio-semantic RSMs demand se- mantic concepts be aligned is highly relevant to understand what RSMs are sensitive to. Previous work 32 identified this shortcoming and proposed translation invariance, partially addressing this issue. We argue translation invariance is insufficient, since models may learn invariances during training, which the translation invariant metric would not be sensitive to. Subsequently, we propose a new spatially permutation invariant similarity measure between samples that allows the detection of similarity whenever model expresses similar semantic vectors in its representations, irrespective of spatial geometry. To highlight the benefits of our similarity, we propose that better similarity measures should allow more accurate retrieval when comparing last-layer representations and should allow better predic- tions about the similarity of class probabilities of a classifier. However, certain limitations in our current evaluation. Specifically, we have not yet compared our method to more established retrieval techniques. Traditional retrieval methods are often not applied to representations directly but utilize a liver-dimensional non-spatial, global vector representing the entire sample. In contrast, we chose to limit ourselves to methods that are directly applied to the representations.
acknowledge
Computational Complexity Aside from quantitative qualitative benefits, the construc- tion of semantic RSMs is time-consuuing, limiting its applicability. This complexity mostly affects layers of large spatial extent, which mostly corresponds to early CNN layers while later layers and ViTs are unproblematic. Our proposed Batch-optimal approximation alleviates this partially, yet application to large-scale representations at higher resolution, like at the output of segmentation architecture with spatial extents of s=65.536 would be too costly. We leave optimizing the compute efficiency or finding better approximations for future work.
Conclusion In conclusion, our investigation into semantic RSMs has shed light on the limitations of spatio-semantic RSMs and introduced a novel approach to disentangle spatial alignment from semantic similarity. The proposed method provides more accurate measure of how representations capture underlying semantic content, showcasing its potential in various applications, particularly in scenarios where spatial alignment cannot be assumed. While challenges such as computational complexity and scalability need to be addressed, the findings open avenues for further research and improvement in the analysis of neural network representations.
References
[]] R. E. Burkard. Quadratic assignment problems. European Journal of Operational Research, 15(3):283-289) 1984.
[2] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248-258 Ieee, 2009.
[3] F. Ding, Denain, and J. Steinhardt. Grounding representation similarity through statistical testing. Advances in Neural Information Processing Systems, 34:1556-1568, 2021.
[4] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint ar Xtt:2010.11929 2020.
[5] A. Gretton, R. Herbrich, A. Smola, O. Bousquet, B. Schölkopf, and A. Hyvärinen. Kernel methods for measuring independence. Journal of Machine Learning Research, 6(12), 2005.
[6] S. Guthe and D. Thuerck. Algorithm: A fast scalable solver for the dense linear (sum) assignment problem. ACM Trans. Math. Softw., 47(2), apr 2021. ISSN 0098-3500. doi: (.1145/3442345 http http://ooo.org///// 10. 1145/3442343
[7] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
[8] R. Jonker and T. Volgenant. A shortest augmenting path algorithm for dense and sparse linear assignment problems. In DGOR/NSOR: Papers of the 16th Annual Meeting
10

---

of DGOR in Cooperation with NSOR/Vortrige der 16. Jahrestugung der DGOR zusammen mit der NSOR, pages 622-622. Springer, 1988.
[9] A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. C.Berg, W.-Y. Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015-4026, 2023.
[10] M. Klabunde, T. Schumacher, M. Strohmaier, and F. Lemmerich. Similarity of neural network models: A survey of functional and representational measures. arXio preprint ar arXio:2305.006 06329, 2023.
[11] A. Kolesnikov, L. Beyer, X. Zhai, J. Puigcerver, J. Yung, S. Gelly, and N. Houlsby. Big transfer (bit): General visual representation learning. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part V 16, pages 491-507. Springer, 2020.
[12] S. Kornblith, M. Norouzi, H. Lee, and G. Hinton. Similarity of neural network representations revisited. In 36th International Conference on Machine Learning, ICML 2019, volume 2019-June, pages 6176-6175, 2019. iSBN 9781510886988. URL https
Saixiv.org/abs//10 00414
[13] N. Kriegeskorte, M. Mur, and P. A. Bandettini. Representational similarity analysis- connecting the branches of systems neuroscience. Frontiers in systems neuroscience, page 4, 2008.
[14] A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report,
2009.
[15] Y. J. Yosinski, J. Clune, H. Lipson, and J. Hopcroft. Convergent learning: Do different neural networks learn the same representations? Technical report, 2015.
[16] T. Lüddecke and A. Ecker. Image segmentation using text and image prompts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7086-7096, June 2022.
[17] S. Marcel and Y. Rodriguez. Torchvision the machine-vision package of torch. In Proceedings of the 18th ACM international conference on Multimedia, pages 1485-1488, 2010.
[18] S. Martello and P. Toth. Linear assignment problems. In North-Holland Mathematics Studies, volume 132, pages 259-282. Elsevier, 1987.
[19] A. S. Morcos, M. Raghu, and S. Bengio. Insights on representational similarity in neural networks with canonical correlation. Technical report, 2018. URL https
larxiv.org/all
05759.
[20] B. Neyshabur, H. Sedghi, and C. Zhang. What is being transferred in transfer learning? Advances in neural information processing systems, 33:512-523,, 2020.
[21] T. Nguyen, M. Raghu, and S. Kornblith. Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth. arXiv preprint ar Xite:2010.111 10110,,537,... 2020.
[22] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez,
D. Haziza, F. Massa, A. El-Nouby, al. Dinov2: Learning robust visual features without supervision. ar Xiv preprint ar arXin:2304.07199 2023.
[23] L. Perron and V. Furnon.
Or-tools.
URL
com/
optimization/
[24] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
[25] M. Raghu, J. Gilmer, J. Yosinski, and J. Sohl-Dickstein. SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability. Advances in Neural Information Processing Systems, 2017-DDecmmm6077--6666 jun 2017. URL http://arxiv. www.voorg/aaoo abs 1706. 05806
[26] M. Raghu, T. Unterthiner, S. Kornblith, C. Zhang, and A. Dosovitskiy. Do vision transformers see like convolutional neural networks? Advances in neural information processing systems, 14:12116-122229 2021.
2021.
11

---

[27] V. V. Ramasesh, E. Dyer, and M. Raghu. Anatomy of catastrophic forgetting: Hidden representations and task semantics. arXiv preprint arXit:2007.07400) 2020.
[28] L. Song, A. Smola, A. Gretton, J. Bedo, and K. Borgwardt. Feature selection via dependence maximization. Journal of Machine Learning Research, 13:1393-1434, 2012. ISSN 15324435.
[29] I. Suuholluts,,, L. Muttenthaler, A. Weller, A. Peng, A. Bobu, B. Kim, B. C. Love, E. Grant, Groen, J. Achterberg, et al. Getting aligned on representational alignment. arXiv
preprint arXio:2310113011 2023.
[30] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau,
E. Burovski, Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson,
K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. Carey, Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, Perktold, R. Cimrman, I. Henriksen, A. Quintero, C. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, SciPy 1.0 SciPy 1.0: Fundamental Algorithhs for Scientific Computing in Python. Nature Methods, 17:261-272. 2020. doi: 00.1088/s41592-019-06666
[31] L. Wang, L. J. Gu, Y. Wu, Z. Hu, K. He, and J. Hopcroft. Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation. Technical report, 2018. URL. httpss//aaravvvoogg/b////e 11750
[32] A. H. Williams, E. Kunz, S, Kormblith, and S. Linderman. Generalized shape metrics on neural representations. Advances in Neural Information Processing Systems, 34:4738-4750 2021.
[33] S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16133-16142, 2023.
[34] J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, and H. Lipson. Understanding neural networks through deep visualization. arviv preprint ar arXiv:1506.0633 06579, 2015.
[35] C. Zhu, F. Xiao, Alvarado, Y. Babaei, J. Hu, H. El-Mohri, Culatana, R. Sumbaly, and Z. Yan. Egoobjects: A large-scale egocentric dataset for fine-grained object understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20110-20120, 2023.
12

---

A Kernel function definitions
Defining the notion of similarity between two vectors is a matter of preference and wanted properties. In this work, we include the Radial Basis Function kernel Eq. (6) and the linear kernel Eq 7 as well as the cosine similarity kernel Eq. (8). We include the former two as they were proposed for RSM construction by Kornblith et al. [2] and the cosine similarity due to its popularity in the retrieval domain, despite generally being applied to class probabilities.
We denote that in our manuscript we refer to the linear kernel as the inner product and dot product interchangeably as they correspond to the same operation.
||x y11
KROF(X, y) exp
(6)
20
klinear (linear(X,y)) (x, y)
Reasine (x, yx (x,
(8)
||x|||y|| |lyl
Different properties The three selected kernels all have different properttes The RBF kernel and the Cosine similarity are bounded between kRBF(Z, y), kcosine (I, y) E [] Vx, y E R, while the kiinear y) is not bounded.
Moreover, the RBF kernel is parametrized by which influences at which rate the distance between the representations results in decrease in similarity. For all our experiments we choose a as the square root of the median Euclidean distance of all distances within mini-batch.
B Translation Sensitivity of a ViT-B/16
Similarly to the translated liny-ImageNet experiment, we prepared a very similar experiment for ViT-B/16 vision transformer that was pre-trained on ImageNet1k. We utilize the implementation and weights provided by torchvision [77]
Given the larger ImageNet images, we resample them to 324 324 pixels and crop two partially overlapping images of size 224 x 224 from it. Given these image pairs, we calculate semantic and spatio-semantic RSMs again, see Fig.
is important to note that our approach intentionally avoids achieving a perfect translation that would lead the same patchified tokens, as we shift the image by factor that is not divisible by 16, the patching window size. We believe this realistic imperfection is preferable to a perfect overlap, where identical tokens would be formed from the exact same set of pixels.
Similarly to the previous partially overlapping crop experiment, we can observe that spatio-semantic RSMs are incapable of identifying the largely identical content of the two partially overlapping crops due to their different localization. The spatio-semantic RSMs can capture this notion of similarity as evidenced by the off-diagonal and the larger gap in the distribution of similarity between partially overlapping samples and independent samples, Fig. 7 (bottom).
Contrary to the CNN example in the main, overall similarity between samples is much lower overall and separation between translated image pairs and random image pairs follows vastly different trajectory. While there is a profound difference, the origin of this difference cannot be clearly made out. We hypothesize that this may be due to the different ways that Transformers process information and learn different representations, as highlighted in Raghu et al. [26].
Moreover, we emphasize, that calculating the optimal permutation is significantly faster for ViTs than the CNNs, as the early tokenization reduces the spatial dimension S substan- tially at an early stage, whereas the iterative downsampling of CNNs makes comparing representations of early layers very costly.
13

---

Spatio-Semantic
VIT-B/16
Semantic RSMs
RSMs
Layer
Layer
Layer 10 12
Value Distribution
Value Distribution
Figure 7: Semantic RSMs do also capture spatial translations for token sequences of ViT's. We calculate spatio-semantic and semantic RSMs with the Radial Basis Function (RBF) kernel for a ViT-B/16 16 with representations extracted from ImageNet. Similarly, we introduce partially-overlapping crops that get tokenized and processed by the ViT. Due to the initial shift, the token sequence does not align anymore between the crops. Similar to the CNNs, spatio-semantic RSMs exhibit low similarity values the off-diagonals, providing limited indication of overlapping content between crops. In contrast, Semantic RSMs prove notably more effective discerning substantial overlap, offering higher similarity values in the off-diagonals and thereby indicating a greater degree of similarity between large portions of the image.
C Pseudo Code
In addition to our provided explanation of the algorithm in the main manuscript, provide the pseudo-code used to compute semantic RSMs in Algorithm [ The only difference between semantic and spatio-semantic RSMs algorithmically is where the optimal permutation is calculated that maximally aligns the two representations. The current definition of spatio-semantic RSMs assumes that spatial locations are corresponding, while semantic RSMs calculate correspondence through similarity matching.
D Additions: Retrieval Experiment
In addition to the provided retrieval examples in the main manuscript, we provide more details on the retrieval experiments in Appendix D.1 an additional table holding the quantitative data of Fig. with varying database sizes in Appendix D.2 and lastly, additional qualitative retrieval examples for each model including direct comparisons of all models for the same query image in Appendix D.3
D.1 Details of Retrieval Experiment
For the retrieval experiment, we utilize the EgoObjects datasstt 35 It contains multiple frames from multiple videos, with multiple videos capturing the same scene under different shifts like lighting conditions, distances, viewing angles, and different motion trajectories. For each frame, multiple objects of different categories can be present and are annotated
14

---

Algorithm 1: Semantic RSM calculation. We calculate the optimal permutation matrix,
resulting in maximal similarity between the representations of samples.
Data: E RXXXXS, Kernel k
Result: Semantic RSM K
== E R N
for i from 0 to N do
for j from to N do
if i < then
cont.
if i + then
P.j argmarr, (zi, Piya));
2)
else
z/j Zji
Kij klzi, 23));
KJi Kig
/ is symmetric */
through bounding box. Moreover, frames vary in spatial resolution, yet large fraction was captured 16:9 format of 1920 x 1028 pixels.
Image preprocessing For our experiments, comprised of 29.5K images, Of these 29.5k we remove all images not in 16:9 format and resize the remaining to the 1920 1028 format. This discards roughly 10k images.
utilize the EgoObjects test set, which is
Of all remaining images, we then draw 2k query images and 5k database images used for extracting embeddings for similarity calculation and later retrieval. Naturally, we sample in a way to keep the 2k query and 5k database image sets non-overlapping. When passing the images to the models for feature extraction each image is preprocessed according to the corresponding Huggingface by the shortest edge to the expected image input dimensions and normalizing the image. The only exception is SAM, of which we use the official implementation, which handles feature extraction and embedding of the image itself.
maagerreprocessor. This mostly represents resizing the image
Feature Extraction and preparation
As mentioned in the main manuscript we use
1. CLIP (ViT) /B2
24
2.
ClipSeg (Rd64)
16
3. DinoV2-Giant
22
4. SAM (ViI B32)
9 and
5. BIT-50
as general feature extractors as they were trained on vast amount of data
Of all models, we use the last hidden layer as image embeddings should they not per-default provide image embeddings as output. After extracting representations we calculate the mean from the database embeddings to zero-center all representations by, query and database representations alike.
RSM construction Given all 2000 query embeddings and 5000 database embeddings, we calculate the RSMs. To parallelize this process we mini-batch the representations into 100 x 100 pairs and populate the 2000 x 5000 matrix in this fashion. We denote that this proved to be necessary for models with large spatial embedding dimensions like SAM, starring 64 x 64 spatial extent. Moreover, we denote that, due to the RBF choosing its
*Last accessed on 22nd of May 2024
15

---

parameter based on the median of the measured values within one batch this patch-wise calculation not optimal for this kernel. The inner product and the cosine similarity kernels are not affected by this.
Retrieval measurement Given the RSMs containing sample-to-sample similaartl measure, we can retrieve the most similar sample of the database for each query from it. As each image can contain multiple objects measuring retrieval performance is not trivial. For both images we quantify how many objects of each class are present in the image, resulting in a count of class instances for each image. With the query image representing the ground truth (GT) and the database representing the prediction, we match class instance counts. Each correctly matched GT instance represents TP, each missed an FN and all unmatched database instances represent an FP.
To formalize: Let Qe be the number of instances of class e in the query image and De be the number of instances of class in the database image. With this, the used Fl metric can be expressed as
TP Σ min(Qe. Dc)
(9)
CEC
FN Σ max(0, Q. - De))
(10)
FP Σ FP. Σ
max(0, De Qc)
(11)
CEC
eEC
T.
Fl 2.TP FP+
(12)
D.2 Additional Quantitative Retrieval data
In addition to the results highlighted in Fig, 3 we provide retrieval results for varying database sizes retrieve from EgoObjects. Specifically, results for database sizes of 2.5k, 5k, and 10k samples are given in Table
16

---

Table 2: Quantitative results of the EgoObject retrieval experiment for multiple models and multiple database sizes. Database Size 5.000 represents Fig, of the main manuscript. It can be observed that models with a greater spatial extent (CLIPSeg and SAM) show greater improvement in retrieval performance over models with a lower spatial extent (DinoV2-Giant, BiT-50, CLIP). Moreover, it can be observed that retrieval improvements decrease with growing database size. This is likely due to the increasing odds of finding one image where spatial position and semantics align w.r.t. the query images. PI: Permutation Invariant, Diff: Difference (PI - None)
Database Size (N)
2,500
5,000
10.000
Invariance
None
Pl
Diff
None
Pl
Diff
None
PI
Diff
Architecture
Kernel
F1@1
Fl@1
F1@1
Cosine Sim.
0.240
0.781
0.541
0.280
0.835
0.555
0.344
0.859
0.515
CLIPSeg
Inner Product
0.233
0.548
0.315
0.281
0.624
0.343
0.328
0.569
0.241
RBF
0.205
0.781
0.576
0.247
0.836
0.589
0.293
0.854
0.561
Cosine Sim.
0.765
0.840
0.075
0.827
0.876
0.049
0.857
0.889
0.032
DinoV2-Giant
Inner Product
0.767
0.839
0.072
0.829
0.876
0.047
0.856
0.890
0.033
RBF
0.735
0.834
0.099
0.810
0.876
0.066
0.843
0.888
0.045
Cosine Sim.
0.670
0.799
0.129
0.750
0.846
0.096
0.800
0.868
0.068
BiT-50
Inner Product
0.659
0.780
0.122
0.730
0.823
0.093
0.782
0.852
0.069
RBF
0.582
0.789
0.208
0.668
0.835
0.167
0.741
0.859
0.118
Cosine Sim.
0.704
0.803
0.100
0.784
0.858
0.073
0.827
0.876
0.049
CLIP
Inner Product
0.703
0.792
0.090
0.780
0.848
0.067
0.823
0.867
0.044
RBF
0.584
0.779
0.196
0.679
0.835
0.156
0.749
0.863
0.114
Cosine Sim.
0.509
0.735
0.226
0.614
0.804
0.190
0.688
0.838
0.151
SAM VIT/B33
Inner RBF
0.511
0.695
0.183
0.616
0.774
0.158
0.688
0.815
0.126
0.371
0.703
0.332
0.474
0.784
0.310
0.556
0.823
0.267
D.3 Additional Qualitative Examples
In addition to the two qualitative examples provided for SAM in the main, additional examples are provided. Qualitative examples are picked from the first 50 query images when retrieving from a database size of 5.000 images. Additional qualitative retrieval examples are provided for Dino in Fig. S CLIP in Fig. 9 BiT-50 in Fig. 10 CLIPSeg in Fig. 11 and SAM in Fig. 12 Moreover, we highlight direct comparison between all models for the same images in Fig. 13 and Fig. 14
Across all models, it can be observed that models retrieve images where semantic content and spatial positions are aligned when using standard cosine similarity. This leads to larger-scale objects like desks, tiled floors, countertops dominating retrieval when imaged from a similar perspective. When decoupling spatial-alignmenn from semantic content, images of the same scene but different perspective get retrieved more often, leading to the same scene appearing more regularly in the nearest neighbors. This effect is currently not quantified in the Fl metric, due to only comparing object presence between the query and the 1st neighbor and not the average F1 between the query and the top 5 neighbors. We opted against using this, as retrieval metrics most commonly consider the maximum match in the top 5 neighbors.
Moreover, it can be observed that models with lower spatial extent can retrieve quite different neighbors as opposed to models with higher spatial extent, see Fig, 13 While Dino and CLiP retrieve very similar 4th and 5th neighbors of the same book object without the smaller mouse and headphone object, CLIPSeg and SAM retrieve scenes with these two objects still present instead.
D.4 Cityscapes Quantitative Retrieval
Since previous results were limited to the EgoObjects dataset we provide an additional quantitative experiment on CityScapes. Analog to before, we use N-500 validation images as the query dataset and the remaining N=2975 training images as the database for retrieval. We utilize the IoU metric to compare the presence of semantic classes between the images.
17

---

Qualitative Examples DinoV2
Cos Sim
(our) Perm. Inv. Cos Sim
Cos Sim
(our) Perm. Inv. Cos Sim
Query Image
1st
Neighbor
2nd Neighbor
3rd Neighbor
4th
Neighbor
5th
Neighbor
(b)
Sim
(our) Perm. Inv. Cos Sim
Sim
(our) Perm. Inv. Cos Bim
Query Image
1st Neighbor
2nd Neighbor
3rd Neighbor
4th Neighbor
5th Neighbor
(d)
Figure 8: Additional qualitative retrieval samples for Dino We visualize the top 5 most similar neighbors for four query images.
18

---

Qualitative Examples of CLIP
Cos Sim
(our) Perm. Inv. Cos Sim
Cos Sim
(our) Perm. Inv. Cos Sim
Query Image
1st Neighbor
2nd Neighbor
3rd Neighbor
4th
Neighbor
5th
Neighbor
(b)
Sim
(our) Perm. Inv. Cos Sim
Sim
(our) Perm. Inv. Cos Sim
Query Image
1st Neighbor
2nd Neighbor
3rd Neighbor
4th Neighbor
5th Neighbor
Figure 9: Additional qualitative retrieval samples for CLIP. We visualize the top 5 most similar neighbors for four query images.
19

---

Qualitative Examples of BiT-50
Cos Sim
(our) Perm. Inv. Cos Sim
Cos Sim
(our) Perm. Cos Sim
Query Image
1st
Neighbor
2nd Neighbor
3rd Neighbor
4th
Neighbor
5th
Neighbor
(b)
Sim
(our) Perm. Inv. Cos Sim
Sim
(our) Perm. Inv. Cos Bim
Query Image
1st Neighbor
2nd Neighbor
3rd Neighbor
4th Neighbor
5th
Neighbor
Figure 10: Additional qualitative retrieval samples for BiT-50. We visualize the top 5 most similar neighbors for four query images.
20

---

Qualitative Examples CLiPSeg
Cos Sim
(our) Perm. Inv. Cos Sim
Cos Sim
(our) Perm. Inv. Cos Sim
Query Image
1st
Neighbor
2nd Neighbor
3rd Neighbor
4th
Neighbor
5th
Neighbor
(b)
Sim
(our) Perm. Inv. Cos Sim
Sim
(our) Perm. Inv. Cos Bim
Query Image
1st Neighbor
2nd
Neighbor
3rd Neighbor
4th Neighbor
5th
Neighbor
Figure 11: Additional qualitative retrieval samples for CLIPSeg. We visualize the top most similar neighbors for four query images.
21

---

Qualitative Examples of SAM
Cos Sim
(our) Perm. Inv. Cos Sim
Cos Sim
(our) Perm. Inv. Cos Sim
Query Image
1st Neighbor
2nd Neighbor
3rd
Neighbor
4th
Neighbor
5th
Neighbor
(b)
Sim
(our) Perm. Inv. Cos Sim
Sim
(our) Perm. Inv. Cos Bim
Query Image
1st Neighbor
2nd Neighbor
3rd Neighbor
4th Neighbor
5th
Neighbor
Figure 12: Additional qualitative retrieval samples for SAM. We visualize the top 5 most similar neighbors for four query images.
22

---

Cos. Sim.
DinoV2
CLIP
BiT-50
CLIPSeg
SAM
Query Image
1st Neighbor
2nd Neighbor
3rd Neighbor
4th Neighbor
5th Neighbor
(our) Perm. Inv. Cos. Sim.
DinoV2
CLIP
BiT-50
CLIPSeg
SAM
Query Image
Neighbor
2nd Neighbor
3rd Neighbor
4th Neighbor
5th Neighbor
Figure 13: Direct comparison of models. We visualize the top 5 most similar images of all models retrieved through cosine similarity or permutation invariant cosine similarity.
23

---

Cos. Sim.
DinoV2
CLIP
CLIPSeg
Query Image
1st Neighbor
Neighbor
3rd Neighbor
4th Neighbor
5th Neighbor
(our) Perm. Inv. Cos. Sim.
DinoV2
CLIP
BiT-50
CLIPSeg
SAM
Query Image
Neighbar
2nd Neighbor
3rd Neighbor
4th Neighbor
5th Neighbor
Figure 14: Direct comparison of models. We visualize the top most similar images of all models retrieved through cosine similarity or permutation invariant cosine similarity.
24

---

Due to the lack of instance labels in the CityScapes dataset, we can't take the quantity of objects into account before, but can only consider if semantic class is present absent. As apparent from Table permutation invariance allows for consistently improved retrieval when utilizing cosine similarity or RBF kernels.
Table 3: Retrieval results for the Cityscapes Dataset. We retrieve the most similar image according to RSMs and calculate the loU query semantic classes and retrieved semantic classes. Overall query images used are validation images of size N=500 and the database are the training images of size N=2975, Differences between metrics are low, due to many images containing a large number of classes and the lack of instance label information. Despite this, permutation invariance improves Cosine Sim and RBF retrieval performance consistently, with the Inner Product showing mixed results.
Cosine Sim.
Inner Product (ours) PI
RBF (ours) PI
Invariance Architectures
(ours) PI
CLIPSeg
0.662
0.694
0.664
0.654
0.664
0.696
Dino
0.689
0.700
0.686
0.696
0.686
0.690
BiT-50
0.679
0.687
0.677
0.677
0.684
0.687
CLIP
0.691
0.701
0.693
0.701
0.678
0,687
SAM VIT /B3
0.690
0.702
0.687
0.677
0.679
0.687
E Details: Output similarity vs Representational Similarity
To measure the correlation between the inter-sample representational similarity and the prediction probability inter-sample similarity we utilize pre-trained classifiers and the imageNetlk [2] dataset. Unlike during the retrieval results this constraints the possible model selections to models trained for classification.
Image preprocessing Test set images of ImageNetlk are randomly sampled without applying any filtering to them. In total, we utilize subset of 2k ImageNet test set samples in this experiment. This may appear small, yet provides sufficient basis as the combinatoric growth increases the absolute number of measurements substantially.
Feature and logit extraction and preparation As mentioned in the main manuscript we use
1.
Res. Net18
7
2.
ResN
7
3.
ResNet101
4.
a DinoV2-Giant based classifier [2
5.
ConvNeXt V2
33
6.
166
4 and
7.
ViI-L/32
4 and
as pre-trained classifiers for predicting the ImageNetlk classes 3 For each sample, we extract the last hidden layer's representations and center them analog to before. For the same sample, we extract the logits and obtain the probability distribution through the softmax, saving the pair for later comparisons.
Correlation measurement For each pair of representations and probabilities, we calculate the similarities between their representations for all three kernel functions, once permutation invariant and once not. Additionally, we calculate the Jensen-Shannon Divergence (JSD)
S Last accessed on 22nd of May 2024
25

---

between the predicted class probability distributions for P and Q. A formal definition of the JSD provided in Eq. 13
1 DKI (P ISD(P Q)
1 D M)
(13)
where M is the pointwise mean of P and Q:
= 2 (P+
(14)
and DKL is the Kullback- divergence defined as:
P(() P(i) ( log Q(i)
DeL DKL (1
(15)
Given the paired JSD and Similarity K between all samples j. we utilize the Pearson correlation p to calculate the correlation between the two. Due to the JSD being for identical probabilities and increasing for more dissimilar values and the Similarity being for perfectly similar representations and for dissimilar representations, the desired correlation between the two should be negative.
E.1 Additional correlation results
In addition to the Pearson correlation between the Jensen Saseen-Shannon-iveereee (JSD) and inter-image similarity, we also present the results of their relationship measured by the Spearman correlation, as shown in Table While the Pearson correlation demonstrated consistently stronger correlations with cosine similarity, the inner product, and the RBF kernel, the Spearman rank correlations are less stable across these methods.
For ResNets, we observe a significant decline in correlation consistency and strength with the exception of ResNet18. Opposed to this, ViIs display notably higher negative correlation values when using cosine similarity and radial basis function kernels, in contrast to the Pearson correlation results.
Table 4: Correlation between the representational similarity and the output probabilities of multiple images. In total 20k samples of the INlk test set are used (as opposed to 2k in the table the main)
Metric Kernel
Pearson Correlation Inner Product Pl
Spearman's Rank Correlation Inner Product
Cosine Sim.
RBF
Cosine Sim.
RBF
Invariance
PI
P1
PI
Architecture
ResNet18
-0.279
-0.328
-0.264
-0.272
-0.174
-0.197
-0.231
-0.337
-0.239
-0.225
-0.435
-0.476
ResNet50
-0.256
-0.305
-0.249
-0.269
0.028
0.015
-0.032
0.007
-0.046
-0.040
0.128
0.132
ResNet101
-0.235
-0.330
-0.211
-0.274
0.076
0.067
-0.007
-0.053
-0.007
-0.077
0.071
0.068
ConvNexxtV2-Bree
-0.162
-0.126
-0.160
-0.184
0.077
0.050
40.017
0.026
-0.013
-0.045
0.143
0.128
ViI-B/16
-0.058
-0.098
-0.056
-0.031
-0.079
-0.120
-0.013
-0.230
-0.021
0.029
-0.220
-0.313
VIT-L/32
-0.142
-0.189
-0.143
-0.152
-0.131
-0.164
-0.034
-0.276
-0.029
-0.014
-0.335
-0.392
DinoV2-Giant
-0.016
-0.013
-0.015
-0.015
-0.042
F Details of the Approximation Algorithms
The computational complexity of determining optimal matchings using the Jonker-Volgenant algorithm 8 scales significantly with O(s²), resulting in substantial computation time for input-patch sizes with spatial dimensions s = 642. To address this challenge, we propose alternative approximate algorithms with reduced computational complexity. In all our approximations, we take advantage of additional information, specifically the L2-norm ||u]]]2 of each semantic concept vector. We assume that achieving high degree of matching involves pairing vectors with the highest norms and high cosine similarity. This assumption guides our design of more efficient matching algorithms.
26

---

1. Greedy:
The simplest approach we employ is breadth-first matching. We determine the order in which to match v, by considering the L2-norm ||2:112 in descending order. We then match the current is with the best, non-assigned vj based on Ag-- The sorting complexity is (s (og())),) making this the fastest approximate algorithm among those tested.
2. TopK-Greedy:
Recognizing that the TopK norm concept vectors ||ei]2 might have significantly higher impact on the final similarity, we attempt to find the optimal matching for only the highest TopK norm concept vectors v and vy. The remaining lower norm concept vectors are assigned using the Greedy algorithm as described above. The process involves an initial sorting based on the semantic concept vectors' L2-norm, followed by optimal matching with O(k2) complexity for the k TopK values and the greedy matching for the remaining values.
3. Batch-Optimal: If the TopK norm concept vectors do not sufficiently approximate an optimal matching, we apply optimal matching for the remaining concept vectors in batches. To achieve this, we create s//b smaller batches, with semantic concept vectors assigned to batches according to their I.2nnorm. All values within a batch are then optimally matched, leading to matching complexity of (f) O(b*)
Evaluating the various approximations, we observe that the Greedy matching yields suboptimal approximation quality and offers marginal to no improvement over the current same-position assignment. Although we do not present the details of the greedy matching, it is important to highlight that it is guaranteed to be worse or equal to the TopK-Greed We include the Greedy algorithm for matching with a & value of 128, as shown in Fig. 5 completeness as a simple baseline.
Furthermore, it is noteworthy that the matching the largest norm concept vectors is insufficient for a good approximation of the optimal matching. This insight suggests that a substantial portion of the overall similarity is contributed by semantic concept vectors not included in the set of highest norms.
matching demonstrates that exclusively
Lastly, we observe that the Batch-Optimal approximation using a small batch size of 128 samples, provides an approximation with less than 10% error compared to the optimal matching. This result underscores the effectiveness of our batching approach based on the 1.2-norm of the concept vectors. It offers reliable estimate for overall similarity, simplifying the matching process significantly.
F1 Runtime Evaluation
In order to assess algorithm performance across different spatial resolutions, we conducted benchmarking study. For each resolution, we randomly selected 10,000 pairs of samples from a ResNet101 trained on Tiny-Imagee ImageNe.. Affinity matrices (Aij) were pre-computed to facilitate permutation (Pij) calculations. The average time taken per matching was then reported for the same single CPU core, as outlined in Table We observe that the OR-Tools implementation outperforms other alternatives, being four times faster than the lapjy implementation ? However, even this optimal approach requires 1.52 seconds per pair on 64 x 64 resolution. Despite the potential for parallelizing sample-wise matching, optimal algorithms face scalability challenges with larger spatial dimensions S. In contrast, the Batch-Optimal approximation offers compelling balance between computation time and approximation quality. Importantly, its complexity scales linearly with S due to the fixed batch size.
G Semantic RSMs and CKA - Qualitative changes
Building upon the success of Linear/RBi (spatio-semantic) CKA to compare systems, we provide some preliminary qualitative comparisons gauging how CKA comparisons are affected by our differently proposed RSM. Unfortunately, hardly any quantitative benchmark
'There is an error in the current version in the main regarding the square root of s, which will be corrected in a revision. We apologize for any confusion 'Implementation on Github https:/ / /ithttt....ooo wttggtt.comm//eeeeee
www.////////////////o////oeoeeee //appjy
27

---

Table 5: We compare different implementations of the optimal Jonker-Volgenant algorithm S against our linearly scaling Batch-Optimal approximation and no matching. The table presents average run-times per pair (T) and average similarity relative the optimal value for 1,000 randomly chosen image pairs. Utilizing representations of varying sizes from 01 trained on Tiny-ImageNet, the optimal solutions are reported relative to the maximum similarity achieved by the optimal algorithms. The Batch-Optimal approximation demonstrates substantial fraction of optimal matching performance with significantly improved scaling.
Hops
Category
Complexity
Matching Algo
Batch
256
1024
a 4096
[%]]
[%]]
T
[%]
Avos
Russ
OR-Tools 23
100
4.26 ms
100
69.6 ms
100
1.52
Optimal
O(s*)
Scipy 3D
100
5.10
100
148
100
8.75 s
lapiv
100
3.73ms
100
97.5 ms
100
6.12
No Match
O(1)
mp.ding(
68.9
1.40 μr
67.0
2.49 μra
57.9
4.36
128
97.6
2.41 ms
94.5
9.08 ms
92.8
43.4 ms
Approximation
(v/(6) (100)-01 O(I))
Batch-Optimal
100
4.56 ms
16.9
95.9
512
100
4.57 ms
98.6
37.1 ms
96.8
171 ms
1024
100
4.57 ms
79.5 ms
391 ms
exists to quantify if a representational similarity metric is better than another. Previously Kornblith et al. [12] evaluated CKA by showing was better at finding layers of the same architecture than SVCCA and PWCCA. This was extended by Ding et al. 33] through the inclusion of statistical testing but remains a rather shallow benchmark. Subsequently, we constrain ourselves to qualitative experiments, leaving quantitative testing to potential future benchmarks.
In all following experiments, all representations are extracted globally and zero-centered along the sample dimension. Given the zero-centered representations spatio-semantic and semantic RSMs are computed in mini-batches of 250 samples. To calculate the semantic RSMs on CIFAR an optimal bipartite matching algorithm is used, while for Tiny-ImageNet and ImageNet we utilize the Batch-Optimal approximation with window size 512.
G.1 CKA between semantic and spatio-semantic RSMs
As initial inspection, we evaluate how different the similarity structures of a model measured through spatio-semante RSMs are to model measured through semantic RSMs. We do this by constructing both RSMs from the same representations of a model. Subsequently, we compare the two alternative RSMs of the same representations to each other through CKA Eq. (2) The diagonal of this matrix represents a direct comparison of identical representatrons, just with another definition of what is considered "similar". This is evaluated for three ResNet18; and three ResNet34s on Tiny-ImageNet with the linear and RBF kernels respectively.
We display the average CKA matrices across architecture seeds for both architectures, as well as the diagonal values of the CKA matrix. Results are shown in Fig. 15
Examining these CKA matrices multiple observations can be made:
A) Despite the diagonal representing a comparison between identical representations, the CKA values are not This indicates that the different way of constructing RSMs changes the perceived similarity structure of the system, measured by CKA.
B) Inspecting the diagonal shows, that earlier layers with greater spatial extent express higher differences in similarity, whereas layers at a later layer and lower spatial extent are less dissimilar. This is consistent with the expectation that, with shrinking spatial extent, alignment of semantic concepts gets more likely.
Given these large changes in CKA similarity, we conclude that the definition of what a model perceives as similar can highly influence inter-system similarity. This is especially relevant when comparing systems across domains, where RSM construction may be domain-specific, disallowing to be consistent with RSM construction. Exemplary when comparing MI. vision systems to human vision models, particular when comparing representations of high spatial extent.
28

---

RBF Kemel RSMs
Bagask vICCA Midiiss
Linear Kernel FISMs
Deekeett
Degaea OlA
-
Figure 15: The dissimilarity between semantic and spatio-semantic RSMs decreases with shrinking spatial extent. Comparison of semantic and spatio-semartic Representational Similarity Matrices (RSMs) for the same model. The dissimilarity early layers decreases with decreasing spatial extent, as illustrated by Centered Kernel Alignment (CKA) values. The left and middle panel shows the CKA comparison between all layers, while the right panels visualize the heatmap's diagonal, emphasizing the evolving similarity trend from early late layers.
G.2 Differences in CKA self-similarity
In the previous paragraph, semantic RSMs were directly compared to spatio-semantic RSMs. While this can influence measurements when models are compared across domains (e.g. CNN Vision to Biological Vision), it does not need to imply that the CKA similarity of vision models changes substantially.
Given that within the same domain, the RSM construction will likely be chosen consistently, one can opt to either: Calculate only spatio-semantic RSMs or only semantic RSMs, due to personal opinions or preferences. Subsequently, the question is not "Are spatio-semantic RSMs similar to semantic RSMs", but "Does the CKA similarity between spalio-semantic RSMs change when calculating semantic RSMs".
To address this question we: A) Compare the CKA matrix difference of the CKA matrix based semantic RSMs to the CKA matrix of spatio-semantic RSMs when comparing model with itself (Intra Model) and B) when comparing between different models (Cross-Model),
Intra-Model CKA Similarly to before, extract representations and form semantic and spatio-semantic RSMs. extraat representations on CIFAR100 [14] (32 x 32),
(64 x 64) and TmageNet-1k (2) (160 x 160) datasets, from 3 differently seeded and trained from scratch ResNet18, ResNet34 and ResNet101 architectures. The semantic RSMs are calculated utilizing the Batch - Optimal matching with 6 512 for matching on Iiny-ImageNet and ImageNet. We calculate semantic and spatio-semantic RSMs with a mini-batch size of 250, subsequently using them for Canonical Correlation Analysis (CKA) calculations. The corresponding cka matrices and their differences are displayed in Fig. 16 If not further specified the experiment uses the linear kernel.
Introspecting the results it can be seen that across all Architectures ResNet18, ResNet34, and ResNet101 largely the same change in similarity structure be observed. For CIFAR100, the very first layers are perceived as less similar to semantic RSMs than with spatio-semantic RSMs, while the CKA between the middle to later layers is more similar. This structure, though does not remain consistent across datasets: When moving from CIFAR100 to Tiny- ImageNet earlier layers appear to become more similar while intermediate layers become less so. On ImageNetlk CKA on semantic RSMs seem to indicate models are more similar. This trend indicates that the influence of semantic RSMs on spatio-semantic RSMs seems to be largely dataset-dependent. Moreover, the overall maximum change in CKA similarity in these matrices between -0.2 and +0.2 for Tine-mmageNet, indicating a modest change in overall CKA.
Cross-Model CKA Aside from evaluating only CKA similarity of RSMs of the same model we extend to comparing RSMs between models, commonly done when comparing models through CKA. ResNet18/101 models trained on CIFAR100 are used with RSMs constructed identically to previously specified. Results are displayed in Fig. 17
29

---

CIFAR100
Tiny Imagelatt
Imagallet11k
Faechan 101
Figure 16: CKA similarity between different Layers of the same ResNet18, ResNet34, ResNet101, on CIFAR100, Tiny-ImageNee and ImageNet-lk. For each row the left-most CKA matrix displays the spatio-semantic RSMs, the middle represents the semantic RSMs while the right represents the difference in CKA similarity between the two. Blue regions indicate where the We observe that similarity within the block structure is largely unchanged, whereas the similarity across the later blocks seems to be more similar and the similarity of the very first blocks is less similar.
CKASem
CKASpa. Sem.
CKASem CKASpa Sem.
ResNet18
ResNet101
Figure 17: CKA similarity between different Layers of different ResNet18 and ResNet101 models on CIFAR100. We observe decrease in similarity at high-resolution layers, whereas similarity between deeper layers largely unchanged.
It can be seen that for both, ResNet18 and ResNet101, the cross-model CKA similarity is mostly negative for the majority of the layers, indicating that CKA on spatio-semantic RSMs estimates models to be more similar than when applying CKA on semantic RSMs. Similarly to before CKA changes range from -0.175 +0.025 providing modest changes.
Concluding the Intra and Cross-Modes CKA experiments it can be seen that the choice of RSMs results in qualitatively different CKA matrices. Unfortunately, due to the lack of quantitative benchmarks, no direct recommendation of which RSM to use for inter-model similarity calculation through CKA can be given.
30

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer[YYe]
Justification:
The experiments in Section Adetonstaem the advantage of introducing permutation invariance for RSMs and directly correspond to the contributions listed Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and / or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations.
No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine include aspirational goals motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In Section comprehensive discussion of the results and their limitations, such as the runtime, is provided.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are violations of these assumptions (e.g,, independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with a few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech- to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will specifically instructed not penalize honesty concerning limitations.
31

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assump- tions and a complete (and correct) proof?
Answer: [NA]
Justification: While the paper provides a comprehensive mathematical formulation, no new theorems were introduced and thus no theoretical proofs were needed.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be com- plemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/ or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: All experiments were performed using public datasets and models. F. A.pe Algorithm provide
The information provided in Appendix all necessary information to reproduce the results. Additionally, we will publish the code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/ or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e-g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
32

---

(c) If the contribution is new model (e.g., large language model), then there should either be a way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-soures models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Code will be provided by acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips
wwwpoodeSubmssssooooolooooo ssionPolicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submis- sion guidelines for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to underparaa understand the results?
Question:
Answer: [Yes]
Justification:
The information provomm in S all necessary information understand the results. Appendix F. Appendix D.I and Algorithm add additional details that are needed to reproduce the exact experimental setting but not necessarily to understand the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
33

---

7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The paper introduces new concept that has direct related methods to compare with. Therefore, no statistical analysis or ranking methods are needed.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
The presented work does not rely on specific compute requirements. However, in Appendix F.1 we compare the runtime of multiple approximation
algorithms to solve the assignment proble.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e-g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:s////eaiisscccco///ccccccca
Answer:
[Yes]
Justification:
The authors have strictly adhered to the ethical guidelines.
Guidelines:
34

---

The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g,, there is a special consideration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We do not any direct societal impact. The work only provides new introspection into the concept of semantic similarity of networks.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g,, disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there direct path to any negative applications, the authors should point out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technol- ogy is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g,, gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: |NA]
Justification:
See broader impact
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
35

---

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g, code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All relevant and previous work is cited and only open-source assets have been used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from particular source (e-g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced the paper well documented and the documentation provided alongside the assets?
Answer:
[Yes]
Justification:
The codebase will be published under CC BY license.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/ code/ model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
participant
Answer:
[NA]
Justification: No crowdsourcing experiments or research with human subjects were performed.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
36

---

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent aeproval/reviens based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No crowdsourcing experiments or research with human subjects were performed.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS
Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
37


Reviews:
[{"review_id": "gbfxyQ8BC9", "reviewer": "Reviewer_Xdnj", "summary": "This paper makes a contribution to the construction of RSMs in the field of vision neural networks and puts forward the concept of semantic RSMs, which is innovative and theoretical.", "questions": "It is suggested to further elaborate the potential and specific scenarios of the research in practical applications to enhance readers' understanding of its practical value.", "limitations": "see weekness", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The proposed semantic RSMs are used for spatial alignment by means of optimal permutation, which is a relatively new and promising method.\n\nThis paper verifies the validity of semantic RSMs through experiments such as image retrieval and probabilistic similarity comparison. An in-depth analysis of the experimental results is carried out, and the advantages of semantic RSMs in specific tasks are pointed out.", "weaknesses": "This paper lacks the experimental verification of specific downstream tasks, such as detection and segmentation, on semantic RSMs. I need to know which scenario is more suitable for RSMs and semantic RSMs.\n\nLack of quantitative comparative data. It is suggested to add tables or charts to show specific performance comparison data between semantic RSMs and existing methods in different tasks (such as image retrieval, class probability similarity comparison, etc.), including accuracy, time complexity and other indicators.\n\nThe discussion of the experimental results was not thorough enough. It is recommended to add a detailed analysis of the experimental results to explain why semantic RSMs perform better on certain tasks, as well as possible reasons and limitations.\n\n \"aligns\" to \"align\" in line 57.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our sincere gratitude for taking the time to read our paper and provide valuable feedback and constructive criticism:\n\n1. _“This paper lacks the experimental verification of specific downstream tasks, such as detection and segmentation, on semantic RSMs. I need to know which scenario is more suitable for RSMs and semantic RSMs.”_ \n\nWe believe this may be a misunderstanding. Semantic RSMs are not algorithms for detection or segmentation (which we do not claim), but rather they measure the similarity between neural network representations of images. If the concern is about applying our method to dense datasets like those used for object detection or segmentation, our EgoObjects experiment (Fig. 3) demonstrates its application to a dense dataset. Additionally, the results in Table 1 show our method applied to ImageNet.\n\n2. _\"Lack of quantitative comparative data. It is suggested to add tables or charts to show specific performance comparison data between semantic RSMs and existing methods in different tasks (such as image retrieval, class probability similarity comparison, etc.), including accuracy, time complexity and other indicators.\"_ \n\nWe have added an additional retrieval experiment on the Cityscapes dataset, comparing semantic and spatio-semantic RSMs for this use case. Regarding the comparison of semantic RSMs with other retrieval methods, such as those using learned embedding spaces, we disagree that this is necessary. The aim of our paper is to find the best way to measure the similarity between representations of two samples. Parametrized methods that alter representations in non-trivial ways would make such comparisons unfair. We emphasize that the main purpose of our paper is to address the flaws in current RSM construction, which often requires spatial alignment. We demonstrate these flaws through retrieval and class probability similarity comparisons and show that our semantic RSMs alleviate these issues.\n\n3. “It is recommended to add a detailed analysis of the experimental results to explain why semantic RSMs perform better on certain tasks, as well as possible reasons and limitations.“\n\nWe add a clarifying section at the end of the retrieval section, right before 4.3 discussing the retrieval results. We cite:\n>“These experiments display clearly that demanding spatial alignment can be a significant shortcoming when semantically similar concepts are misaligned. In Fig. 4, the network learned to represent the objects very similarly, despite a shift in perspective, but due the same objects not aligning anymore, spatio-semantic similarity fails to recognize this.\nThis effect should generalize to other datasets where objects are not heavily centered. For datasets with heavy object-centric behavior, like ImageNet, this should be less pronounced.”\n\n4. Typos \nWe made sure the entire manuscript has the respective errors fixed.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PKbHW0M8sb", "reviewer": "Reviewer_sf44", "summary": "The authors introduce semantic RSMs, which are designed to be invariant to the spatial arrangement of elements within images. These semantic RSMs assess similarity by treating the problem as one of set-matching, where the focus is on matching semantic content rather than spatial details. This approach not only aligns more closely with human perception but also improves the relevance and accuracy of image retrieval tasks. The paper claims that semantic RSMs offer a more robust measure of similarity by comparing them to traditional spatio-semantic methods.", "questions": "- How well does the method scale to very large datasets or to more complex neural networks that handle highly varied or abstract visual content?\n\n- How does the method perform under noisy conditions or when semantic parsing is imperfect due to occlusions or poor image quality?", "limitations": "- To better evaluate the impact of semantic RSMs, a set of diverse metrics should be established: Evaluate semantic RSMs against traditional spatio-semantic RSMs and other state-of-the-art similarity measures to highlight the improvements or shortcomings.\n\n- The authors should explicitly state the primary objectives of employing semantic RSMs. Show that semantic RSMs can make neural network decisions more interpretable by aligning more closely with how humans perceive images.\n\n- Apply semantic RSMs in specific use cases like medical imaging, satellite image analysis, and autonomous driving where ignoring spatial arrangements can be particularly detrimental or beneficial, providing a nuanced view of their applicability.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The focus on semantic content rather than spatial arrangement aligns more closely with human perception, potentially leading to more intuitive and relevant comparisons of neural network responses.\n\n-  By being invariant to spatial permutations, this method can effectively compare images where the same objects appear in different locations.\n\n- Semantic RSMs can be used as drop-in replacements for traditional RSMs.", "weaknesses": "- Employing algorithms like Hungarian matching to find the optimal permutation matrix can be computationally expensive.\n\n- The effectiveness of this approach relies heavily on accurate identification and parsing of semantic concepts within images, which can be challenging in complex scenes or under conditions of visual ambiguity.\n\n- While focusing on semantic content is generally advantageous, completely ignoring spatial information can sometimes omit useful contextual cues that contribute to overall image understanding. For example, \n\n[contextual cues] a picture of a dining table with plates, utensils, and food arranged in a specific way might convey a meal setting, which could be lost if the spatial relationships are ignored. \n\n[object interactions] Images where interactions between objects are important, such as a cat sitting on a mat, might lose their interpretative meaning if spatial information is disregarded. The semantic content (cat, mat) remains the same, but the relationship changes based on their arrangement.\n\n[abstract content] In abstract art or images with non-literal interpretations, spatial composition itself can carry meaning and affect how the content is perceived and classified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed feedback. We can see that a great deal of time and thought went into it. However, we believe there may be a few misunderstandings that we would like to clarify:\n\n### Questions\n**Q1: How well does the method scale to very large datasets[...]**\n\nR1: The scaling behavior depends on the _spatial extent_ of representations and the _use-case_ in which the method is applied. Details on this can be found in the common response.\n\n**Q2: How well does the method scale [...] to more complex neural networks that handle highly varied or abstract visual content?**\n\nR2: We are not quite sure what you mean by _\"more complex neural networks of varied/abstract visual content\"_. Our experiments already apply the method to various state-of-the-art networks (CLIP, DINOv2, SAM, ResNets, ConvNeXT), which are capable of handling diverse visual content. Additionally, we evaluate our method on standard ImageNet data as well as on EgoObjects, which we believe qualifies as a more complex dataset aimed at robotics.\n\n**Q3: How does the method perform under noisy conditions or when semantic parsing is imperfect due to occlusions or poor image quality?** \n\nR3: This is an interesting question! Poor conditions or visual ambiguity can degrade representations if the network cannot handle them, potentially harming, for example, the retrieval performance of our method. However, **this is not a unique issue of our method but of all methods using learned representations, such as spatio-semantic RSMs**. Even standard cosine similarity on global representations would likely experience similar degradation.\n\n### Limitations\n\n1. _\"Apply semantic RSMs in specific use cases like medical imaging, satellite image analysis, and autonomous driving where ignoring spatial arrangements can be particularly detrimental or beneficial, providing a nuanced view of their applicability.\"_\n\nWe added an additional experiment for autonomous driving using the Cityscapes dataset, demonstrating improvements of semantic RSMs over spatio-semantic RSMs in this context.\n\n2. To better evaluate the impact of semantic RSMs, a set of diverse metrics should be established: Evaluate semantic RSMs against traditional spatio-semantic RSMs and other state-of-the-art similarity measures to highlight the improvements or shortcomings.\n\nIn this paper, we compared the current, most commonly used methods for constructing RSMs, as introduced in the [CKA paper](https://arxiv.org/pdf/1905.00414) by Kornblith et al. (Inner Product and RBF spatio-semantic RSMs), with the addition of cosine similarity. We also introduced two metrics to assess the “goodness” of RSMs, which we believe are sufficient to demonstrate the benefits of our proposed method.\n\n### Weaknessses\n1. **[Runtime of Hungarian matching]**: We clearly and transparently state the runtime as a limitation in our paper and provide some algorithms to alleviate this. Reviewer zPy6 even commends us on this. To put our compute time into perspective: In representation similarity analysis many much more slow to compute metrics exist, e.g. the [IMDScore](https://openreview.net/forum?id=HyebplHYwB) which takes >12 hours for a single layer-to-layer comparison between two representations of [10.000x2048] samples using 32 CPU cores we ran in a different project or the RSMNormDifference which can take multiple hours as well. The former was published in ICLR20. This is somewhat of a duplication to the common response,  but we felt it necessary to reiterate.\n2. **[Visual ambiguity can negatively impact]**: See Quesions R3\n3. **[Ignoring spatial context can be bad]:** We really appreciated these thought experiments as they go into a lot of detail on how representations work. \n   1. **[Contextual cues]**: Networks progressively combine/merge information to build semantic concepts. E.g. Ears, eyes and whiskers in a neighbourhood, could lead to expression of a cat's head and so on. Similarly we expect a network to learn associations between dishes and cutlery on a table to a meal setting. If the network learns this association, breaking the spatial location with our method is not an issue, as the semantic concept vector will not only carry `<plate>` but `<plate, part_of_meal_table>` information somewhere. \n    2. **[abstract content]**: Distance and composition can carry meaning, but if the network learns to represent them similarly, this can be beneficial. For example, if you take a picture of an abstract painting, move back 5 meters, and take another picture, the spatial composition shifts despite the image being identical. Similarly, moving sideways changes the angle. If the network still represents the same semantic meaning despite these shifts, it can still be recognized.\n    3. **[object interactions]** **(Very related to context cues)** We believe that networks merge semantic information hierarchically within a neighborhood, forming increasingly abstract semantic concepts. Due to this spatial proximity, representations of a cat on a mat are encoded similarly, leading to more frequent retrievals of cats on mats, even when spatial constraints are ignored. Evidence for this is provided by CLIP models, which use global aggregation of their outputs—disregarding spatial relations—to generate final embeddings. This method effectively identifies related images and captions, such as “a cat sitting on a mat,” and we assume this approach is applicable to our setting as well.\n\nWe hope that our responses have clarified the open questions and if we answered the questions sufficiently, it would be great if you could consider increasing your rating.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hHZT2eJs5z", "reviewer": "Reviewer_xk1Y", "summary": "This paper proposes Semantic RSMs to understand the internal representations in deep neural networks. The authors argue that the current RSMs are limited by their coupling of semantic and spatial information, which restricts the assessment of similarity. The proposed semantic RSMs are spatial permutation invariant and focus solely on semantic similarity. The proposed method is shown to enhance retrieval performance and provide a more accurate reflection of the predictive behavior of classifiers.", "questions": "I'm not an expert in this field, so I tend to start by looking at what other reviewers think of the paper.", "limitations": "Yes, the authors have addressed the limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-written and easy to follow.\n2. The introduction of semantic RSMs is a significant contribution, potentially leading to more meaningful comparisons between neural network models.\n3. The empirical demonstration of improved retrieval performance using semantic RSMs is convincing and adds practical value to the theoretical development.", "weaknesses": "1. While the paper does highlight the high computational complexity as a limitation, it would benefit from a more detailed discussion on the scalability of the proposed method to larger models and datasets and the approximation error.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the time and effort you invested in reviewing our manuscript.\n\nRegarding your comment that _\"it would benefit from a more detailed discussion on the scalability of the proposed method to larger models and datasets and the approximation error.\"_\n\nWe addressed a part of this regarding scaling to larger datasets in the general response already. Regarding the question of scaling to larger models, the scaling of our method is just dependent on representation size, while model size plays no role at all. \nWe add section to discuss the approximation error at the end of the Section 4.4: We cite: _\"The fastest of the Batch-Optimal approximation methods shows $<8\\%$% error while improving run-time $\\times 36$ relative to the fastest optimal algorithm for spatial extent $4096$, while no spatial alignment shows $42\\%$% deviation [\"or approximation error\"] from the optimal matching.\"_\n\nWe hope that this helps to improve clarity on the approximation error quality of the approximations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6DQ75fZQIm", "reviewer": "Reviewer_zPy6", "summary": "In this paper, the authors propose a new method to measure similarity between responses of deep neural networks in vision. They reformulate the commonly used strategy to compute Representational Similarity Matrices (RSMs) by acknowledging the superiority of the semantic information over the spatio-semantic information in the creation of RSMs. The authors perform different experiments to show the improvement over the baseline method caused by their reformulation.", "questions": "- Why do the authors use Pearson correlation to examine the relationship between the Jensen-Shannon Divergence and the representational similarity? E.g. Kendall/Spearman correlation can be more robust.\n- The authors should focus on the differences between the results obtained for CNNs and ViTs (see the comment in the weaknesses section). \n- The statement in the introduction “we argue that the spatial location of semantic objects does neither influence human perception nor deep learning classifiers.” is a little bit too bold - the paper does not examine human perception, therefore it would be better to leave only the deep learning here. \n- Also, a minor thing is that some typos, grammatical and formatting errors can be found in the paper (e.g. the sentence starting in l79, l205: a RSM -> an RSM, l25: SAMor CLIPSeg ,the retrieval performance)\n- The authors could provide more examples of their method for different networks to enable their better qualitative assessment which is now limited (e.g. in the appendix).", "limitations": "The authors adequately discussed the biggest limitation of their work (the computational cost of their method). They could also better highlight the limited data used in the experiments caused by the mentioned computational constraints.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is clearly written and tackles an important topic.\n- The idea is original and besides its limitation connected to high computational needs, it could serve as an inspiration for future works.\n- Although the experiments are not exhaustive, they are convincing and coherent and the gained insights seem relevant.\n- Being transparent with the limitation of the method and trying to provide the means to mitigate it is a plus.", "weaknesses": "- The authors should better discuss the differences between the results obtained for ViTs and CNNs in their study, which are quite well visible. E.g. while in the case of an examined CNN, the spatio-semantic RSM does not reflect well the similarities between translated images, in the case of the examined ViT (appendix), these similarities can be observed. The other thing is that the experiment is slightly different, because in the experiment with CNNs much smaller images are used than in the ViT experiment. Also, the differences are also visible in Table 1 (ResNets obtain much higher absolute correlation values for the baseline and the proposed methods than ViTs). \n- The authors provided few visual examples of the results of their method. It would be good to provide more of them (e.g. for different similarity metrics used, for more images and for more netorks) to enable more comprehensive qualitative evaluation (they could be placed in the appendix). \n- The use of some methods at work is not well justified (e.g. Pearson correlation).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> _Why do the authors use Pearson correlation to examine the relationship between the Jensen-Shannon Divergence and the representational similarity? E.g. Kendall/Spearman correlation can be more robust._ and _\"The use of some methods at work is not well justified (e.g. Pearson correlation).\"_\n\nWe have added a brief explanation in Section 4.3 regarding our choice of Pearson correlation: \"We chose to use the pearson correlation, as it allows observing a direct linear behavior between representational similarity and predictive similarity.\" IWe believe this is particularly appropriate for cosine similarity and the radial basis function, which are bounded. However, we also provide Spearman correlation results in the appendix, along with a discussion. The corresponding table can also be found in the rebuttal PDF.\nOverall, we observe that Spearman correlation is mostly lower, with large ResNets being impacted strongly while ResNet-18 is notably consistent and stable in its correlations. ViTs interestingly show a much higher correlation for cosine and radial basis function similarities compared to Pearson correlations.\n\n>The authors should focus on the differences between the results obtained for CNNs and ViTs (see the comment in the weaknesses section).\n\nWe extended our discussion on CNNs vs ViT's in the appendix, going in depth into the differences of the two translation experiments provided. \n\n> The statement in the introduction “we argue that the spatial location of semantic objects does neither influence human perception nor deep learning classifiers.” is a little bit too bold - the paper does not examine human perception, therefore it would be better to leave only the deep learning here.\n\nWe cut the statement short to “we argue that the spatial location of semantic objects does not influence deep learning classifiers.”\n\n> The authors could provide more examples of their method for different networks to enable their better qualitative assessment which is now limited (e.g. in the appendix).\n\nAs displayed in the rebuttal pdf, we will add plenty qualitative visualizations with a discussion in the appendix for many models. Additionally, as we added the Cityscapes experiment, we will also provide qualitative results on this dataset.\n\n>Also, a minor thing is that some typos, grammatical and formatting errors can be found in the paper (e.g. the sentence starting in l79, l205: a RSM -> an RSM, l25: SAMor CLIPSeg ,the retrieval performance)\n\nWe want to apologize for this oversight, we have re-proofread our manuscript and pushed it through a spell-checker to fix any potential typos and grammatical errors.\n\n> They could also better highlight the limited data used in the experiments caused by the mentioned computational constraints.\n\nInstead of highlighting this, we increased our overall sample size, as mentioned in the main response. We hope this to be a sufficient adaptation.\n\nFind the similarity table with spearman's rank correlation below (N=20.000).\n\n| Metric          | pearson_corr |        |               |        |         |        | spearman_rank |        |               |        |         |        |\n|-----------------|-------------:|-------:|--------------:|-------:|--------:|-------:|--------------:|-------:|--------------:|-------:|--------:|-------:|\n| Kernel          |   cosine_sim |        | inner_product |        | rbf |        |    cosine_sim |        | inner_product |        | rbf |        |\n| Invariance      |            - |     PI |             - |     PI |       - |     PI |             - |     PI |             - |     PI |       - |     PI |\n| Architecture    |              |        |               |        |         |        |               |        |               |        |         |        |\n| ResNet18        |       -0.279 | -0.328 |        -0.264 | -0.272 |  -0.174 | -0.197 |        -0.231 | -0.337 |        -0.239 | -0.225 |  -0.435 | -0.476 |\n| ResNet50        |       -0.256 | -0.305 |        -0.249 | -0.269 |   0.028 |  0.015 |        -0.032 |  0.007 |        -0.046 | -0.040 |   0.128 |  0.132 |\n| ResNet101       |       -0.235 | -0.330 |        -0.211 | -0.274 |   0.076 |  0.067 |        -0.007 | -0.053 |        -0.007 | -0.077 |   0.071 |  0.068 |\n| ConvNextV2-Base |       -0.162 | -0.126 |        -0.160 | -0.184 |   0.077 |  0.050 |        -0.017 |  0.026 |        -0.013 | -0.045 |   0.143 |  0.128 |\n| ViT-B/16        |       -0.058 | -0.098 |        -0.056 | -0.031 |  -0.079 | -0.120 |        -0.013 | -0.230 |        -0.021 |  0.029 |  -0.220 | -0.313 |\n| ViT-L/32        |       -0.142 | -0.189 |        -0.143 | -0.152 |  -0.131 | -0.164 |        -0.034 | -0.276 |        -0.029 | -0.014 |  -0.335 | -0.392 |\n| DinoV2-Giant    |       -0.016 | -0.046 |        -0.016 | -0.030 |  -0.013 | -0.052 |        -0.014 | -0.037 |        -0.015 | -0.022 |  -0.015 | -0.042 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thank you to all the reviewers for their time and effort in reviewing our paper. We appreciate the feedback and tried to mitigate issues to the best of our abilities. We recognize different viewpoints, but some criticisms seem based on misunderstandings, which may have led to undeservedly lower ratings. We hope to clarify and address these issues in this rebuttal.

### Added content:
While not relevant for all reviewers, we want to highlight the content added during the rebuttal as transparently as possible: 
1. We conducted an additional experiment comparing semantic RSMs and spatio-semantic RSMs for retrieval using the Cityscapes dataset for autonomous driving. Results are provided in the rebuttal PDF. Our findings demonstrate that semantic RSMs are preferable to spatio-semantic RSMs for retrieval in this setting. More details are included in the figure caption, and we will offer a more comprehensive discussion in the appendix. Qualitative examples are also included but could not be fit on the single page of the rebuttal PDF.
2. We increased the sample size of our correlation experiment in Table 1 from 2,000 to 20,000 samples. Results are provided in the rebuttal PDF. Additionally, we present further quantitative results from our EgoObjects retrieval experiment, as shown in main Figure 4. We include results for 2,500, 5,000, and 10,000 database samples in table format.
3. Finally, we include additional qualitative retrieval results for various models. In the rebuttal PDF, we provide visualizations from CLIPSeg, in a downscaled excerpt.
4. Aside from this we edited small paragraphs of text, where criticised by the Reviewers.
### Common question: How well does the method scale to large datasets?
As multiple reviewers had questions about scaling behavior, we want to use this space to discuss scaling in some more depth, and put this into perspective to currently exiting similarity methods common in the representational similarity analysis space.

The scaling behavior of our poposed method depends on mostly two factors
1. Spatial Extent and Use-Case:
    1. Spatial Extent: Smaller spatial extents result in less overhead during matching. For example, ViT models, such as ViT-L/16, have a spatial extent of 257 (with 1 class token) and can be compared essentially in the same way as traditional spatio-semantic RSMs. For CNNs, the situation varies with depth. Early layers, such as those in ResNet architectures, have a large spatial extent due to downsampling (e.g., a 224x224 image downsampled 4x results in a spatial extent of 3136), making early CNN layers significantly more expensive—by a factor of 12.25. In contrast, later layers with a smaller spatial extent show more favorable scalability for CNNs.
    2.	Use-Case: When comparing neural networks using RSMs, batched calculations are common. This approach reduces computational burden and memory constraints, making semantic RSMs scale effectively for this purpose, even with larger datasets. However, for retrieval, the scaling is more costly, as each query must be compared to the entire database. Therefore, retrieval scales linearly with the size of the database being queried.
   
We already provide very specific runtimes in the Appendix Table 2 for our proposed approximations and the relative optimal algorithms. Given these numbers and a specific datasets and use-case in question one could estimate expected runtimes in a single threaded fashion.
To aid clarity we will add the discussion above to this section in the appendix.
Aside from runtime alone, one needs to take into account that the _dense_ embeddings of the image need to be stored for each model and image as well, which can have a substantial storage demand when e.g. trying to do retrieval on ImageNet1k.

With the runtime being such a focal point though, **we want to emphasize that the runtime of many representational similarity measures are not optimal**. E.g. there exist measures that compute much more slowly than ours, e.g. the [IMDScore](https://openreview.net/forum?id=HyebplHYwB) which takes >12 hours for a single layer-to-layer comparison between two representations of [10.000x2048] samples using 32 CPU cores, which we used in a different project or the RSMNormDifference which can take multiple hours as well. With the IMDScore having been published in ICLR20 we hope we can convince you that runtime limitations are not a disqualifying factor for such methods.

Meta-review: This paper deals with extending the concept of Representational Similarity Matrices (RSM) to semantic RSM, by measuring semantic similarity between input responses via solving a set-matching problem, in order to achieve permutation invariance.  The idea is validated on image retrieval and probabilistic similarity comparisons.

It has received 2x weak accepts, 1x borderline accept, and 1x borderline reject.  Reviewers have major concerns on lacking quantitative comparative data, lacking thorough discussions and analysis of experimental results, computational costs with the utilization of Hungarian algorithms, conceptual inadequacy of semantics disregarding spatial configuration, applicability under noisy semantic content detection and in complex scenarios with multiple objects of the same class, lacking more visual examples and questionable metrics etc.  The authors have provided extensive rebuttal with additional experimental results, which reviewers find mostly helpful and render an overall acceptance consensus.

Representation similarity is an ambiguous concept.   The reality lies between global feature similarity and set-matched local feature similarity for individual images.  Nevertheless, the AC recommends acceptance based on its different perspective, even at one extreme of permutation invariant matching.   Please revise the paper per reviewers' comments and additional materials in the rebuttal, strengthening the position of the paper with more convincing results and discussions.   Thank you.

~~~~~~~~~~~~~~~~~~~~~~~~
Here are extra comments from SAC.
(1) Please remove the period at the end of title. In addition, check the usage of punctuation in Line 31-33.
(2) Please use the official template. The current one suffers from the issue of paragraph spacing. 
(3) It would be nice to adjust orphan lines.
(4) Please coordinate the font sizes in figures, where some fonts are too large in Figure 1 and some fonts are too small in Figure 2.
Decision: Accept (poster)