ReSi: A Comprehensive Benchmark for Representational Similarity Measures

## Abstract

Abstract Measuring the similarity of different representations of neural architectures is a fundamental task and an open research challenge for the machine learning community.
This paper presents the first comprehensive benchmark for evaluating representational similarity measures based on well-defined groundings of similarity.
The representational similarity ( ReSi ) benchmark consists of (i) six carefully designed tests for similarity measures, (ii) 23 similarity measures, (iii) eleven neural network architectures, and (iv) six datasets, spanning over the graph, language, and vision domains.
The benchmark opens up several important avenues of research on representational similarity that enable novel explorations and applications of neural architectures.
We demonstrate the utility of the ReSi benchmark by conducting experiments on various neural network architectures, real world datasets and similarity measures.
All components of the benchmark are publicly available 1 1 1 https://github.com/mklabunde/resi and thereby facilitate systematic reproduction and production of research results.
The benchmark is extensible, future research can build on and further expand it.
We believe that the ReSi benchmark can serve as a sound platform catalyzing future research that aims to systematically evaluate existing and explore novel ways of comparing representations of neural architectures.

## 1 Introduction

Representations are fundamental concepts of deep learning, which have garnered significant interest due to their ability to cast light on the opaque inner workings of neural networks.
Studying and analyzing them has enabled insights into numerous problems, e.g., understanding learning dynamics [, ], catastrophic forgetting [], and language changes over time [].
Such analyses commonly involve measuring similarity of representations, which resulted in a plethora of similarity measures proposed in the literature [, ].
However, these similarity measures have often been proposed in an ad-hoc manner, without comprehensive comparison to already existing similarity measures.
Moreover, they have often been proposed in conjunction with new quality criteria that were deemed desirable, with previously defined quality criteria being ignored.
So far, only the few most popular measures have been compared [, ] or analyzed in more detail [, , ].

In this work, we present the first comprehensive benchmark for representational similarity measures.
It comprises six tests that postulate different ground truth assumptions about the similarities between representations that measures could capture.
We implement these tests over several architectures and datasets in the graph, language, and vision domains.
The ReSi benchmark enables tests for 23 similarity measures that have been proposed in literature and illustrate how the results can provide insights into the properties and strengths and weaknesses of these measures, as well as guide researchers and practitioners in choosing an appropriate measure for an application at hand.
All benchmark code and the corresponding models are openly accessible online.

## 2 Grounding Representational Similarity

Figure: Figure 1: Grounding similarity. In all tests within the ReSi benchmark, we design a set of models for which we can establish a ground-truth about the similarity of their representations. The left heatmap illustrates the true similarity between a set of models, the other heatmaps the similarity values that different similarity measures assign to each model pair via their representations. We rank similarity measures by their ability to capture the ground truth. In practice, a ground-truth similarity between models is usually hard to attain. For the ReSi benchmark, we design tests where similarity is practically grounded.
Refer to caption: x1.png

Before presenting the ReSi benchmark, we briefly introduce key terms and notations for representational similarity, and discuss how ground-truths for representational similarity can be established.

### 2.1 Representational Similarity

The ReSi benchmark is designed to evaluate the quality of measures that aim to quantify the similarity of neural representations.
Such representations can be derived from applying a neural network model

$$ $f=f^{(L)}\circ f^{(L-1)}\circ\dots\circ f^{(1)},$ (1) $$

where each function $f^{(l)}:\mathbb{R}^{D^{\prime}}\longrightarrow\mathbb{R}^{D}$ denotes a single layer,
on a matrix of $N$ inputs $\bm{X}\in\mathbb{R}^{N\times p}$ – for simplicity, we assume each input to be in vector format even though, as in the vision domain, these can also be multidimensional matrices.
Then, at any layer $l$ one can extract the model’s *representation*

$$ $\bm{R}:=\bm{R}^{(l)}=\left(f^{(l)}\circ f^{(l-1)}\circ\dots\circ f^{(1)}\right )(\bm{X})\in\mathbb{R}^{N\times D}.$ (2) $$

Unless noted otherwise, we always consider representations from the final inner layer of a neural network.
We define a *representational similarity measure* as a mapping $m:\mathbb{R}^{N\times D}\times\mathbb{R}^{N\times D^{\prime}}\longrightarrow
\mathbb{R}$ that assigns a similarity score $m(\bm{R},\bm{R^{\prime}})$ to a pair of representations $\bm{R},\bm{R^{\prime}}$.
For brevity, throughout this work we will often simply denote these measures as *similarity measures*.

### 2.2 Grounding Similarity

All representational similarity measures quantify representational similarity with a single scalar, but the opinions when two representations should be similar diverge:
For example, similarity can be related to comparing the pairwise distances in a representation [], the ability to directly align two representations [, ], or their topology [].

While these motivations are usually theoretically connected to differences in models, in practice, differences in models are not necessarily captured by these approaches.
However, these differences, e.g., in their behavior, are often of interest because they cannot be easily measured.
Similarity measures present a potential shortcut to identifying these differences.
Complicating this is that model difference is multi-faceted:
One can distinguish models by their behavior like accuracy, robustness to augmentations or domain shifts, and their preference on texture or shapes, or by aspects like differences in training data or their human-likeness to name a few examples.
This multi-dimensionality leads to a crucial problem in measuring representational similarity, namely the lack of a general ground-truth that measures should reflect.

Due to this plurality in model behavior, a number of possible targets to ground representational similarity have been proposed in literature.
For our purposes, we focus on the following two broad approaches to establish a ground truth for similarity.
1. Grounding by Prediction.
A straightforward way to obtain a ground-truth for model similarity is to consider the differences in the predictive behaviors of a pair of models: when a two models yield different predictions, they should also differ in their representations.
This approach allows one to ignore where the source of difference between the models originates from, but simultaneously implies that one cannot be certain whether ground truth similarities stems from differences in the classifier or the representations.
Previous efforts that have used this way to ground similarity include the study by Ding et al. [], who correlated representational similarity with accuracy difference, or the work by Barannikov et al. [], who ground similarity to differences in individual predictions.
2. Grounding by Design.
Through careful design, one can construct groups of representations for which one can impose a ground truth about similarity by relation.
For instance, one can demand that representations from the same group should be more similar to each other than representations from different groups.
One example for such a design is displayed by Kornblith et al. [], who trained multiple models of the same architecture, and formed groups of representations based on the depth of their producing layer.
Then, they demanded that representations of the same layer should be more similar to each other than to representations from other layers.
These relative comparisons have the advantage, that no deeper insights about the often opaque and non-linear similarity measures are needed.
However, the validity of the evaluation hinges on the validity of the assumption that representations actually follow the expected groups, which requires clear justification.

## 3 ReSi: A Benchmark for Representational Similarity Measures

We present the main components of the ReSi benchmark. It consists of (i) six carefully designed tests for similarity measures, (ii) 23 similarity measures, (iii) eleven neural network architectures, and (iv) six datasets, spanning over graph, language, and vision domains.
Further, we discuss how the results for our benchmark are evaluated.

**Table 1: *Overview of the tests included in the ReSi benchmark.* We design 6 tests that define different ground truths for representational similarity.**
| Test | Intuition | Grounding Type |
| --- | --- | --- |
| Correlation to Accuracy Difference | Can variation in performance of classifiers trained on representations be captured? | Prediction |
| Correlation to Output Difference | Can variation in individual predictions of classifiers trained on representations be captured? | Prediction |
| Label Randomization | Can models that were trained on different labels be distinguished? | Design |
| Shortcut Features | Can models that rely on different features be distinguished? | Design |
| Augmentation | Can models with different levels of training augmentation be distinguished? | Design |
| Layer Monotonicity | Does layer similarity decrease with increased layer distance within a model? | Design |

### 3.1 Tests

We provide descriptions of six tests implemented within the ReSi benchmark.
We begin with discussing two tests which are grounded by predictions, and then describe four tests which are grounded by design.
An overview of all tests is given in Table 1.

Test 1: Correlation to Accuracy Difference.
When two models are trained under similar conditions but yield differences in their performance, this can be seen as a signal that the underlying representations are different.
Following this intuition, we correlate the representational similarity of a model pair with the absolute difference in their accuracies, thus similarity is *grounded by predictions*.
Training Protocol.
We roughly follow the protocol established by Ding et al. [], for which ten models are trained on each dataset, varying only by the training seeds.
Afterward, we compute the accuracy of the models on the test set.

Test 2: Correlation to Output Difference.
This test follows the same intuition as Test 1, with the difference that it focuses on the individual output differences of each model rather than the aggregated accuracy of models.
Given that models with similar accuracy may still yield very different predictions on individual instances, this provides a more fine-grained signal which can serve as a more robust grounding for the similarity of the underlying representations.
Thus, similarity is again *grounded in predictions.*
Training Protocol.
Due to the similarities to Test 1, we use the same models for this experiment.

Test 3: Label Randomization.
In this experiment, we train models on the same input data, but with labels randomized to different degrees.
The expectation is that models that learn to predict the true labels learn different representations compared to the models that learn to memorize random labels.
These differently trained models are grouped by their degree of label randomization during training. We evaluate if similarity within a group is higher than between groups.
Therefore, representational similarity is *grounded by design* of these groups.
Training Protocol.
For each domain, we create at least two groups of models.
We always train one group on fully correct labels and one group on fully random labels.
Additional groups have partially randomized labels (25%, 50%, 75%).
Across all domains, we always trained five models per group for each architecture.

Test 4: Shortcut Affinity.
The ability to identify whether two models use similar or different features can be a desirable property for a similarity measure.
Hence, we create a scenario, in which we control feature usage by introducing artificial shortcut features to the training data.
Specific features correspond to each label, thus leaking them, but not necessarily perfectly—we can add incorrect shortcut features for some instances.
Groups of models are then formed based on the degree to which the shortcut features in the training data match with the true labels. This way, the additional features can be useful or irrelevant for classification, resulting in some models learning to use them and others learning to ignore it.
We can verify the degree to which models rely on the shortcuts by evaluating their performance on data with uncorrelated shortcut features.
Similar to Test 3, we evaluate if similarities within the same groups are higher than between groups.
That way, representational similarity is *grounded by design* of these groups.
Training Protocol.
For each domain, we construct one group of models trained on data with shortcuts that always leak the correct label, one group trained on data with completely random shortcut features, and additional groups in which the labels were leaked on a fixed percentage of training labels.
Representations are then extracted on a fixed test set, on which random shortcuts were introduced.
Each group consists of five models, in which training seed affected the model and the shortcuts.

Test 5: Augmentation.
Augmentation is commonly used to "teach" models to become invariant to changes in the input domain which do not affect the labels.
In this test, we evaluate if the similarity measures are able to capture robustness to such changes, by developing models with varying amounts of augmentation.
Similarly to before, we train groups of models with different degrees of augmentation.
Models of the same group should yield more similar representations than models from other groups trained on differently augmented data – again, similarity is *grounded by design* of the models.
Training Protocol.
In all domains, we trained one group of reference models on standard data, and additional groups of models with varying degree of augmentation on the training data.
Each group consists of five models, which only vary in their training seed.
Representations are computed on the standard, non-augmented test data.

Test 6: Layer Monotonicity.
As noted in Section 2.1, the individual layers $f^{(l)}$ of a neural network $f$ all yield representations $\bm{R}^{(l)}$ that can be compared.
Given that these layers also represent a sequence of transformations of the input data $\bm{X}$, i.e., $\bm{R}^{(l)}=f^{(l)}(\bm{R}^{(l-1)})$, it seems intuitive that representations of neighboring layers should be more similar than representations of layers that are further away, given the greater amount of transformations between the further-apart layers.
Thus, in this experiment, we extract the layer-wise representations of a neural network and test whether a representational similarity measure can distinguish pairs of layers based on their distance to each other.
Similarity of representations is thereby *grounded by design*.
Training Protocol.
We reuse the trained models of Test 1 and Test 2, as no training scenario changes were made – only for graph neural networks, we increased the number of layers to five inner layers to enable a sufficient number of comparisons.

### 3.2 Representational Similarity Measures

ReSi evaluates 23 different similarity measures. A brief overview is provided in Table 2.
We directly used the reference implementations of the measures where possible.
Otherwise, we closely followed the given definitions and recommendations, also in the preprocessing of representations.

Hyperparameters.
For all neighborhood-based measures (Jaccard, RankSim, 2nd-cos), we set the neighborhood size to $k=10$ and determined nearest neighbors based on cosine similarity.
For CKA, we chose the commonly used linear kernel.
For RSA, we applied Pearson correlation as inner and Spearman correlation as outer similarity function, and we computed the RSM in RSMDiff based on Euclidean distance.
In GULP, we set the regularization weight to $\lambda=0$.
Finally, for the IMD score, we use 8,000 approximation steps and five repetitions.
All other measures did not provide any hyperparameters.

**Table 2: Similarity measures included in the ReSi *benchmark*. For each measure, we present their abbreviations as used in our plots and tables, a reference, and the type of measure as specified in the survey by Klabunde et al. []. RSM abbreviates *Representational Similarity Matrix*, i.e., a matrix of pairwise similarities between the instances. CCA denotes *Canonical Correlation Analysis.***
| Abbreviation | Measure | Reference | Measure Type | Preprocessing |
| --- | --- | --- | --- | --- |
| 2nd-Cos | Second Order Cosine Similarity | [] | Neighbors |  |
| AlignCos | Aligned Cosine Similarity | [] | Alignment |  |
| AngShape | Orthogonal Angular Shape Metric | [] | Alignment | Centered columns and unit matrix norm |
| CKA | Centered Kernel Alignment | [] | RSM | Centered columns |
| ConcDiff | Concentricity Difference | [] | Statistic |  |
| DistCorr | Distance Correlation | [] | RSM |  |
| EOS | Eigenspace Overlap Score | [] | RSM |  |
| GULP | GULP | [] | RSM | Centered rows and row norm to $\sqrt{N}$ |
| HardCorr | Hard Correlation Match | [] | Alignment |  |
| IMD | IMD Score | [] | Topology |  |
| Jaccard | Jaccard Similarity | [, ] | Neighbors |  |
| LinReg | Linear Regression | [] | Alignment | Centered columns |
| MagDiff | Magnitude Difference | [] | Statistic |  |
| OrthProc | Orthogonal Procrustes | [] | Alignment | Centered columns and unit matrix norm |
| PWCCA | Permutation-Weighted CCA | [] | CCA |  |
| PermProc | Permutation Procrustes | [] | Alignment |  |
| ProcDist | Procrustes Size-and-Shape-Distance | [] | Alignment | Centered columns |
| RSA | Representational Similarity Analysis | [] | RSM |  |
| RSMDiff | RSM Norm Difference | [, ] | RSM |  |
| RankSim | Rank Similarity | [] | Neighbors |  |
| SVCCA | Singular Value CCA | [] | CCA |  |
| SoftCorr | Soft Correlation Match | [] | Alignment |  |
| UnifDiff | Uniformity Difference | [] | Statistic |  |

### 3.3 Models

Overall, the ReSi benchmark utilizes a range of various graph, language, and vision models.
Details on parameter choices can be found in Appendix A.
All trained models are publicly available.
Graphs.
As graph neural network architectures, we chose the classic *GCN* [], *GraphSAGE* [], and *GAT* [] models, using their respective implementation from PyTorch Geometric [].
For each experiment and dataset, we trained these models from scratch.
Language.
We focus on the popular *BERT* architecture [].
Sellam et al. [] provide 25 BERT models trained with different seeds.
In our experiments, we fine-tuned these models on the given datasets to avoid computationally expensive pretraining.
Vision.
For the ReSi benchmark, we focused on prominent classification architecture families, namely *ResNets* [], *ViT’s* [] and older *VGG’s* [].
To capture the effect of scaling architecture sizes, we included
ResNet18, ResNet34, ResNet101, VGG11, VGG19, ViT-B/32 and the ViT-L/32 architectures.
All models were trained from scratch on the (possibly augmented) dataset, apart from the ViTs which were initialized with pre-trained weights from IN21k.

### 3.4 Datasets

We provide a brief overview of the datasets used within the ReSi benchmark.
More detailed descriptions of the datasets can be found in Appendix A.1.1.
Graphs.
We focus on graph datasets that provide multiclass labels for node classification, and for which dataset splits into training, validation and test sets are already available.
Specifically, we select the *Cora* [], *Flickr* [], and *OGBN-Arxiv* [] datasets.
For the Cora graph, we extract representations for the complete test set of 1,000 instances, whereas for Flickr and OGBN-Arxiv, we subsampled the test set to 10,000 instances for computational reasons.
Language.
We use two classification datasets: SST2 [] is a collection of sentences extracted from movie reviews labeled with positive or negative sentiment.
MNLI [] is a dataset of premise-hypothesis pairs labeled with the relation between premise and hypothesis.
We use the validation and validation-matched subsets to extract representations for SST2 and MNLI, respectively.
Vision.
For the vision domain, we use the ImageNet100 dataset.
This dataset is derived from Imagenet1k [] by subsampling 100 randomly chosen classes.
This reduces total sample size by 90%, reducing overall training time, while keeping image resolution and content similar to the prominent ImageNet1k dataset.
For all models, image resolution was fixed to $224\times 224$.

### 3.5 Evaluation

Lastly, we describe how we evaluate and quantify the performance of representational similarity measures within the ReSi benchmark tests.
Due to the different nature of the two approaches we use to ground representational similarity, we present the corresponding evaluation approaches separately.

##### Grounding by Prediction.

When grounding similarity of representations to the predictions of their corresponding classification models $f$, we specifically consider the outputs $\bm{O}:=f(\bm{X})\in\mathbb{R}^{N\times C}$,
where we assume that each row $\bm{O}_{i}=f(\bm{X}_{i})\in\mathbb{R}^{C}$ consists of the instance-wise class probability scores for $C$ given classes.

In Test 1, we leverage these outputs to correlate representational similarity to absolute differences in *accuracy*.
Thus, letting $q_{\text{acc}}(\bm{O}):=q_{\text{acc}}(\bm{O},\bm{y})$ denote the accuracy of an output $\bm{O}$ with respect to ground-truth labels $\bm{y}\in\mathbb{R}^{N}$, we compute the absolute difference in accuracies

$$ $\Delta_{\text{acc}}(\bm{O},\bm{O^{\prime}})=|q_{\text{acc}}(\bm{O})-q_{\text{ acc}}(\bm{O^{\prime}})|.$ (3) $$

Then, given a similarity measure $m$, and letting $\mathcal{F}$ denote the set of models trained for this test, for all model pairs $f,f^{\prime}\in\mathcal{F}$ we collect the representations similarity scores $m(\bm{R},\bm{R^{\prime}})$ as well as the accuracy differences $\Delta_{\text{acc}}(\bm{O},\bm{O^{\prime}})$, and report the *Spearman correlation* between these sets of values.

In Test 2, we more specifically consider differences in the instance-wise predictions $\bm{O}_{i},\bm{O^{\prime}}_{i}$ rather than differences in aggregate performance scores.
Toward that end, for all pairs of models $f,f^{\prime}\in\mathcal{F}$, we both compute the disagreement

$$ $\Delta_{\operatorname{Dis}}(\bm{O},\bm{O^{\prime}})=\tfrac{1}{N}\textstyle\sum _{i=1}^{N}\mathbbm{1}\{\arg\max_{j}\bm{O}_{i,j}\neq\arg\max_{j}\bm{O^{\prime}} _{i,j}\},$ (4) $$

between their hard predictions, and the average *Jensen-Shannon divergence (JSD)*

$$ $\Delta_{\text{JSD}}(\bm{O},\bm{O^{\prime}})=\tfrac{1}{2N}\textstyle\sum_{i=1}^ {N}\operatorname{JSD}(\bm{O}_{i}\|\bm{O}_{i}^{\prime})$ (5) $$

of the class-wise probability scores, and report the *Spearman correlation* of both measures with the corresponding set of representational similarities $m(\bm{R},\bm{R^{\prime}})$.

##### Grounding by Design.

For all the tests of this category, we do not consider functional outputs $\bm{O}$ of individual models as a ground-truth for representational similarity anymore.
Instead, we have created multiple groups of representations $\mathcal{G}$, such as groups of representations from models that were trained with the same degree of label randomization as in Test 3, through which we impose a ground truth about similarity by relation.
In Test 3, 4 and 5, we postulate that for any similarity measure $m$ it should hold that representations $\bm{R},\bm{R^{\prime}}\in\mathcal{G}$ from the same group should be more similar to each other than representations $\bm{R}\in\mathcal{G},\bm{R}^{*}\in\mathcal{G}^{*}$ from different groups, i.e.

$$ $m(\bm{R},\bm{R}^{*})\leq m(\bm{R},\bm{R^{\prime}}),$ (6) $$

where we assume that for $m$, higher values indicate more similarity.
Then, one trivial way to evaluate the performance of a similarity measure $m$ is be to count the number of times inequality (6) holds over all combinations of groups and representations, and then report the relative *conformity rate* over all comparisons.
Given that this measure can, however, be biased by the ratio of inter-group vs. intra-group pairs, we additionally adapt the *Area under the Precision-Recall curve (AUPRC)* measure to our context.
This is done by assigning a label $y(\bm{R},\bm{R^{\prime}}):=\mathbbm{1}\{\mathcal{G}=\mathcal{G}^{\prime}\}$
to each pair of representations $\bm{R}\in\mathcal{G},\bm{R^{\prime}}\in\mathcal{G}^{\prime}$ that can be compared within the given set of groups, and then interpreting the representational similarities $m(\bm{R},\bm{R^{\prime}})$ as decision scores based on which one should be able to "predict" the label $y(\bm{R},\bm{R^{\prime}})$.

Finally, in Test 6, we group representations $\bm{R}^{(l)}$ by the layers $l$ they were extracted from.
However, in contrast to the previous tests, we postulate that an ordinal relationship between the layers has to hold.
Specifically, given a model with $L$ layers, for all tuples $1\leq i\leq j<k\leq l\leq L$ we require a measure $m$ to satisfy

$$ $m(\bm{R}^{(i)},\bm{R}^{(l)})\leq m(\bm{R}^{(j)},\bm{R}^{(k)}),$ (7) $$

assuming that for $m$, higher values indicate more similarity.
As in the previous tests, we report the corresponding *conformity rate*.
Further, we report the *Spearman correlation* between the similarities $m(\bm{R}^{(i)},\bm{R}^{(j)})$ and the distance $j-i$ of the corresponding layers over all tuples $1\leq i<j\leq L$.

Figure: Figure 2: *Aggregated ranks of measures across all models and tests separated by domain.* Lower is better. Measures are ordered by their median rank. No measure stands out across all domains.
Refer to caption: x2.png

## 4 Benchmark Results

In the following, we illustrate selected benchmark results. Due to limited space, we focus on presenting (a) an aggregated overview and (b) an exemplary detailed result for a single dataset and a single architecture.

For an aggregated result overview, we rank all measures for each combination of test, dataset, and architecture. The rank distribution across tests in each domain is shown in Figure 2.
We observe that no measure outperforms the others consistently across the domains, with most measures having high variance in their rankings.

However, one can identify some domain-specific trends, such as the neighborhood-based second-order cosine similarity (2nd-cos), Jaccard similarity, or rank similarity measures performing well in the graph domain. On the other hand, angle-based measures performing well on language models.
Further, the popular CKA measure appears to perform particularly well in the vision domain.
Orthogonal Procrustes, which has overall performed well in the analysis by Ding et al. [], also appears to consistently rank among the top third of measures.
Conversely, other prominent measures such as RSA or SVCCA do not appear to stand out in general.

The ReSi benchmark can also provide more detailed results for analysis.
To provide some examples, Table 3 presents the outcomes for a single dataset and architecture in each domain.
All values are denoted such that higher values correspond to better adherence to similarity groundings.

Again, we observe that even for a single dataset there is no single measure that outperforms the others in all tests.
Instead, most measures appear to perform well in some tasks but do not in others.
For instance, measures such as the Eigenspace overlap score or GULP appear to be able to perfectly distinguish layers of a GraphSAGE model, but appear incapable to identify whether they have learned specific shortcuts, which most other measures appear to be capable of.
One can also identify model-specific trends, such as SVCCA correlating particularly well with predictions from BERT models, but not as much with other domains.
In addition, considering the performance of descriptive statistics-based measures such as uniformity difference or concentricity difference may provide additional geometric insights on representations. For instance, the uniformity of BERT representations appears to inform relatively well about the degree to which it was trained on augmented data.

**Table 3: Exemplary results for selected datasets and models. We show GraphSage on flickr for graphs, BERT on MNLI for language, and ResNet18 on ImageNet100 for vision. Evaluation measures are AUPRC for tests grounded by design and Spearman correlation for the tests grounded by prediction. Our multi-dimensional benchmark highlights that similarity measures have different strengths and weaknesses. For example, 2nd-Cos (first row) captures differences in augmentation well, but correlates only weakly with differences in output probabilities as measured by JSD. Further, PWCCA is instable and often does not return values.**
| Type | Grounding by Prediction | Grounding by Design |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Test | Acc Correlation | JSD Correlation | Random Labels | Shortcut Affinity | Augmentation | Layer Monotonicity |  |  |  |  |  |  |  |  |  |  |  |  |
| Modality | Graph | Lang. | Vision | Graph | Lang. | Vision | Graph | Lang. | Vision | Graph | Lang. | Vision | Graph | Lang. | Vision | Graph | Lang. | Vision |
| 2nd-Cos | -0.09 | -0.26 | -0.08 | 0.15 | 0.16 | -0.13 | 0.42 | 0.39 | 0.71 | 1.00 | 0.59 | 1.00 | 0.92 | 0.44 | 0.78 | 0.96 | 0.94 | 0.55 |
| AlignCos | 0.39 | 0.25 | -0.08 | 0.44 | 0.37 | 0.08 | 0.42 | 1.00 | 0.72 | 1.00 | 0.58 | 1.00 | 0.70 | 0.35 | 0.71 | 0.68 | 1.00 | 0.52 |
| AngShape | 0.43 | 0.28 | 0.21 | 0.63 | 0.26 | 0.24 | 0.43 | 0.90 | 0.72 | 1.00 | 0.60 | 1.00 | 0.76 | 0.38 | 0.71 | 0.98 | 1.00 | 0.55 |
| CKA | 0.37 | 0.18 | 0.36 | 0.58 | 0.30 | 0.30 | 0.66 | 0.75 | 1.00 | 1.00 | 0.59 | 1.00 | 0.75 | 0.48 | 0.90 | 0.89 | 0.99 | 0.87 |
| ConcDiff | -0.06 | -0.00 | -0.11 | -0.03 | 0.02 | -0.29 | 0.57 | 1.00 | 0.99 | 0.18 | 0.38 | 0.53 | 0.35 | 0.28 | 0.43 | -0.27 | 0.99 | -0.78 |
| DistCorr | 0.36 | 0.15 | 0.31 | 0.43 | 0.39 | 0.26 | 0.43 | 0.75 | 1.00 | 1.00 | 0.58 | 1.00 | 0.79 | 0.45 | 0.83 | 0.99 | 0.97 | 0.97 |
| EOS | 0.22 | 0.03 | 0.05 | 0.38 | 0.36 | 0.09 | 0.42 | 0.62 | 0.84 | 0.43 | 0.57 | 1.00 | 0.53 | 0.27 | 0.93 | 1.00 | 0.99 | 0.88 |
| GULP | 0.22 | -0.01 | 0.02 | 0.38 | 0.35 | 0.07 | 0.42 | 0.61 | 0.89 | 0.43 | 0.56 | 1.00 | 0.54 | 0.26 | 0.92 | 1.00 | 0.51 | 0.53 |
| HardCorr | 0.32 | 0.04 | 0.21 | 0.50 | -0.27 | 0.28 | 0.46 | 0.75 | 0.72 | 1.00 | 0.55 | 0.97 | 0.72 | 0.24 | 0.46 | 0.80 | 1.00 | 0.01 |
| IMD | 0.37 | -0.26 | 0.16 | 0.31 | -0.39 | -0.10 | 0.34 | 0.73 | 1.00 | 0.97 | 0.53 | 0.67 | 0.59 | 0.43 | 0.56 | 0.82 | 0.44 | -0.00 |
| Jaccard | 0.07 | -0.21 | -0.11 | 0.42 | 0.13 | 0.36 | 0.43 | 0.69 | 1.00 | 0.83 | 0.56 | 1.00 | 0.88 | 0.35 | 0.79 | 0.97 | 0.96 | 0.55 |
| LinReg | 0.28 | 0.18 | 0.18 | 0.48 | 0.28 | 0.21 | 0.45 | 0.57 | nan | 0.61 | 0.46 | 0.99 | 0.81 | 0.24 | 0.94 | 1.00 | 0.66 | 0.55 |
| MagDiff | -0.01 | 0.22 | -0.16 | 0.06 | 0.01 | -0.38 | 0.72 | 0.33 | 1.00 | 0.78 | 0.48 | 0.37 | 0.17 | 0.16 | 0.37 | 0.50 | 0.52 | -0.37 |
| OrthProc | 0.43 | 0.28 | 0.21 | 0.63 | 0.26 | 0.24 | 0.43 | 0.90 | 0.72 | 1.00 | 0.60 | 1.00 | 0.76 | 0.38 | 0.71 | 0.98 | 0.99 | 0.55 |
| PWCCA | nan | 0.01 | -0.02 | nan | 0.22 | 0.13 | nan | 0.65 | nan | nan | 0.56 | 0.99 | nan | 0.18 | 0.90 | nan | 1.00 | 0.11 |
| PermProc | -0.04 | 0.09 | 0.07 | -0.10 | -0.06 | 0.18 | 0.90 | 0.44 | 0.70 | 1.00 | 0.52 | 0.72 | 0.69 | 0.18 | 0.41 | 0.68 | 0.73 | 0.20 |
| ProcDist | 0.01 | 0.28 | 0.08 | -0.18 | 0.07 | 0.10 | 0.62 | 0.98 | 0.72 | 1.00 | 0.54 | 1.00 | 0.81 | 0.32 | 0.58 | 1.00 | 0.92 | 0.55 |
| RSA | 0.30 | 0.00 | 0.06 | 0.63 | 0.27 | 0.12 | 0.42 | 0.46 | 0.75 | 1.00 | 0.58 | 1.00 | 0.72 | 0.49 | 0.98 | 0.99 | 1.00 | 0.97 |
| RSMDiff | -0.16 | 0.30 | 0.09 | -0.04 | -0.18 | -0.41 | 0.92 | 0.52 | 1.00 | 0.92 | 0.28 | 0.57 | 0.93 | 0.35 | 0.45 | 0.65 | 0.84 | -0.33 |
| RankSim | 0.14 | -0.09 | 0.07 | 0.30 | 0.08 | -0.15 | 0.43 | 0.63 | 1.00 | 0.77 | 0.58 | 0.99 | 0.88 | 0.33 | 0.71 | 0.97 | 0.90 | 0.55 |
| SVCCA | -0.12 | 0.32 | 0.29 | 0.23 | 0.47 | 0.21 | 0.80 | 0.69 | 1.00 | 0.93 | 0.42 | 0.55 | 0.67 | 0.44 | 0.40 | 0.44 | 0.91 | 0.20 |
| SoftCorr | 0.31 | 0.11 | 0.27 | 0.53 | -0.23 | 0.45 | 0.45 | 0.75 | 0.72 | 1.00 | 0.55 | 0.97 | 0.58 | 0.28 | 0.45 | 0.89 | 0.99 | 0.11 |
| UnifDiff | 0.01 | 0.14 | -0.18 | 0.02 | -0.02 | -0.34 | 0.90 | 0.76 | 0.42 | 0.50 | 0.40 | 0.75 | 0.24 | 0.61 | 0.17 | 0.89 | 0.81 | 0.18 |

## 5 Related Work

Despite the large amount of representational similarity measures that have been proposed in the literature [], only few efforts have focused on comparatively analyzing existing measures.
In the following, we briefly provide an overview of existing work.
Grounding by Prediction.
Ding et al. [] first proposed grounding similarity of representations to performance differences of probes trained on them.
They benchmarked three similarity measures with tests focusing on identifying effects of seed variation or manually manipulating the representations.
Compared to ReSi, they focus only on a single test in our framework.
Their work was extended by Hayne et al. [] with a focus on measuring similarity of high-dimensional representations in CNNs.
Similar to Test 2 in ReSi, Barannikov et al. [] tested the ability of their similarity measure to capture prediction disagreement.
Grounding by Design.
Similar to Tests 3-5 in the ReSi benchmark, a number of previous works have also tested whether similarity measures could distinguish representations based on differences in their underlying models.
Boix-Adserà et al. [] considered representations of models from different architectures, and required similarity measures to distinguish representations by model family.
Tang et al. [] tested whether measures could distinguish models based on differences in the data they were trained on.
In a setting similar to Test 6, Kornblith et al. [] grouped representations of models that only differ in their training seed by layer depth, and tested whether measures assign more similarity to representations from corresponding layers.
Finally, a number of analyses have tested whether similarity measures can detect model-independent changes in representations.
Specifically, it has been explored whether measures could distinguish representations based on different synthetic clusters [], based on different levels of noise [, ], or based on subsampled dimensions [].

## 6 Discussion and Conclusions

In this paper, we have presented ReSi, the first comprehensive benchmark for representational similarity measures. By applying its six tests on a wide range of 23 similarity measures to evaluate their capabilities on graph, language, and vision representations, we have demonstrated that it can yield rich insights both for researchers who may be looking for theoretical understanding of representational similarity and for practitioners that are looking for an appropriate measure for a problem at hand.
We provide all code publicly online, and invite the machine learning community to use it to test (potentially new) measures, or to extend it to include even more models or even additional tests.

We have put considerable effort into properly grounding the benchmark tests.
Specifically, when *grounding by design*, we have not only carefully set up distinct training conditions, but also verified from the validation performance of the models under consideration that these indeed follow the intended behavior and adapted the generation process if necessary.
Still, we cannot fully control what models learn. Even if two models behave identically, their internal processes, and thereby their representations, could still differ, potentially confounding the resulting test scores.
When *grounding by predictions*, we have deliberately restricted the tests to models that only vary in their training seed, to reduce the potential for additional confounders as much as possible.
Furthermore, in our tests we solely focus on last-layer representations.
Checking the validity of the existing tests at other layers could be done by evaluating whether probes follow the same behavioral patterns as the full model.

While ReSi implements 23 similarity measures, we do not claim to include all existing measures. Additionally, we did not explore the effects of different preprocessing approaches or parameter choices of individual measures. Given that such factors can lead to different results [, ], ReSi opens up the way for a systematic evaluation of such steps in future work.
Similarly, the six tests implemented in ReSi should be considered a comprehensive but extensible foundation of tests that users can build upon.
We see a lot of potential in augmenting ReSi with additional tests, aiming for a unifying test suite that can inform the whole community of researchers and practitioners.
Finally, we have observed vast differences in the run time of the implemented similarity measures, which can limit large scale comparisons.
Thus, one could additionally consider computational performance of similarity measures as an interesting criterion for future evaluations of similarity measures.

## Acknowledgements

This work is supported by the Deutsche Forschungsgemeinschaft (DFG, German Research
Foundation under Grant No.: 196649072.
The authors acknowledge support by the state of Baden-Württemberg through bwHPC
and DFG through grant INST 35/1597-1 FUGG.

## Appendix A Experiment Details

Complementing the higher level experiment descriptions in the main manuscript, a more detailed description of them is provided in this section. This includes general information about chosen model architectures, datasets and hyperparameters for each domain, with additional details that are unique to each experiments provided later.

### A.1 General Experiment Details

Across all modalities and experiments, efforts were made to keep architectures, training hyperparameters or dataset choices as static as possible, while minimizing overall compute effort needed to compare representations with all these metrics. The settings unique to each domain are reported in their dedicated following sections, while shared choices are explained here:

For all tests except Test 6 (layer monotonicity), we only utilize the representations of the last inner layer, directly before the linear layers of the classifier for our comparisons.
We chose to do so as we can verify our assumptions of a model learning shortcuts or learning to be more robust to augmentations with the predictions directly derived from these representations.
In an earlier layer, for example, we would not be able to verify whether a model learned the shortcut or not.
For the output correlation, a similar argument can be made, as predictions are directly derived from these representations.
Only for the layer monotonicity test, intermediate representations were used.

Figure: Figure 3: Training, validation, and test accuracy of all graph neural network models on all datasets with the standard parametrization.
Refer to caption: x5.png

#### A.1.1 Graph Settings

##### Models and Parametrization.

As graph neural network architectures, we chose the *GCN* [], *GraphSAGE* [], and *GAT* [] models due to their widespread prominence.
In our implementation, we used the respective model classes as provided in the PyTorch Geometric [] package.

For each experiment and dataset, we trained these models from scratch.
Regarding the hyperparameters, we roughly followed values that were used for these models in existing benchmarks.
An overview of the hyperparameter choices for each algorithm and dataset are presented in Table 4.
The layer count also includes the classification layer, thus the number of inner layers is one less than the number given in the table.
For the GAT model, we always used $h=8$ attention heads.
We always used the *Adam* optimizer [] as implemented in PyTorch [] to optimize a cross-entropy loss on the given multiclass datasets.
To validate the performance, we plot the training, validation and test accuracies of the final models, in Figure 3.

All these parameter choices were consistently applied in all our benchmark tests, except for Test 6 (layer monotonicity), where we increased the number of layers to six, yielding five inner layers.

##### Datasets.

We focus on graph datasets that provide multiclass labels for node classification, and for which dataset splits into training, validation and test sets are already available.
Thus, we selected

- 1.
the citation network *Cora*, for which nodes represent documents and edges represent citation links,
- 2.
the *Flickr* [] network, where nodes represent one images uploaded to Flickr, and edges are formed if two images share some common properties,
- 3.
the *OGBN-Arxiv* citation network between Computer Science (CS) arXiv papers. Each node is an arXiv paper, and each directed edge indicates that one paper cites another one.

Specifics on the datasets can be found in Table 5.

**Table 4: Hyperparameters for all architectures on the respective dataset in the graph domain.**
| Dataset | Architecture | Dimension | Layers | Activation | Dropout Rate | Learning Rate | Weight Decay | Epochs |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | GCN | 64 | 3 | relu | 0.5 | 0.01 | 0.0 | 200 |
| Cora | GraphSAGE | 64 | 3 | relu | 0.5 | 0.005 | 0.0005 | 200 |
|  | GAT | 64 | 2 | elu | 0.6 | 0.005 | 0.0005 | 200 |
|  | GCN | 256 | 3 | relu | 0.5 | 0.01 | 0.0 | 200 |
| Flickr | GraphSAGE | 256 | 3 | relu | 0.5 | 0.01 | 0.0 | 200 |
|  | GAT | 256 | 3 | relu | 0.5 | 0.01 | 0.0 | 200 |
|  | GCN | 256 | 3 | relu | 0.5 | 0.01 | 0.0 | 500 |
| OGBN-Arxiv | GraphSAGE | 256 | 3 | relu | 0.5 | 0.005 | 0.0005 | 500 |
|  | GAT | 256 | 3 | relu | 0.5 | 0.01 | 0.0 | 500 |

**Table 5: Statistics of the graph datasets used within the ReSi benchmark**
| Dataset | Nodes | Edges | Features | Classes |
| --- | --- | --- | --- | --- |
| Cora | 2,708 | 10,556 | 1,433 | 7 |
| Flickr | 89,250 | 899,756 | 500 | 7 |
| OGBN-Arxiv | 169,343 | 1,166,243 | 128 | 40 |

#### A.1.2 Language Settings.

##### Models.

We focus on BERT models [] as Sellam et al. [] provide 25 reproductions of the original models with different seeds.
Using these models, we avoid training BERT models, which is not feasible for us.
Other models like LLMs would be interesting as well and can be explored in the future.

##### Representation Extraction.

BERT produces one representation per token of each input.
As we focus on the last-layer representations, we use the representation of the CLS token as the representation for the whole input.
As this token is used by the final classifier, we argue that all relevant information for that input will be contained in this representation.

While it is possible to extract representations for all tokens, inputs with many tokens would take up more rows in the final representation matrix, which could bias similarity estimates towards similarity of long inputs.
Additionally, the runtime of many representational similarity measures scale quadratically in the number of rows of the representations, which limits the size of the representations given the many comparisons that we run in our benchmark.

##### Datasets.

We use two classification datasets: SST2 [] is a collection of sentences extracted from movie reviews labeled with positive or negative sentiment.
MNLI [] is a dataset of premise-hypothesis pairs.
They are labeled according to whether the hypothesis follows from the premise.
We use the validation and validation-matched subsets to extract representations for SST2 and MNLI, respectively.

##### Training Hyperparameters.

As we use the pretrained models from Sellam et al. [], we only finetune the models for our experiments.
While the number of epochs varies between 3 and 10 depending on the experiment, we always use the checkpoint with best validation performance.
We generally use a linear learning rate schedule with 10% warmup to a maximum of 5e-5, evaluate every 1000 steps, and use batch size of 64.
Otherwise, we use default hyperparameters of the transformers library(^2^22[https://huggingface.co/docs/transformers/v4.40.2/en/main_classes/trainer#transformers.TrainingArguments](https://huggingface.co/docs/transformers/v4.40.2/en/main_classes/trainer#transformers.TrainingArguments)).

#### A.1.3 Vision Settings

##### Architecture Choices.

In the vision domain a plethora of architectures exist which could have been used in the scope of the benchmark.
To narrow the scope and to keep computational overhead manageable only architectures for classification were considered. Moreover a subset of architecture of prominent classification architecture families were evaluated, namely ResNets [], ViT’s [] and older VGG’s [].
To capture if architecture size influences the measures, different sizes for the architectures were evaluated, resulting in
1. ResNet18,
2. ResNet34,
3. ResNet101,
4. VGG19,
5. ViT-B/32and
6. ViT-L/32
as the final architecture choice. Pre-trained checkpoint for ViT-B/32 and ViT-L/32 use are taken from huggingface provided under the Apache 2.0. Specific checkpoints are ’google/vit-base-patch32-224-in21k’ and ’google/vit-large-patch32-224-in21k’.

##### Representation extraction.

In order to measure representational similarity, representations need to be extracted at different positions in the architecture.
In order to capture the full system behavior of a model, representations are only extracted at positions where no skip-connections exist.
For VGG networks this would not narrow down choices due to lack of skip connections, yet for ResNets and ViT’s this narrows the possibility of choices where representations can be extracted.

##### Representation Extraction.

Across all experiments representations are extracted on unseen test images that models have not seen during training.
For ImageNet100 the 50 validation cases per class are used, resulting in $N=5k$ samples.

##### Training Hyperparameters.

The ReSi benchmark requires models that can be differentiated by their training setting.
Hence, it is mandatory to either train models from scratch or to fine-tune them accordingly.
For all models of the vision domain the choice was made to train the models from scratch for each respective dataset, with the exception of ViT’s whose encoders were initialized from a pre-trained IN21k checkpoint. The chosen training hyperparameters are kept static across Tests, with different data being the only difference. A selection of hyperparameters is displayed in Table 6. While the parameter choices worked well for ResNets and VGGs, the ViT performance was lower than the ResNets, despite attempts to optimize their training settings. We assume this may originate from the fewer overall steps taken due to the $10\times$ lower dataset size relative to full ImageNet1k.

**Table 6: Vision domain: Training Hyperparameters for all architectures on the ImageNet100 dataset. Aside from the listed parameters in the table, we denote that all models shared more hyperparameters, namely label smoothing $0.1$ and a cosine annealing learning rate schedule.**
| Dataset | Architecture | Batch Size | Learning Rate | Weight Decay | Optimizer | Epochs |
| --- | --- | --- | --- | --- | --- | --- |
|  | ResNet18 | 128 | 0.1 | 4e-5 | SGD | 200 |
|  | ResNet34 | 128 | 0.1 | 4e-5 | SGD | 200 |
|  | ResNet101 | 128 | 0.1 | 4e-5 | SGD | 200 |
| ImageNet100 | VGG11 | 128 | 0.1 | 4e-5 | SGD | 200 |
|  | VGG19 | 128 | 0.1 | 4e-5 | SGD | 200 |
|  | ViT-B32 | 512 | 3e-3 | 0.1 | AdamW | 300 |
|  | ViT-L32 | 512 | 3e-3 | 0.1 | AdamW | 300 |

#### A.1.4 Compute Resources

To conduct the experiments, a broad spectrum of hardware was used: For the model training GPU nodes with up to 80GB VRAM were employed. Depending on domain, representations were either extracted on GPU nodes and saved to disk for later processing, or extracted on demand on CPU nodes. Lastly, the representational similarity measures were calculated between representations on CPU nodes with 6-256 CPU cores and working memory between 80 and 1024 GB.

To extend the current tests with additional similarity measures, practitioners need to have CPU compute resources to compute their own similarity measures on the existing model’s representations.
To introduce novel tests, practitioners would need to provide GPU compute resources for network training and subsequent CPU resources to evaluate all similarity measures on the new models.

### A.2 Test 3: Label Randomization - Details

Figure: Figure 4: Test accuracy of all graph neural network models on all datasets, resulting from models trained on labels that have been randomized to varying degrees.
Refer to caption: x6.png

##### Graphs.

We trained groups of five models each, for which $r\in\{25\%,50\%,75\%,100\%\}$ of all labels are randomized during training.
The models in each group were initialized with different training seeds, and different seeds also imply different randomization of labels.
Given that we exclusively consider predictions on datasets with $L\geq 7$ classes, for all labels that were drawn to be randomized, we randomly drew a new label from the set of all existing labels, excluding the actual ground-truth label, with uniform probability.
After observing the performance of the models with randomized labels on validation and test data, we decided to only include models for which $r\in\{25\%,100\%\}$ were altered during training for this test, along with five models trained on the regular data.
THis is mainly due to the models already having shown behavior indicating that for higher degrees of randomization than 25%, there was hardly any relationship left between features and true labels that the models would have learned.
We illustrate the performance of the models trained under varying degrees of labels in Figure 4, where we can see that the models are strongly affected by such randomization.

##### Language.

We create datasets where $r\in\{25\%,50\%,75\%,100\%\}$ of all labels are randomized so they can only be learned by memorization.
If an instance should get a randomized label, we pick uniformly at random from a new set of 5 labels that are entirely unrelated to the data.
After validating behavior, we use models without randomization as well 75% and 100%.

##### Vision.

We create datasets where $r\in\{25\%,50\%,75\%,100\%\}$ of all labels in the training set are randomized, forcing models to memorize. Whenever a label is chosen to be randomized, we randomly sample from all 100 existing labels, and replace the true label with the sampled one. Moreover we re-use the first 5 models from Test 1 and 2 representing a datasets with $r=0$.
After validating behavior, we use models with 0%, 50%, 100%, with clusters visualized in fig. 5.

Figure: Figure 5: Training with varying degrees of random labels results in models of highly different accuracies. We highlight the remaining 3 clusters used in the vision Label Randomization test.
Refer to caption: x7.png

### A.3 Test 4: Shortcut Affinity - Details

##### Graphs.

For all graph datasets, shortcuts are added by appending a one-hot encoding of the corresponding labels to the node features.
Nodes which are not assigned a true level shortcut obtain a one-hot encoding of a random label instead.
We trained models with a ratio of $\rho\in\{0\%,25\%,50\%,75\%,100\%\}$ true shortcuts in the training data, but only considered the models trained on with $\rho\in\{0\%,50\%,100\%\}$ shortcuts in the training data, as otherwise there would not have been a very clear separation of the models.
We illustrate the performance of models influenced by the remaining degrees of shortcut likelihood in the training data in Figure 6, where we can see that most models are indeed affected. In particular, GraphSAGE appears very sensitive to shortcuts, whereas other methods appear more robust - this may also be due to differences in the way node information is updated in the convolutional layers.
Overall, on Cora the effect was the smallest, which is likely due to the models fitting easily to the training labels even without a shortcut.

For all groups and seeds, we extracted representations based on the same fixed dataset, which contained one-hot encodings of random labels.

Figure: Figure 6: Test accuracy of all graph neural network models on all datasets, resulting from models trained on data with different amounts of shortcuts that leak the true label.
Refer to caption: x8.png

Figure: Figure 7: Validation accuracy of BERT models used in tests. Groups are clearly separated.
Refer to caption: x9.png

##### Language.

We add shortcuts to the data by prepending a new special token to each sentence.
The embedding matrix of BERT is extended with one token per class and trained during finetuning.
We create datasets with 5 levels of shortcut strength, i.e., the rate at which the shortcut leaks the correct label.
As completely incorrect shortcut tokens would perfectly leak the label in binary problems like SST2, the lowest rate is the relative frequency of the majority class.
Then we make 4 equally sized steps up to perfect shortcuts.
This means that SST2 datasets have shortcut rates $r\in\{55.8\%,66.8\%,77.9\%,88.9\%,100\%\}$ and for MNLI $r\in\{35.4\%,67.7\%,51.55\%,83.85\%,100\%\}$.
We train models with 5 different seeds for each shortcut dataset, but after validating sufficiently different behavior (see Figure 7) only use the models with the lowest and the two highest rates.

##### Vision.

In the vision domain, shortcuts are created through insertion of colored dot.
Each dot has a static color, a diameter of $5$ and is placed randomly in the image. To turn this into a shortcut, each class is assigned a unique color code, which is uniformly drawn once and shared across all experiments. By varying how often the pre-determined color appears on an image containing the corresponding class label, one can control how useful the color dot feature is to the class in the image. A few exemplary images of the shortcuts are shown in fig. 8.

Figure: Figure 8: Visualization of the shortcuts introduced in the Vision domain. We statically assign each class label a color code representing the class. How often the actual color label can be controlled, making the shortcut more or less useful. In the top row, the cases where color dots are placed randomly are displayed, while at the bottom the color labels of the dot always correspond to the class label.
Refer to caption: extracted/5768575/figs/vision/shortcut/Dot_corr_0_IN100_image_0.png

In our experiments, we train models under 5 degrees of correlation
$\rho\in\{0\%,25\%,50\%,75\%,100\%\}$, providing a more or less potent shortcut. To validate that these different ratios lead to sufficiently different behavior that our grouping assumption holds, we measure the test set accuracy of all models with $r=0\%$. Models that learned to utilize the shortcut will show a stronger decrease in accuracy as the shortcut does not provide any meaningful signal anymore.
In order to provide clearly distinguishable clusters, we discarded models trained with $r\in\{25\%,50\%\}$.
The accuracy clusters after removing the two groups are shown in fig. 9.

Figure: Figure 9: Exemplary visualization of the validation through accuracy inspection on the example of Test 3: Shortcut Affinity in the vision domain. Models trained with highly correlated shortcuts show lower accuracy when the correlation of the shortcut feature with the image label is removed. By evaluating all trained models under this setting, we remove groups that are not well separated from others.
Refer to caption: x10.png

### A.4 Test 5: Augmentation - Details

Figure: Figure 10: Test accuracy of all graph neural network models on all datasets, resulting from models trained on data with different amounts of shortcuts that leak the true label.
Refer to caption: x11.png

##### Graphs.

To augment the graph data during training, we applied the *DropEdge* approach [], which, at each training epoch, samples a fixed fraction of edges that are removed from the network for this specific epoch.
Based on this approach, we trained four groups of augmented models, for which $r\in\{20\%,40\%,60\%,80\%\}$ of all edges were removed at each epoch.
Again, after considering the performance of the models on validation and test data, we chose to only include the models with
$r\in\{20\%,80\%\}$ for this test along with a group of models trained with standard parameters, which was due to the overall weak effect impact that the intermediate steps appeared to have.
The performance of these remaining models is shown in Figure 10, where we overall observe weak, but visible effect from augmentation.

##### Language.

BERT is trained on large-scale data of varying quality and, as preliminary experiments showed, highly robust to augmentation like word casing, synonyms, and typos(^3^33We tested most of the augmentations from [https://langtest.org/docs/pages/tests/robustness](https://langtest.org/docs/pages/tests/robustness).).
Therefore, we chose more aggressive augmentation that can also scale to large amounts of data.
We randomly replace words with synonyms, delete and swap words, and randomly insert a synonym of a word in the sentence in a random position, as described by Wei and Zou [] and implemented in the textattack library(^4^44[https://textattack.readthedocs.io/](https://textattack.readthedocs.io/)) [].
To create datasets with different level of augmentation, we augment $r\in\{0\%,25\%,50\%,75\%,100\%\}$ of all words per sentence.
We train 5 models with different seeds per dataset.
For SST2, we use the 0% and 100% models; for MNLI, we use 0%, 25%, and 100%.

##### Vision.

For the augmentation test we utilize increasing levels of additive gaussian noise with 5 different noise variance limits $var\in\{(0,0),(0,1000),(0,2000),(0,3000),(0,4000)\}$, we refer to these levels as Off, S, M, L and Max. Exemplary images of the augmented images are displayed in fig. 11. Aside from the noise, only RandomResizeCrop and HorizontalFlip augmentations are used.
After validation model behavior we use Max, S and M, see fig. 12 for the resulting clusters.

Figure: Figure 11: Visualization of the additive gaussian noise used in the vision augmentation test. We display an examplary image highlighting the gradually increasing level of noise introduced. From left to right (Off, S, M, L, Max).
Refer to caption: extracted/5768575/figs/vision/aug/0_off_image_15.png

Figure: Figure 12: Accuracy clusters of the Test 5: Augmentation in the Vision domain. Models trained with varying degrees of additive gaussian noise show lower accuracy on noise-free samples. Two groups were removed to improve separation.
Refer to caption: x12.png

### A.5 Test 6: Layer Monotonicity - Details

##### Graphs.

Since for the graph neural network models, we never used more than three layers in the previous experiments, for this test we trained a new set of five graph neural network models, each with five inner layers.
That way, we can obtain a sufficient number of comparisons, while still preserving decent performance—with higher number of layers, the performance began to strongly deteriorate.
As all inner layer shared the same number of channels, ne preprocessing was necessary to compare the layer-wise representations.

##### NLP.

We use ten models trained on standard data with varying seeds for each dataset.
We use the CLS token representations across all 12 transformer layers and the embedding layer.

##### Vision

For the layer monotonicity test, we use the 10 models from Test 1 and 2. Differently to those tests we extract representations not only before the classifier, but we also need to extract representations in intermediate layers.

For CNN architecture representations will need to be compared that may be of different spatial extent.
This is problematic for some metrics that use the spatial extent to increase their sample size $N$, as they assume features at the same spatial position correspond to each other. Moreover it is problematic, as for width and height $(w,h)$ of $(112,112)$ this increases the total number of values by $>10.000$. To avoid memory issues, we only extract representations from layers of the architecture with global stride $\geq 8$ resulting in $(w,h)$ $(28,28)$ and overall we limit the number of layers to the last 6.
The representations of these layers are then downscaled to a final $7\times 7$ through AveragePooling, resulting in the same spatial extent for all representations, with the exception of the classifier layer without spatial extent.
When comparing representations of spatial extent, we rearrange $w$ and $h$ into the channel dimension $c$ – (’n c h w’ -> ’n (c h w)’) – unless the metric either, requires $n>p$ or, if computation time of the measure scales prohibitively. For these measures we move it into the sample dimension – (’n c h w’ -> ’n (c h w)’), but reduce the sample dimension $(nhw)->(n)$ by only using every 49th value due to memory and time constraints.
For comparisons between non-spatial and spatial representations, we employ average pooling, removing any spatial extent.

For ViT’s we follow the setup of the language domain and only use the classifier token, offering a simple and compute efficient solution.

We denote, that this resampling and sub-sampling of the representations in the CNN case is suboptimal, yet this issue originates from the limitations of the similarity measure, be it needing $n>p$ or scaling prohibitively.

## Appendix B Full Benchmark Results

In the following, we provide the full results of all experiments within our benchmark.

### B.1 Graph Results

We begin with presenting the results from the graph domain.

**Table 7: Graph - Test 1: Correlation to Accuracy Difference**
| Dataset | Cora | Flickr | OGBN-Arxiv |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE |
| AlignCos | -0.37 | -0.19 | 0.15 | 0.09 | 0.17 | 0.39 | -0.18 | -0.07 | 0.20 |
| CKA | -0.08 | -0.31 | -0.13 | -0.09 | -0.15 | 0.37 | -0.13 | 0.04 | 0.06 |
| ConcDiff | -0.10 | -0.02 | -0.21 | 0.08 | -0.11 | -0.06 | -0.19 | -0.25 | -0.00 |
| DistCorr | -0.11 | -0.30 | -0.13 | -0.04 | 0.20 | 0.36 | -0.13 | 0.14 | 0.01 |
| EOS | -0.00 | 0.29 | 0.08 | 0.40 | 0.23 | 0.22 | 0.05 | -0.05 | 0.01 |
| GULP | -0.04 | 0.25 | 0.09 | -0.09 | 0.04 | 0.22 | 0.05 | -0.18 | -0.01 |
| HardCorr | -0.26 | 0.03 | -0.12 | -0.15 | 0.18 | 0.32 | 0.06 | 0.13 | -0.04 |
| IMD | -0.06 | -0.12 | 0.04 | -0.09 | -0.10 | 0.37 | -0.15 | -0.22 | -0.08 |
| Jaccard | -0.23 | -0.04 | -0.10 | -0.06 | 0.08 | 0.07 | -0.14 | -0.27 | -0.13 |
| LinReg | -0.21 | 0.01 | -0.21 | -0.12 | -0.23 | 0.28 | 0.05 | -0.23 | -0.02 |
| MagDiff | -0.07 | 0.05 | -0.11 | 0.10 | 0.18 | -0.01 | -0.10 | -0.21 | 0.03 |
| ShapeMet | -0.24 | -0.27 | -0.26 | -0.09 | 0.19 | 0.43 | -0.17 | -0.13 | -0.06 |
| OrthProc | -0.24 | -0.27 | -0.26 | -0.09 | 0.19 | 0.43 | -0.17 | -0.13 | -0.06 |
| PermProc | -0.32 | 0.06 | 0.17 | 0.09 | 0.35 | -0.04 | -0.09 | -0.05 | 0.38 |
| ProcDist | -0.26 | -0.05 | -0.01 | 0.09 | 0.01 | 0.01 | -0.22 | -0.24 | 0.35 |
| RSA | -0.33 | -0.14 | 0.06 | 0.03 | 0.33 | 0.30 | -0.24 | 0.56 | 0.17 |
| RSMDiff | -0.14 | -0.19 | 0.07 | 0.13 | -0.18 | -0.16 | 0.02 | -0.05 | -0.19 |
| RankSim | 0.04 | -0.04 | -0.09 | -0.05 | 0.08 | 0.14 | -0.31 | -0.26 | -0.00 |
| SVCCA | -0.12 | -0.04 | -0.10 | -0.10 | -0.21 | -0.12 | 0.12 | -0.18 | -0.03 |
| 2nd-Cos | -0.21 | -0.13 | 0.05 | -0.00 | 0.39 | -0.09 | -0.21 | -0.42 | -0.09 |
| SoftCorr | -0.09 | -0.13 | -0.06 | -0.20 | 0.12 | 0.31 | 0.03 | 0.07 | -0.02 |
| UnifDiff | 0.01 | -0.13 | -0.03 | 0.07 | -0.18 | 0.01 | -0.16 | -0.18 | -0.04 |

**Table 8: Graph - Test 2: Correlation to Output Difference**
| Measure | Disagreement | JSD |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | Cora | Flickr | OGBN-Arxiv | Cora | Flickr | OGBN-Arxiv |  |  |  |  |  |  |  |  |  |  |  |  |
| Model | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE |
| AlignCos | -0.05 | -0.08 | 0.27 | -0.08 | 0.15 | 0.37 | 0.17 | -0.10 | 0.00 | 0.17 | 0.02 | 0.38 | -0.01 | 0.31 | 0.44 | 0.28 | -0.03 | 0.05 |
| CKA | 0.45 | 0.36 | 0.00 | 0.06 | -0.21 | 0.53 | 0.23 | 0.03 | -0.04 | 0.53 | 0.43 | -0.03 | 0.17 | 0.03 | 0.58 | 0.38 | 0.12 | -0.02 |
| ConcDiff | 0.03 | -0.04 | -0.10 | 0.03 | -0.21 | -0.04 | -0.16 | -0.25 | 0.07 | 0.01 | 0.03 | 0.04 | 0.03 | -0.17 | -0.03 | -0.22 | -0.13 | 0.02 |
| DistCorr | 0.46 | 0.53 | -0.08 | -0.03 | 0.17 | 0.40 | 0.20 | 0.11 | 0.08 | 0.60 | 0.60 | 0.05 | 0.03 | 0.46 | 0.43 | 0.36 | 0.16 | 0.12 |
| EOS | -0.09 | 0.13 | 0.02 | 0.23 | 0.02 | 0.33 | 0.12 | -0.25 | -0.02 | -0.04 | 0.17 | 0.22 | 0.11 | -0.03 | 0.38 | 0.37 | -0.17 | 0.11 |
| GULP | -0.13 | -0.17 | -0.09 | -0.01 | 0.03 | 0.33 | 0.10 | 0.15 | -0.04 | -0.04 | -0.10 | 0.18 | 0.13 | 0.15 | 0.38 | 0.35 | 0.10 | 0.11 |
| HardCorr | 0.26 | 0.45 | 0.10 | -0.09 | 0.40 | 0.46 | 0.24 | 0.02 | -0.24 | 0.52 | 0.48 | 0.16 | 0.09 | 0.53 | 0.50 | 0.46 | -0.05 | -0.28 |
| IMD | -0.16 | 0.20 | -0.11 | -0.03 | -0.10 | 0.34 | 0.00 | -0.11 | -0.02 | -0.11 | 0.24 | -0.40 | 0.03 | -0.08 | 0.31 | -0.04 | -0.08 | -0.05 |
| Jaccard | 0.12 | 0.44 | 0.33 | -0.00 | 0.04 | 0.42 | 0.22 | 0.14 | 0.01 | 0.38 | 0.44 | 0.45 | 0.11 | 0.33 | 0.42 | 0.37 | 0.20 | 0.09 |
| LinReg | -0.06 | 0.57 | 0.09 | 0.06 | 0.12 | 0.46 | 0.22 | -0.10 | -0.19 | 0.19 | 0.58 | 0.33 | 0.18 | 0.15 | 0.48 | 0.47 | -0.09 | -0.17 |
| MagDiff | -0.07 | -0.07 | -0.25 | -0.20 | 0.06 | 0.07 | 0.22 | -0.25 | -0.19 | -0.15 | 0.00 | 0.06 | -0.26 | 0.03 | 0.06 | 0.08 | -0.13 | -0.13 |
| ShapeMet | 0.05 | 0.62 | 0.16 | 0.03 | 0.19 | 0.57 | 0.27 | 0.02 | -0.16 | 0.28 | 0.70 | 0.27 | 0.13 | 0.43 | 0.63 | 0.44 | 0.04 | -0.15 |
| OrthProc | 0.05 | 0.62 | 0.16 | 0.03 | 0.19 | 0.57 | 0.27 | 0.02 | -0.16 | 0.28 | 0.70 | 0.27 | 0.13 | 0.43 | 0.63 | 0.44 | 0.04 | -0.15 |
| PermProc | -0.14 | 0.24 | 0.33 | -0.27 | 0.18 | -0.10 | 0.22 | 0.25 | -0.41 | 0.13 | 0.29 | 0.10 | -0.42 | 0.29 | -0.10 | 0.06 | 0.34 | -0.55 |
| ProcDist | -0.17 | 0.45 | 0.24 | -0.27 | -0.13 | -0.15 | 0.30 | 0.26 | -0.38 | 0.05 | 0.52 | 0.08 | -0.38 | 0.02 | -0.18 | 0.13 | 0.36 | -0.50 |
| RSA | -0.02 | 0.06 | 0.33 | 0.06 | 0.35 | 0.59 | 0.45 | 0.17 | 0.15 | 0.20 | 0.12 | 0.13 | 0.14 | 0.52 | 0.63 | 0.46 | 0.20 | 0.14 |
| RSMDiff | -0.15 | 0.05 | 0.14 | -0.26 | -0.30 | -0.00 | 0.36 | -0.14 | 0.09 | -0.07 | 0.11 | -0.01 | -0.37 | -0.22 | -0.04 | 0.33 | -0.20 | 0.11 |
| RankSim | 0.44 | 0.34 | 0.23 | 0.05 | 0.06 | 0.24 | 0.10 | 0.03 | 0.27 | 0.51 | 0.36 | 0.56 | 0.21 | 0.33 | 0.30 | 0.13 | 0.02 | 0.36 |
| SVCCA | 0.17 | -0.06 | 0.04 | 0.03 | -0.27 | 0.21 | -0.15 | 0.08 | 0.11 | 0.26 | -0.09 | 0.02 | 0.13 | -0.04 | 0.23 | -0.17 | 0.19 | 0.16 |
| 2nd-Cos | 0.30 | 0.40 | 0.25 | -0.10 | 0.30 | 0.14 | 0.06 | 0.04 | 0.27 | 0.53 | 0.43 | 0.54 | 0.07 | 0.47 | 0.15 | 0.11 | 0.11 | 0.34 |
| SoftCorr | 0.07 | 0.49 | 0.06 | 0.02 | 0.39 | 0.49 | 0.35 | 0.03 | -0.26 | 0.21 | 0.53 | -0.01 | 0.23 | 0.53 | 0.53 | 0.56 | 0.02 | -0.28 |
| UnifDiff | -0.15 | 0.07 | 0.16 | 0.18 | -0.34 | 0.04 | 0.06 | -0.33 | 0.10 | -0.11 | 0.13 | 0.27 | 0.21 | -0.32 | 0.02 | 0.12 | -0.34 | 0.12 |

**Table 9: Graph - Test 3: Label Randomization**
| Measure | AUPRC | Conformity Rate |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | Cora | Flickr | OGBN-Arxiv | Cora | Flickr | OGBN-Arxiv |  |  |  |  |  |  |  |  |  |  |  |  |
| Model | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE |
| AlignCos | 0.43 | 0.46 | 0.48 | 0.29 | 0.84 | 0.42 | 0.93 | 0.98 | 0.70 | 0.60 | 0.66 | 0.66 | 0.58 | 0.94 | 0.50 | 0.97 | 1.00 | 0.67 |
| CKA | 0.42 | 0.43 | 0.42 | 0.27 | 0.73 | 0.66 | 1.00 | 1.00 | 1.00 | 0.51 | 0.55 | 0.51 | 0.54 | 0.91 | 0.86 | 1.00 | 1.00 | 1.00 |
| ConcDiff | 0.39 | 0.20 | 0.39 | 0.21 | 0.37 | 0.57 | 1.00 | 0.96 | 1.00 | 0.74 | 0.54 | 0.78 | 0.48 | 0.72 | 0.85 | 1.00 | 0.99 | 1.00 |
| DistCorr | 0.42 | 0.43 | 0.44 | 0.22 | 0.86 | 0.43 | 1.00 | 1.00 | 1.00 | 0.52 | 0.59 | 0.63 | 0.52 | 0.95 | 0.56 | 1.00 | 1.00 | 1.00 |
| EOS | 0.28 | 0.30 | 0.26 | 0.22 | 0.41 | 0.42 | 0.34 | 0.25 | 0.43 | 0.66 | 0.65 | 0.55 | 0.58 | 0.50 | 0.52 | 0.51 | 0.50 | 0.54 |
| GULP | 0.28 | 0.28 | 0.26 | 0.23 | 0.19 | 0.42 | 0.30 | 0.30 | 0.43 | 0.66 | 0.63 | 0.56 | 0.57 | 0.50 | 0.51 | 0.56 | 0.49 | 0.55 |
| HardCorr | 0.42 | 0.42 | 0.42 | 0.33 | 0.77 | 0.46 | 0.54 | 0.83 | 0.83 | 0.51 | 0.53 | 0.51 | 0.67 | 0.94 | 0.68 | 0.77 | 0.97 | 0.97 |
| IMD | 0.82 | 0.74 | 0.97 | 0.23 | 0.23 | 0.34 | 0.88 | 1.00 | 1.00 | 0.96 | 0.93 | 0.99 | 0.54 | 0.47 | 0.69 | 0.97 | 1.00 | 1.00 |
| Jaccard | 0.43 | 0.42 | 0.42 | 0.29 | 0.56 | 0.43 | 0.78 | 0.83 | 0.43 | 0.55 | 0.53 | 0.51 | 0.57 | 0.77 | 0.58 | 0.93 | 0.97 | 0.53 |
| LinReg | 0.46 | 0.49 | 0.43 | 0.23 | 0.22 | 0.45 | 0.47 | 0.44 | 0.68 | 0.69 | 0.71 | 0.58 | 0.52 | 0.49 | 0.66 | 0.63 | 0.63 | 0.81 |
| MagDiff | 0.37 | 0.20 | 0.24 | 0.18 | 0.66 | 0.72 | 0.34 | 0.55 | 0.49 | 0.74 | 0.51 | 0.60 | 0.47 | 0.89 | 0.93 | 0.75 | 0.83 | 0.86 |
| ShapeMet | 0.42 | 0.43 | 0.42 | 0.27 | 0.88 | 0.43 | 0.77 | 0.83 | 0.73 | 0.52 | 0.54 | 0.51 | 0.53 | 0.95 | 0.60 | 0.88 | 0.97 | 0.84 |
| OrthProc | 0.42 | 0.43 | 0.42 | 0.27 | 0.88 | 0.43 | 0.77 | 0.83 | 0.73 | 0.52 | 0.54 | 0.51 | 0.53 | 0.95 | 0.60 | 0.88 | 0.97 | 0.84 |
| PermProc | 0.44 | 0.44 | 0.39 | 0.19 | 0.77 | 0.90 | 0.93 | 0.68 | 0.72 | 0.61 | 0.64 | 0.71 | 0.50 | 0.94 | 0.97 | 0.98 | 0.84 | 0.88 |
| ProcDist | 0.43 | 0.45 | 0.45 | 0.19 | 0.79 | 0.62 | 1.00 | 0.92 | 0.98 | 0.60 | 0.67 | 0.68 | 0.51 | 0.93 | 0.88 | 1.00 | 0.98 | 1.00 |
| RSA | 0.43 | 0.46 | 0.44 | 0.33 | 0.74 | 0.42 | 0.49 | 0.96 | 0.43 | 0.54 | 0.69 | 0.61 | 0.64 | 0.89 | 0.52 | 0.63 | 0.99 | 0.58 |
| RSMDiff | 0.78 | 0.52 | 0.53 | 0.19 | 0.71 | 0.92 | 1.00 | 1.00 | 1.00 | 0.86 | 0.73 | 0.83 | 0.51 | 0.91 | 0.97 | 1.00 | 1.00 | 1.00 |
| RankSim | 0.47 | 0.45 | 0.42 | 0.33 | 0.47 | 0.43 | 0.78 | 0.84 | 0.55 | 0.64 | 0.65 | 0.54 | 0.58 | 0.66 | 0.53 | 0.93 | 0.97 | 0.63 |
| SVCCA | 0.42 | 0.35 | 0.31 | 0.27 | 0.33 | 0.80 | 0.93 | 1.00 | 1.00 | 0.59 | 0.62 | 0.56 | 0.54 | 0.69 | 0.91 | 0.96 | 1.00 | 1.00 |
| 2nd-Cos | 0.73 | 0.56 | 0.73 | 0.36 | 0.61 | 0.42 | 0.95 | 0.99 | 0.96 | 0.86 | 0.80 | 0.87 | 0.66 | 0.82 | 0.50 | 0.99 | 1.00 | 0.99 |
| SoftCorr | 0.42 | 0.42 | 0.42 | 0.33 | 0.60 | 0.45 | 0.55 | 0.83 | 0.82 | 0.50 | 0.53 | 0.51 | 0.65 | 0.85 | 0.67 | 0.71 | 0.97 | 0.96 |
| UnifDiff | 0.32 | 0.33 | 0.50 | 0.40 | 0.53 | 0.90 | 0.33 | 0.53 | 0.54 | 0.67 | 0.70 | 0.81 | 0.75 | 0.84 | 0.96 | 0.66 | 0.78 | 0.81 |

**Table 10: Graph - Test 4: Shortcut Affinity**
| Measure | AUPRC | Conformity Rate |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | Cora | Flickr | OGBN-Arxiv | Cora | Flickr | OGBN-Arxiv |  |  |  |  |  |  |  |  |  |  |  |  |
| Model | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE |
| AlignCos | 0.30 | 0.32 | 0.60 | 0.48 | 0.89 | 1.00 | 1.00 | 1.00 | 1.00 | 0.59 | 0.69 | 0.88 | 0.77 | 0.98 | 1.00 | 1.00 | 1.00 | 1.00 |
| CKA | 0.46 | 0.27 | 0.78 | 0.33 | 0.28 | 1.00 | 1.00 | 1.00 | 0.98 | 0.64 | 0.66 | 0.90 | 0.65 | 0.57 | 1.00 | 1.00 | 1.00 | 1.00 |
| ConcDiff | 0.20 | 0.18 | 0.17 | 0.32 | 0.18 | 0.18 | 1.00 | 0.81 | 0.96 | 0.50 | 0.47 | 0.43 | 0.61 | 0.50 | 0.46 | 1.00 | 0.96 | 0.99 |
| DistCorr | 0.46 | 0.28 | 0.82 | 0.32 | 0.33 | 1.00 | 1.00 | 1.00 | 0.99 | 0.62 | 0.69 | 0.92 | 0.72 | 0.69 | 1.00 | 1.00 | 1.00 | 1.00 |
| EOS | 0.16 | 0.22 | 0.50 | 0.54 | 0.46 | 0.43 | 0.97 | 1.00 | 0.72 | 0.42 | 0.57 | 0.74 | 0.81 | 0.87 | 0.60 | 0.99 | 1.00 | 0.83 |
| GULP | 0.17 | 0.20 | 0.50 | 0.45 | 0.20 | 0.43 | 0.96 | 0.99 | 0.72 | 0.44 | 0.53 | 0.74 | 0.81 | 0.49 | 0.60 | 0.99 | 1.00 | 0.83 |
| HardCorr | 0.22 | 0.20 | 0.35 | 0.52 | 0.55 | 1.00 | 0.83 | 0.80 | 0.72 | 0.58 | 0.56 | 0.77 | 0.81 | 0.83 | 1.00 | 0.97 | 0.96 | 0.83 |
| IMD | 0.60 | 0.73 | 0.79 | 0.36 | 0.76 | 0.97 | 0.92 | 0.61 | 0.92 | 0.88 | 0.93 | 0.96 | 0.62 | 0.94 | 0.99 | 0.98 | 0.84 | 0.98 |
| Jaccard | 0.38 | 0.31 | 0.78 | 0.54 | 1.00 | 0.83 | 1.00 | 1.00 | 1.00 | 0.71 | 0.70 | 0.93 | 0.84 | 1.00 | 0.96 | 1.00 | 1.00 | 1.00 |
| LinReg | 0.33 | 0.30 | 0.74 | 0.36 | 0.37 | 0.61 | 1.00 | 1.00 | 1.00 | 0.62 | 0.62 | 0.95 | 0.72 | 0.70 | 0.81 | 1.00 | 1.00 | 1.00 |
| MagDiff | 0.19 | 0.27 | 0.15 | 0.20 | 0.27 | 0.78 | 1.00 | 0.55 | 0.53 | 0.49 | 0.48 | 0.41 | 0.48 | 0.67 | 0.89 | 1.00 | 0.82 | 0.82 |
| ShapeMet | 0.37 | 0.27 | 0.79 | 0.31 | 0.58 | 1.00 | 1.00 | 1.00 | 1.00 | 0.62 | 0.67 | 0.91 | 0.72 | 0.80 | 1.00 | 1.00 | 1.00 | 1.00 |
| OrthProc | 0.37 | 0.27 | 0.79 | 0.31 | 0.58 | 1.00 | 1.00 | 1.00 | 1.00 | 0.62 | 0.67 | 0.91 | 0.72 | 0.80 | 1.00 | 1.00 | 1.00 | 1.00 |
| PermProc | 0.22 | 0.21 | 0.27 | 0.27 | 0.42 | 1.00 | 0.77 | 0.65 | 0.43 | 0.52 | 0.59 | 0.67 | 0.58 | 0.72 | 1.00 | 0.91 | 0.86 | 0.60 |
| ProcDist | 0.37 | 0.25 | 0.81 | 0.33 | 0.60 | 1.00 | 1.00 | 1.00 | 1.00 | 0.60 | 0.64 | 0.91 | 0.61 | 0.82 | 1.00 | 1.00 | 1.00 | 1.00 |
| RSA | 0.58 | 0.30 | 0.43 | 0.52 | 0.89 | 1.00 | 0.90 | 0.97 | 0.98 | 0.77 | 0.70 | 0.78 | 0.81 | 0.97 | 1.00 | 0.98 | 0.99 | 0.99 |
| RSMDiff | 0.69 | 0.46 | 0.50 | 0.36 | 0.48 | 0.92 | 1.00 | 1.00 | 1.00 | 0.91 | 0.70 | 0.76 | 0.59 | 0.75 | 0.97 | 1.00 | 1.00 | 1.00 |
| RankSim | 0.36 | 0.33 | 0.64 | 0.54 | 1.00 | 0.77 | 1.00 | 1.00 | 1.00 | 0.72 | 0.65 | 0.89 | 0.83 | 1.00 | 0.92 | 1.00 | 1.00 | 1.00 |
| SVCCA | 0.57 | 0.19 | 0.36 | 0.32 | 0.24 | 0.93 | 0.83 | 1.00 | 0.97 | 0.70 | 0.46 | 0.60 | 0.66 | 0.57 | 0.97 | 0.91 | 1.00 | 0.99 |
| 2nd-Cos | 0.47 | 0.34 | 0.91 | 0.52 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.69 | 0.74 | 0.97 | 0.82 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| SoftCorr | 0.22 | 0.21 | 0.40 | 0.57 | 0.66 | 1.00 | 0.83 | 0.82 | 0.72 | 0.58 | 0.57 | 0.79 | 0.82 | 0.87 | 1.00 | 0.97 | 0.96 | 0.83 |
| UnifDiff | 0.19 | 0.19 | 0.64 | 0.48 | 0.59 | 0.50 | 0.97 | 0.99 | 0.72 | 0.47 | 0.52 | 0.91 | 0.67 | 0.83 | 0.77 | 0.99 | 1.00 | 0.86 |

**Table 11: Graph - Test 5: Augmentation**
| Measure | AUPRC | Conformity Rate |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | Cora | Flickr | OGBN-Arxiv | Cora | Flickr | OGBN-Arxiv |  |  |  |  |  |  |  |  |  |  |  |  |
| Model | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE |
| AlignCos | 0.91 | 0.51 | 0.63 | 0.54 | 0.69 | 0.70 | 1.00 | 0.74 | 0.43 | 0.98 | 0.78 | 0.88 | 0.81 | 0.87 | 0.90 | 1.00 | 0.87 | 0.54 |
| CKA | 0.91 | 0.70 | 0.73 | 0.51 | 0.57 | 0.75 | 0.99 | 1.00 | 1.00 | 0.97 | 0.90 | 0.92 | 0.74 | 0.83 | 0.91 | 1.00 | 1.00 | 1.00 |
| ConcDiff | 0.62 | 0.17 | 0.50 | 0.53 | 0.41 | 0.35 | 0.51 | 0.43 | 0.53 | 0.87 | 0.49 | 0.78 | 0.74 | 0.78 | 0.68 | 0.84 | 0.76 | 0.80 |
| DistCorr | 0.89 | 0.79 | 0.73 | 0.61 | 0.60 | 0.79 | 0.99 | 1.00 | 1.00 | 0.97 | 0.93 | 0.91 | 0.81 | 0.84 | 0.94 | 1.00 | 1.00 | 1.00 |
| EOS | 0.34 | 0.53 | 0.62 | 0.46 | 0.68 | 0.53 | 1.00 | 0.82 | 0.49 | 0.75 | 0.86 | 0.90 | 0.70 | 0.88 | 0.80 | 1.00 | 0.96 | 0.75 |
| GULP | 0.35 | 0.45 | 0.61 | 0.55 | 0.22 | 0.54 | 1.00 | 0.56 | 0.48 | 0.76 | 0.79 | 0.89 | 0.84 | 0.52 | 0.81 | 1.00 | 0.80 | 0.75 |
| HardCorr | 0.51 | 0.51 | 0.63 | 0.57 | 0.71 | 0.72 | 0.54 | 0.47 | 0.51 | 0.81 | 0.85 | 0.89 | 0.85 | 0.89 | 0.92 | 0.79 | 0.59 | 0.77 |
| IMD | 0.24 | 0.71 | 0.35 | 0.28 | 0.59 | 0.59 | 1.00 | 1.00 | 1.00 | 0.47 | 0.93 | 0.64 | 0.53 | 0.87 | 0.87 | 1.00 | 1.00 | 1.00 |
| Jaccard | 0.95 | 0.78 | 0.81 | 0.58 | 0.74 | 0.88 | 1.00 | 1.00 | 0.99 | 0.98 | 0.90 | 0.95 | 0.84 | 0.88 | 0.97 | 1.00 | 1.00 | 1.00 |
| LinReg | 0.87 | 0.94 | 0.85 | 0.34 | 0.30 | 0.81 | 0.81 | 0.72 | 0.72 | 0.95 | 0.98 | 0.96 | 0.69 | 0.53 | 0.93 | 0.93 | 0.83 | 0.83 |
| MagDiff | 0.54 | 0.20 | 0.39 | 0.43 | 0.59 | 0.17 | 0.69 | 0.77 | 1.00 | 0.83 | 0.51 | 0.74 | 0.66 | 0.87 | 0.45 | 0.91 | 0.93 | 1.00 |
| ShapeMet | 0.83 | 0.81 | 0.78 | 0.54 | 0.63 | 0.76 | 0.74 | 0.72 | 0.72 | 0.95 | 0.95 | 0.93 | 0.77 | 0.85 | 0.94 | 0.89 | 0.83 | 0.83 |
| OrthProc | 0.83 | 0.81 | 0.78 | 0.54 | 0.63 | 0.76 | 0.74 | 0.72 | 0.72 | 0.95 | 0.95 | 0.93 | 0.77 | 0.85 | 0.94 | 0.89 | 0.83 | 0.83 |
| PermProc | 0.42 | 0.70 | 0.61 | 0.40 | 0.65 | 0.69 | 0.94 | 0.88 | 0.73 | 0.51 | 0.78 | 0.89 | 0.62 | 0.87 | 0.90 | 0.98 | 0.97 | 0.87 |
| ProcDist | 0.75 | 0.76 | 0.69 | 0.40 | 0.70 | 0.81 | 1.00 | 1.00 | 1.00 | 0.90 | 0.88 | 0.90 | 0.63 | 0.89 | 0.95 | 1.00 | 1.00 | 1.00 |
| RSA | 0.78 | 0.39 | 0.57 | 0.59 | 0.68 | 0.72 | 0.67 | 0.75 | 0.49 | 0.94 | 0.68 | 0.84 | 0.84 | 0.86 | 0.92 | 0.85 | 0.89 | 0.78 |
| RSMDiff | 1.00 | 0.99 | 0.79 | 0.40 | 0.64 | 0.93 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.93 | 0.62 | 0.87 | 0.97 | 1.00 | 1.00 | 1.00 |
| RankSim | 0.95 | 0.54 | 0.62 | 0.55 | 0.75 | 0.88 | 1.00 | 1.00 | 1.00 | 0.98 | 0.72 | 0.88 | 0.83 | 0.89 | 0.96 | 1.00 | 1.00 | 1.00 |
| SVCCA | 0.34 | 0.27 | 0.29 | 0.49 | 0.64 | 0.67 | 0.82 | 0.89 | 0.81 | 0.58 | 0.54 | 0.55 | 0.68 | 0.84 | 0.86 | 0.91 | 0.89 | 0.85 |
| 2nd-Cos | 0.97 | 0.73 | 0.86 | 0.62 | 0.78 | 0.92 | 1.00 | 1.00 | 0.99 | 0.99 | 0.88 | 0.97 | 0.84 | 0.91 | 0.98 | 1.00 | 1.00 | 1.00 |
| SoftCorr | 0.50 | 0.63 | 0.70 | 0.55 | 0.73 | 0.58 | 0.45 | 0.43 | 0.46 | 0.80 | 0.89 | 0.92 | 0.84 | 0.87 | 0.85 | 0.66 | 0.51 | 0.71 |
| UnifDiff | 1.00 | 0.75 | 0.53 | 0.25 | 0.68 | 0.24 | 0.45 | 0.76 | 0.53 | 1.00 | 0.91 | 0.86 | 0.66 | 0.90 | 0.52 | 0.78 | 0.88 | 0.81 |

**Table 12: Graph - Test 6: Layer Monotonicity**
| Measure | Spearman Correlation | Conformity Rate |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | Cora | Flickr | OGBN-Arxiv | Cora | Flickr | OGBN-Arxiv |  |  |  |  |  |  |  |  |  |  |  |  |
| Model | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE | GAT | GCN | SAGE |
| AlignCos | 1.00 | 0.98 | 1.00 | 0.04 | 0.33 | 0.68 | 0.84 | 0.84 | 0.93 | 1.00 | 0.97 | 1.00 | 0.46 | 0.59 | 0.77 | 0.89 | 0.83 | 0.93 |
| CKA | 1.00 | 0.98 | 1.00 | 0.39 | 0.60 | 0.89 | 0.96 | 0.85 | 0.46 | 1.00 | 0.98 | 1.00 | 0.59 | 0.79 | 0.92 | 0.95 | 0.87 | 0.64 |
| ConcDiff | 0.40 | 0.85 | 0.25 | 0.24 | 0.38 | -0.27 | 0.40 | 0.73 | 0.39 | 0.62 | 0.85 | 0.52 | 0.58 | 0.57 | 0.36 | 0.67 | 0.70 | 0.74 |
| DistCorr | 1.00 | 1.00 | 1.00 | 0.47 | 0.63 | 0.99 | 0.93 | 0.81 | 0.52 | 1.00 | 1.00 | 1.00 | 0.63 | 0.81 | 0.99 | 0.92 | 0.88 | 0.65 |
| EOS | 1.00 | 1.00 | 1.00 | 0.92 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.92 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| GULP | 1.00 | 1.00 | 1.00 | 0.33 | 0.54 | 1.00 | 1.00 | 0.73 | 1.00 | 1.00 | 1.00 | 1.00 | 0.72 | 0.83 | 1.00 | 1.00 | 0.82 | 1.00 |
| HardCorr | 0.54 | 0.83 | 0.98 | 0.21 | 0.63 | 0.80 | 1.00 | 0.83 | 0.62 | 0.76 | 0.91 | 0.99 | 0.68 | 0.81 | 0.84 | 1.00 | 0.86 | 0.72 |
| IMD | 1.00 | 1.00 | 1.00 | 0.97 | 1.00 | 0.82 | 1.00 | 0.55 | 1.00 | 1.00 | 1.00 | 1.00 | 0.97 | 1.00 | 0.94 | 1.00 | 0.85 | 1.00 |
| Jaccard | 1.00 | 1.00 | 1.00 | 0.95 | 0.95 | 0.97 | 1.00 | 0.97 | 0.99 | 1.00 | 1.00 | 1.00 | 0.96 | 0.94 | 0.96 | 1.00 | 0.96 | 0.98 |
| LinReg | 1.00 | 1.00 | 1.00 | 0.03 | 0.45 | 1.00 | 1.00 | 0.99 | 1.00 | 1.00 | 1.00 | 1.00 | 0.59 | 0.67 | 1.00 | 1.00 | 0.99 | 1.00 |
| MagDiff | 0.89 | 0.55 | 0.49 | 0.38 | 0.58 | 0.50 | 0.63 | 0.06 | 0.29 | 0.92 | 0.63 | 0.72 | 0.58 | 0.65 | 0.63 | 0.87 | 0.52 | 0.71 |
| ShapeMet | 1.00 | 1.00 | 1.00 | 0.45 | 0.62 | 0.98 | 0.99 | 0.97 | 0.99 | 1.00 | 1.00 | 1.00 | 0.66 | 0.80 | 0.99 | 0.99 | 0.95 | 0.99 |
| OrthProc | 1.00 | 1.00 | 1.00 | 0.45 | 0.62 | 0.98 | 0.99 | 0.97 | 0.99 | 1.00 | 1.00 | 1.00 | 0.66 | 0.80 | 0.99 | 0.99 | 0.95 | 0.99 |
| PermProc | 1.00 | 0.92 | 1.00 | 0.63 | 0.60 | 0.68 | 1.00 | 0.24 | 0.88 | 1.00 | 0.91 | 1.00 | 0.75 | 0.68 | 0.72 | 1.00 | 0.69 | 0.87 |
| ProcDist | 1.00 | 0.99 | 1.00 | 0.62 | 0.66 | 1.00 | 1.00 | 0.83 | 0.85 | 1.00 | 0.99 | 1.00 | 0.78 | 0.75 | 1.00 | 1.00 | 0.93 | 0.96 |
| RSA | 1.00 | 0.84 | 0.91 | 0.44 | 0.58 | 0.99 | 0.94 | 0.52 | 0.97 | 1.00 | 0.90 | 0.97 | 0.70 | 0.70 | 0.98 | 0.94 | 0.81 | 0.97 |
| RSMDiff | 1.00 | 0.99 | 1.00 | 0.81 | 0.54 | 0.65 | 1.00 | 0.85 | 0.93 | 1.00 | 0.99 | 1.00 | 0.92 | 0.66 | 0.68 | 1.00 | 0.85 | 0.93 |
| RankSim | 1.00 | 1.00 | 1.00 | 0.99 | 0.95 | 0.97 | 1.00 | 0.99 | 0.99 | 1.00 | 1.00 | 1.00 | 0.98 | 0.94 | 0.95 | 1.00 | 0.99 | 0.98 |
| SVCCA | 0.61 | 0.41 | 0.78 | 0.47 | 0.53 | 0.44 | 0.33 | -0.17 | -0.10 | 0.81 | 0.63 | 0.80 | 0.57 | 0.75 | 0.69 | 0.56 | 0.49 | 0.41 |
| 2nd-Cos | 1.00 | 1.00 | 1.00 | 0.52 | 0.81 | 0.96 | 1.00 | 0.97 | 1.00 | 1.00 | 1.00 | 1.00 | 0.67 | 0.91 | 0.96 | 1.00 | 0.95 | 1.00 |
| SoftCorr | 0.50 | 0.96 | 0.99 | 0.11 | 0.79 | 0.89 | 1.00 | 0.91 | 0.62 | 0.78 | 0.95 | 0.99 | 0.64 | 0.89 | 0.88 | 1.00 | 0.91 | 0.73 |
| UnifDiff | 0.59 | 0.67 | 0.67 | 0.82 | 0.82 | 0.89 | 0.68 | 0.33 | -0.45 | 0.69 | 0.70 | 0.70 | 0.94 | 0.94 | 0.92 | 0.83 | 0.60 | 0.41 |

### B.2 Language Results

All results are shown in Table 13.

**Table 13: Language Test 1-6: Results of all Test for the language domain. In all cases, we use BERT models.**
| Type | Grounding by Prediction | Grounding by Design |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Test | Acc Corr. | JSD Corr. | Disagr. Corr. | Random Labels | Shortcuts | Augmentation | Layer Mono. |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Eval. | Spearman | Spearman | Spearman | AUPRC | Conformity Rate | AUPRC | Conformity Rate | AUPRC | Conformity Rate | Conformity Rate | Spearman |  |  |  |  |  |  |  |  |  |  |  |
| Dataset | MNLI | SST2 | MNLI | SST2 | MNLI | SST2 | MNLI | SST2 | MNLI | SST2 | MNLI | SST2 | MNLI | SST2 | MNLI | SST2 | MNLI | SST2 | MNLI | SST2 | MNLI | SST2 |
| 2nd-Cos | -0.26 | 0.30 | 0.16 | 0.41 | 0.55 | -0.00 | 0.39 | 0.37 | nan | 0.64 | 0.59 | 0.55 | 0.82 | 0.84 | 0.44 | 0.40 | 0.64 | 0.64 | 0.94 | 0.96 | 0.94 | 0.97 |
| AlignCos | 0.25 | 0.02 | 0.37 | 0.40 | -0.16 | 0.15 | 1.00 | 0.99 | 1.00 | 1.00 | 0.58 | 0.45 | 0.89 | 0.79 | 0.35 | 0.45 | 0.80 | 0.63 | 1.00 | 0.99 | 1.00 | 0.99 |
| AngShape | 0.28 | -0.14 | 0.26 | 0.29 | -0.02 | 0.39 | 0.90 | 0.49 | 0.98 | 0.75 | 0.60 | 0.31 | 0.90 | 0.77 | 0.38 | 0.52 | 0.84 | 0.62 | 0.99 | 0.98 | 1.00 | 0.99 |
| CKA | 0.18 | -0.06 | 0.30 | 0.39 | -0.01 | 0.43 | 0.75 | 0.59 | 0.90 | 0.78 | 0.59 | 0.26 | 0.88 | 0.67 | 0.48 | 0.61 | 0.87 | 0.68 | 0.98 | 0.98 | 0.99 | 0.96 |
| ConcDiff | -0.00 | 0.12 | 0.02 | 0.05 | -0.31 | -0.21 | 1.00 | 0.96 | 1.00 | 0.99 | 0.38 | 0.26 | 0.75 | 0.67 | 0.28 | 0.32 | 0.61 | 0.51 | 0.99 | 0.96 | 0.99 | 0.96 |
| DistCorr | 0.15 | -0.10 | 0.39 | 0.42 | 0.12 | 0.44 | 0.75 | 0.59 | 0.89 | 0.80 | 0.58 | 0.26 | 0.88 | 0.68 | 0.45 | 0.62 | 0.85 | 0.69 | 0.96 | 0.97 | 0.97 | 0.98 |
| EOS | 0.03 | -0.38 | 0.36 | -0.38 | 0.01 | -0.22 | 0.62 | 0.36 | 0.88 | 0.60 | 0.57 | 0.25 | 0.85 | 0.59 | 0.27 | 0.30 | 0.71 | 0.49 | 0.98 | 0.91 | 0.99 | 0.92 |
| GULP | -0.01 | -0.36 | 0.35 | -0.47 | 0.02 | -0.27 | 0.61 | 0.28 | 0.89 | 0.54 | 0.56 | 0.25 | 0.84 | 0.65 | 0.26 | 0.33 | 0.69 | 0.47 | 0.84 | 0.78 | 0.51 | 0.45 |
| HardCorr | 0.04 | -0.33 | -0.27 | -0.19 | -0.43 | 0.22 | 0.75 | 0.44 | 0.90 | 0.77 | 0.55 | 0.23 | 0.82 | 0.64 | 0.24 | 0.34 | 0.67 | 0.49 | 1.00 | 0.99 | 1.00 | 0.99 |
| IMD | -0.26 | -0.06 | -0.39 | 0.13 | -0.06 | 0.35 | 0.73 | 0.47 | 0.92 | 0.79 | 0.53 | 0.34 | 0.82 | 0.72 | 0.43 | 0.30 | 0.75 | 0.47 | 0.68 | 0.64 | 0.44 | 0.54 |
| Jaccard | -0.21 | 0.17 | 0.13 | 0.46 | 0.17 | 0.09 | 0.69 | 0.35 | 0.84 | 0.69 | 0.56 | 0.55 | 0.81 | 0.86 | 0.35 | 0.39 | 0.74 | 0.61 | 0.95 | 0.94 | 0.96 | 0.96 |
| LinReg | 0.18 | -0.38 | 0.28 | -0.14 | -0.03 | 0.21 | 0.57 | 0.40 | 0.85 | 0.57 | 0.46 | 0.37 | 0.85 | 0.74 | 0.24 | 0.40 | 0.66 | 0.55 | 0.83 | 0.93 | 0.66 | 0.90 |
| MagDiff | 0.22 | -0.13 | 0.01 | 0.02 | -0.03 | -0.06 | 0.33 | 0.38 | 0.75 | 0.69 | 0.48 | 0.35 | 0.81 | 0.73 | 0.16 | 0.31 | 0.45 | 0.46 | 0.64 | 0.62 | 0.52 | 0.48 |
| OrthProc | 0.28 | -0.14 | 0.26 | 0.29 | -0.02 | 0.39 | 0.90 | 0.49 | 0.98 | 0.75 | 0.60 | 0.31 | 0.90 | 0.77 | 0.38 | 0.52 | 0.84 | 0.62 | 0.98 | 0.98 | 0.99 | 0.99 |
| PWCCA | 0.01 | -0.33 | 0.22 | -0.39 | -0.37 | -0.28 | 0.65 | 0.27 | 0.91 | 0.48 | 0.56 | 0.35 | 0.86 | 0.69 | 0.18 | 0.35 | 0.49 | 0.38 | 1.00 | 1.00 | 1.00 | 1.00 |
| PermProc | 0.09 | -0.09 | -0.06 | -0.13 | -0.30 | 0.04 | 0.44 | 0.45 | 0.68 | 0.73 | 0.52 | 0.46 | 0.82 | 0.78 | 0.18 | 0.31 | 0.49 | 0.44 | 0.80 | 0.83 | 0.73 | 0.75 |
| ProcDist | 0.28 | 0.10 | 0.07 | 0.40 | -0.38 | 0.27 | 0.98 | 0.86 | 0.99 | 0.95 | 0.54 | 0.43 | 0.87 | 0.77 | 0.32 | 0.43 | 0.73 | 0.64 | 0.96 | 0.97 | 0.92 | 0.89 |
| RSA | 0.00 | -0.23 | 0.27 | 0.34 | 0.19 | 0.52 | 0.46 | 0.48 | 0.64 | 0.73 | 0.58 | 0.36 | 0.87 | 0.68 | 0.49 | 0.61 | 0.86 | 0.68 | 0.99 | 0.95 | 1.00 | 0.96 |
| RSMDiff | 0.30 | 0.20 | -0.18 | 0.35 | -0.19 | -0.03 | 0.52 | 0.91 | 0.67 | 0.97 | 0.28 | 0.34 | 0.66 | 0.69 | 0.35 | 0.34 | 0.66 | 0.58 | 1.00 | 0.95 | nan | 0.84 |
| RankSim | -0.09 | 0.14 | 0.08 | 0.49 | 0.05 | 0.11 | 0.63 | 0.34 | 0.82 | 0.63 | 0.58 | 0.55 | 0.82 | 0.86 | 0.33 | 0.36 | 0.71 | 0.57 | 0.93 | 0.94 | 0.90 | 0.96 |
| SVCCA | 0.32 | -0.08 | 0.47 | 0.37 | 0.00 | 0.40 | 0.69 | 0.64 | 0.86 | 0.80 | 0.42 | 0.25 | 0.78 | 0.62 | 0.44 | 0.61 | 0.77 | 0.68 | 0.91 | 0.88 | 0.91 | 0.64 |
| SoftCorr | 0.11 | -0.33 | -0.23 | -0.18 | -0.42 | 0.24 | 0.75 | 0.48 | 0.92 | 0.80 | 0.55 | 0.22 | 0.82 | 0.55 | 0.28 | 0.41 | 0.72 | 0.52 | 0.98 | 0.96 | 0.99 | 0.95 |
| UnifDiff | 0.14 | 0.35 | -0.02 | 0.35 | -0.14 | -0.10 | 0.76 | 0.60 | 0.91 | 0.86 | 0.40 | 0.35 | 0.69 | 0.55 | 0.61 | 0.33 | 0.84 | 0.46 | 0.94 | 0.85 | 0.81 | 0.77 |

### B.3 Vision Results

We present all tables containing the results of the vision domain. Results are presented in test order, with each test featuring one table.
Some entries in the result tables feature ’NaN’ values. This was caused by various reasons: (i) Numerical instability due to operations like Singular Value Decompositions or negative values in square roots and (ii) identical similarity values, leading to failure of correlation values. Whenever such failure occured for an entire group of models the entire measure was excluded for this case, as e.g. removing the model group of gauss noise M may simplify the overall task of distinguishing models.

**Table 14: Vision - Test 1: Correlation to Accuracy difference. Full results of the correlation between the similarity measures and the absolute accuracy differences for all architectures.**
| Test | Accuracy Correlation |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Measure | Spearman |  |  |  |  |  |  |
| Architecture | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViT B32 | ViT L32 |
| 2nd-Cos | -0.079 | -0.150 | 0.046 | -0.201 | -0.180 | -0.217 | 0.165 |
| AlignCos | -0.077 | -0.353 | -0.014 | -0.134 | -0.119 | 0.074 | 0.045 |
| AngShape | 0.210 | -0.162 | 0.155 | -0.021 | 0.025 | 0.067 | 0.062 |
| CKA | 0.357 | -0.071 | 0.160 | 0.030 | -0.205 | -0.258 | 0.045 |
| ConcDiff | -0.106 | 0.341 | -0.043 | -0.111 | -0.125 | 0.003 | 0.176 |
| DistCorr | 0.309 | -0.080 | 0.081 | 0.031 | -0.211 | -0.259 | 0.032 |
| EOS | 0.048 | -0.171 | 0.112 | -0.220 | 0.080 | 0.470 | 0.034 |
| GULP | 0.022 | -0.181 | 0.125 | -0.172 | 0.104 | 0.182 | 0.035 |
| HardCorr | 0.210 | 0.131 | -0.009 | -0.015 | -0.028 | 0.346 | -0.173 |
| IMD | 0.167 | -0.201 | 0.118 | 0.084 | -0.170 | -0.233 | -0.188 |
| Jaccard | -0.114 | -0.129 | -0.038 | -0.215 | 0.066 | -0.013 | 0.251 |
| LinReg | 0.187 | -0.111 | 0.091 | -0.039 | 0.088 | 0.153 | 0.055 |
| MagDiff | -0.161 | 0.021 | -0.063 | -0.075 | -0.117 | 0.065 | 0.145 |
| OrthProc | 0.210 | -0.163 | 0.155 | -0.025 | 0.025 | 0.070 | 0.062 |
| PWCCA | -0.022 | -0.202 | -0.085 | -0.184 | 0.110 | 0.837 | 0.088 |
| PermProc | 0.070 | 0.093 | 0.085 | 0.138 | -0.016 | -0.059 | -0.328 |
| ProcDist | 0.080 | 0.001 | 0.143 | 0.130 | 0.082 | 0.163 | 0.046 |
| RSA | 0.056 | -0.172 | 0.088 | 0.238 | -0.347 | -0.125 | -0.105 |
| RSMDiff | 0.087 | -0.105 | 0.106 | -0.038 | -0.079 | 0.010 | -0.060 |
| RankSim | 0.071 | 0.035 | 0.126 | -0.011 | 0.026 | 0.175 | 0.350 |
| SVCCA | 0.295 | 0.269 | -0.004 | -0.038 | -0.295 | -0.012 | -0.165 |
| SoftCorr | 0.269 | 0.081 | 0.041 | -0.026 | -0.101 | 0.359 | -0.194 |
| UnifDiff | -0.184 | nan | nan | 0.169 | -0.042 | nan | nan |

**Table 15: Vision - Test 2: Correlation to Output difference. Full results of the correlation between the similarity measures and the output differences for all architectures.**
| Test | Output Correlation |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Measure | JSD | Disagreement |  |  |  |  |  |  |  |  |  |  |  |  |
| Architecture | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViT B32 | ViT L32 | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViTB32 | ViT L32 |
| 2nd-Cos | -0.127 | 0.164 | 0.286 | 0.071 | -0.295 | 0.430 | -0.353 | -0.210 | 0.452 | 0.099 | 0.108 | -0.068 | 0.188 | -0.275 |
| AlignCos | 0.076 | 0.045 | 0.382 | 0.101 | -0.195 | -0.220 | -0.064 | 0.192 | 0.504 | 0.167 | 0.160 | -0.125 | -0.077 | 0.004 |
| AngShape | 0.243 | 0.216 | 0.340 | -0.012 | 0.187 | -0.259 | -0.153 | 0.238 | 0.404 | 0.194 | -0.137 | -0.331 | -0.114 | -0.060 |
| CKA | 0.299 | 0.083 | 0.299 | -0.127 | -0.058 | 0.044 | -0.067 | 0.369 | 0.083 | 0.224 | 0.009 | -0.239 | 0.003 | -0.020 |
| ConcDiff | -0.293 | 0.245 | -0.112 | -0.172 | -0.133 | -0.110 | -0.370 | -0.087 | 0.001 | -0.211 | -0.114 | -0.059 | -0.081 | -0.286 |
| DistCorr | 0.260 | 0.050 | 0.311 | -0.099 | 0.043 | 0.052 | -0.116 | 0.362 | 0.010 | 0.285 | -0.004 | -0.250 | 0.017 | -0.052 |
| EOS | 0.087 | 0.489 | 0.331 | -0.105 | 0.153 | -0.185 | -0.281 | 0.114 | 0.255 | 0.144 | -0.306 | -0.411 | 0.013 | -0.152 |
| GULP | 0.071 | 0.493 | 0.355 | -0.055 | 0.146 | -0.047 | -0.279 | 0.098 | 0.265 | 0.130 | -0.269 | -0.415 | 0.190 | -0.150 |
| HardCorr | 0.283 | 0.305 | 0.019 | -0.217 | -0.050 | 0.029 | -0.260 | 0.282 | -0.058 | -0.087 | -0.147 | -0.176 | 0.273 | -0.163 |
| IMD | -0.110 | 0.084 | 0.210 | 0.204 | 0.108 | 0.413 | 0.070 | -0.068 | 0.001 | 0.260 | 0.260 | 0.024 | 0.233 | 0.072 |
| Jaccard | 0.356 | 0.255 | 0.323 | 0.048 | 0.333 | 0.470 | -0.298 | 0.253 | 0.470 | 0.235 | 0.135 | -0.113 | 0.342 | -0.181 |
| LinReg | 0.215 | 0.212 | 0.414 | -0.006 | 0.254 | -0.129 | -0.138 | 0.195 | 0.252 | 0.295 | -0.172 | -0.236 | 0.039 | -0.067 |
| MagDiff | -0.383 | -0.202 | 0.019 | -0.164 | -0.280 | 0.020 | -0.316 | -0.174 | -0.222 | -0.055 | -0.088 | 0.042 | -0.014 | -0.223 |
| OrthProc | 0.243 | 0.215 | 0.340 | -0.016 | 0.187 | -0.257 | -0.153 | 0.238 | 0.403 | 0.194 | -0.136 | -0.331 | -0.114 | -0.060 |
| PWCCA | 0.130 | 0.149 | 0.151 | -0.129 | 0.153 | 0.598 | -0.239 | 0.270 | 0.334 | 0.055 | -0.116 | -0.363 | -0.837 | -0.185 |
| PermProc | 0.179 | 0.181 | 0.272 | -0.180 | 0.062 | 0.365 | -0.055 | 0.131 | -0.245 | -0.038 | 0.018 | 0.205 | 0.374 | 0.095 |
| ProcDist | 0.102 | 0.137 | 0.385 | -0.050 | 0.272 | -0.053 | 0.016 | 0.082 | -0.076 | 0.108 | -0.102 | -0.065 | -0.065 | 0.081 |
| RSA | 0.116 | 0.179 | 0.092 | -0.184 | -0.187 | -0.109 | -0.202 | 0.193 | 0.334 | 0.103 | -0.050 | 0.042 | 0.001 | -0.045 |
| RSMDiff | -0.414 | -0.217 | 0.301 | -0.270 | 0.066 | 0.020 | -0.282 | -0.174 | -0.195 | 0.179 | -0.009 | -0.030 | -0.209 | -0.174 |
| RankSim | -0.147 | -0.002 | 0.215 | 0.013 | 0.050 | 0.251 | -0.087 | -0.104 | -0.054 | 0.018 | 0.015 | -0.319 | 0.140 | -0.328 |
| SVCCA | 0.213 | -0.001 | 0.249 | -0.113 | 0.164 | 0.047 | 0.182 | 0.391 | 0.075 | 0.135 | 0.034 | 0.010 | -0.055 | 0.075 |
| SoftCorr | 0.453 | 0.270 | 0.108 | -0.040 | -0.165 | 0.013 | -0.305 | 0.469 | -0.125 | -0.073 | -0.033 | -0.289 | 0.273 | -0.164 |
| UnifDiff | -0.338 | nan | nan | 0.036 | -0.166 | nan | nan | -0.019 | nan | nan | 0.169 | 0.387 | nan | nan |

**Table 16: Vision - Test 3: Label Randomization. Full results of the similarity measures for distinguishing model groups trained with differing degrees of randomized labels. Curiously the ViT L32 expresses a significant amount of NaNs in the similarity measure.**
| Test | Label Randomization |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Measure | AUPRC | Conformity Rate |  |  |  |  |  |  |  |  |  |  |  |  |
| Architecture | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViT B32 | ViT L32 | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViT B32 | ViT L32 |
| 2nd-Cos | 0.712 | 0.707 | 0.705 | 0.699 | 0.701 | 0.537 | 0.690 | 0.212 | 0.247 | 0.257 | 0.333 | 0.333 | 0.212 | 0.333 |
| AlignCos | 0.724 | 0.724 | 0.849 | 0.724 | 0.457 | 0.579 | 1.000 | 0.167 | 0.167 | 0.058 | 0.167 | 0.453 | 0.246 | 0.000 |
| AngShape | 0.724 | 0.724 | 1.000 | 1.000 | 0.802 | 0.546 | 0.913 | 0.167 | 0.167 | 0.000 | 0.000 | 0.180 | 0.302 | 0.056 |
| CKA | 1.000 | 1.000 | 1.000 | 1.000 | 0.765 | 0.556 | 0.891 | 0.000 | 0.000 | 0.000 | 0.000 | 0.246 | 0.281 | 0.113 |
| ConcDiff | 0.989 | 1.000 | 1.000 | 1.000 | 0.573 | 0.731 | 1.000 | 0.003 | 0.000 | 0.000 | 0.000 | 0.161 | 0.100 | 0.000 |
| DistCorr | 1.000 | 1.000 | 1.000 | 1.000 | 0.768 | 0.588 | 0.904 | 0.000 | 0.000 | 0.000 | 0.000 | 0.233 | 0.245 | 0.097 |
| EOS | 0.837 | 0.724 | 0.703 | 0.553 | 0.495 | 0.623 | 0.701 | 0.047 | 0.166 | 0.271 | 0.245 | 0.413 | 0.213 | 0.250 |
| GULP | 0.893 | 0.725 | 0.649 | 0.851 | 0.438 | 0.468 | 0.655 | 0.030 | 0.162 | 0.068 | 0.051 | 0.376 | 0.171 | 0.074 |
| HardCorr | 0.724 | 0.724 | 0.724 | 1.000 | 0.599 | 0.532 | 0.906 | 0.167 | 0.167 | 0.167 | 0.000 | 0.327 | 0.318 | 0.059 |
| IMD | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 0.726 | 0.801 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.226 | 0.200 |
| Jaccard | 1.000 | 1.000 | 1.000 | 1.000 | 0.732 | 0.496 | 0.832 | 0.000 | 0.000 | 0.000 | 0.000 | 0.300 | 0.289 | 0.144 |
| LinReg | nan | nan | nan | nan | nan | 0.338 | 0.478 | nan | nan | nan | nan | nan | 0.501 | 0.276 |
| MagDiff | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 0.712 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.181 |
| OrthProc | 0.724 | 0.724 | 1.000 | 1.000 | 0.802 | 0.546 | 0.913 | 0.167 | 0.167 | 0.000 | 0.000 | 0.180 | 0.302 | 0.056 |
| PermProc | 0.699 | 0.699 | 0.706 | 1.000 | 0.724 | 0.423 | 0.704 | 0.333 | 0.333 | 0.253 | 0.000 | 0.167 | 0.500 | 0.222 |
| ProcDist | 0.724 | 0.708 | 0.729 | 1.000 | 0.724 | 0.699 | 0.751 | 0.167 | 0.239 | 0.147 | 0.000 | 0.167 | 0.333 | 0.231 |
| RSA | 0.749 | 0.724 | 1.000 | 1.000 | 0.762 | 0.491 | 0.857 | 0.111 | 0.167 | 0.000 | 0.000 | 0.246 | 0.352 | 0.128 |
| RSMDiff | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 0.752 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.222 |
| RankSim | 1.000 | 1.000 | 1.000 | 0.983 | 0.732 | 0.746 | 0.769 | 0.000 | 0.000 | 0.000 | 0.007 | 0.285 | 0.188 | 0.161 |
| SVCCA | 1.000 | 0.936 | 0.952 | 0.897 | 0.784 | 0.524 | 0.576 | 0.000 | 0.030 | 0.042 | 0.105 | 0.112 | 0.285 | 0.231 |
| SoftCorr | 0.724 | 0.724 | 0.724 | 0.846 | 0.458 | 0.519 | 0.906 | 0.167 | 0.167 | 0.167 | 0.041 | 0.435 | 0.375 | 0.059 |
| UnifDiff | 0.422 | 0.415 | 0.524 | 0.750 | 0.574 | 0.524 | 0.201 | 0.380 | 0.433 | 0.333 | 0.100 | 0.190 | 0.333 | 0.681 |

**Table 17: Vision - Test 4: Shortcut Affinity. Full results of the similarity measures for distinguishing model groups trained with shortcuts of various correlation to the image label.**
| Test | Shortcut Affinity |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Measure | AUPRC | Conformity Rate |  |  |  |  |  |  |  |  |  |  |  |  |
| Architecture | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViT B32 | ViT L32 | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViT B32 | ViT L32 |
| 2nd-Cos | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | nan | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | nan |
| AlignCos | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | nan | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | nan |
| AngShape | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | nan | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | nan |
| CKA | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | nan | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | nan |
| ConcDiff | 0.533 | 0.696 | 0.498 | 0.784 | 0.267 | 0.238 | nan | 0.173 | 0.139 | 0.187 | 0.053 | 0.334 | 0.378 | nan |
| DistCorr | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | nan | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | nan |
| EOS | 1.000 | 1.000 | 0.994 | 0.950 | 0.878 | 0.917 | nan | 0.000 | 0.000 | 0.001 | 0.019 | 0.042 | 0.014 | nan |
| GULP | 1.000 | 1.000 | 1.000 | 0.956 | 0.883 | 1.000 | nan | 0.000 | 0.000 | 0.000 | 0.017 | 0.040 | 0.000 | nan |
| HardCorr | 0.971 | 1.000 | 0.994 | 0.925 | 0.909 | 0.972 | nan | 0.010 | 0.000 | 0.001 | 0.027 | 0.017 | 0.005 | nan |
| IMD | 0.675 | 0.748 | 0.548 | 0.778 | 0.647 | 0.383 | 0.336 | 0.125 | 0.121 | 0.270 | 0.074 | 0.235 | 0.260 | 0.335 |
| Jaccard | 1.000 | 0.997 | 1.000 | 1.000 | 1.000 | 1.000 | nan | 0.000 | 0.001 | 0.000 | 0.000 | 0.000 | 0.000 | nan |
| LinReg | 0.985 | 1.000 | 1.000 | 1.000 | 0.979 | 0.462 | nan | 0.005 | 0.000 | 0.000 | 0.000 | 0.009 | 0.117 | nan |
| MagDiff | 0.374 | 0.368 | 0.444 | 0.526 | 0.227 | 0.231 | nan | 0.379 | 0.246 | 0.207 | 0.147 | 0.490 | 0.388 | nan |
| OrthProc | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | nan | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | nan |
| PWCCA | 0.994 | 1.000 | 0.648 | 0.990 | 0.753 | nan | 0.932 | 0.001 | 0.000 | 0.286 | 0.003 | 0.096 | nan | 0.034 |
| PermProc | 0.718 | 0.796 | 0.940 | 0.658 | 0.817 | 1.000 | nan | 0.107 | 0.061 | 0.014 | 0.130 | 0.068 | 0.000 | nan |
| ProcDist | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | nan | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | nan |
| RSA | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | nan | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | nan |
| RSMDiff | 0.572 | 0.425 | 0.592 | 0.872 | 0.586 | 0.506 | nan | 0.187 | 0.289 | 0.183 | 0.027 | 0.197 | 0.166 | nan |
| RankSim | 0.990 | 0.994 | 1.000 | 0.997 | 1.000 | 1.000 | nan | 0.003 | 0.001 | 0.000 | 0.001 | 0.000 | 0.000 | nan |
| SVCCA | 0.554 | 0.681 | 0.515 | 0.683 | 0.287 | 0.479 | nan | 0.191 | 0.159 | 0.193 | 0.087 | 0.379 | 0.290 | nan |
| SoftCorr | 0.974 | 0.984 | 0.989 | 0.898 | 0.835 | 1.000 | nan | 0.009 | 0.003 | 0.003 | 0.031 | 0.042 | 0.000 | nan |
| UnifDiff | 0.750 | 0.734 | 0.866 | 0.596 | 0.546 | 0.734 | nan | 0.100 | 0.087 | 0.043 | 0.169 | 0.167 | 0.117 | nan |

**Table 18: Vision - Test 5: Augmentation. Full results of the similarity measures for distinguishing model groups based on different additive gaussian noise augmentation used during training.**
| Test | Augmentation |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Measure | AUPRC | Conformity Rate |  |  |  |  |  |  |  |  |  |  |  |  |
| Architecture | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViT B32 | ViT L32 | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViT B32 | ViT L32 |
| 2nd-Cos | 0.783 | 0.879 | 0.914 | 0.763 | 0.803 | 0.583 | 0.837 | 0.049 | 0.024 | 0.019 | 0.065 | 0.059 | 0.152 | 0.061 |
| AlignCos | 0.709 | 0.838 | 0.903 | 1.000 | 0.807 | 0.757 | 0.745 | 0.061 | 0.036 | 0.035 | 0.000 | 0.050 | 0.079 | 0.062 |
| AngShape | 0.712 | 0.515 | 0.559 | 0.990 | 0.737 | 0.741 | 0.801 | 0.095 | 0.194 | 0.175 | 0.002 | 0.065 | 0.087 | 0.041 |
| CKA | 0.899 | 0.730 | 0.674 | 0.994 | 0.754 | 0.375 | 0.713 | 0.025 | 0.075 | 0.113 | 0.001 | 0.052 | 0.272 | 0.062 |
| ConcDiff | 0.426 | 0.960 | 0.663 | 1.000 | 0.297 | 0.406 | 0.276 | 0.247 | 0.009 | 0.086 | 0.000 | 0.331 | 0.256 | 0.491 |
| DistCorr | 0.828 | 0.677 | 0.619 | 0.994 | 0.779 | 0.385 | 0.703 | 0.043 | 0.104 | 0.133 | 0.001 | 0.044 | 0.259 | 0.064 |
| EOS | 0.929 | 0.590 | 0.690 | 0.983 | 0.875 | 0.600 | 0.716 | 0.017 | 0.141 | 0.107 | 0.005 | 0.039 | 0.147 | 0.123 |
| GULP | 0.924 | 0.598 | 0.692 | 0.985 | 0.878 | 0.445 | 0.719 | 0.018 | 0.137 | 0.105 | 0.004 | 0.039 | 0.147 | 0.121 |
| HardCorr | 0.456 | 0.443 | 0.504 | 0.587 | 0.434 | 0.424 | 0.643 | 0.291 | 0.325 | 0.239 | 0.160 | 0.177 | 0.220 | 0.137 |
| IMD | 0.563 | 0.653 | 0.644 | 0.782 | 0.753 | 0.234 | 0.211 | 0.181 | 0.099 | 0.115 | 0.057 | 0.077 | 0.479 | 0.500 |
| Jaccard | 0.792 | 0.724 | 0.857 | 0.947 | 0.807 | 0.719 | 0.808 | 0.053 | 0.075 | 0.050 | 0.015 | 0.059 | 0.087 | 0.069 |
| LinReg | 0.941 | 0.640 | 0.751 | 1.000 | 0.735 | 0.365 | 0.849 | 0.017 | 0.098 | 0.078 | 0.000 | 0.057 | 0.346 | 0.059 |
| MagDiff | 0.366 | 0.807 | 0.457 | 0.988 | 0.223 | 0.426 | 0.180 | 0.261 | 0.055 | 0.209 | 0.003 | 0.457 | 0.255 | 0.541 |
| OrthProc | 0.712 | 0.515 | 0.559 | 0.990 | 0.737 | 0.741 | 0.801 | 0.095 | 0.194 | 0.175 | 0.002 | 0.065 | 0.087 | 0.041 |
| PWCCA | 0.899 | 0.568 | 0.509 | 0.958 | 0.735 | nan | 0.693 | 0.028 | 0.135 | 0.211 | 0.011 | 0.078 | nan | 0.107 |
| PermProc | 0.405 | 0.335 | 0.537 | 0.630 | 0.549 | 0.279 | 0.614 | 0.358 | 0.427 | 0.153 | 0.129 | 0.207 | 0.352 | 0.145 |
| ProcDist | 0.581 | 0.447 | 0.673 | 0.955 | 0.792 | 0.806 | 0.906 | 0.180 | 0.351 | 0.105 | 0.011 | 0.065 | 0.075 | 0.026 |
| RSA | 0.981 | 0.821 | 0.784 | 0.754 | 0.461 | 0.338 | 0.593 | 0.005 | 0.047 | 0.079 | 0.072 | 0.191 | 0.287 | 0.117 |
| RSMDiff | 0.446 | 0.468 | 0.547 | 0.566 | 0.579 | 0.240 | 0.322 | 0.215 | 0.210 | 0.215 | 0.153 | 0.129 | 0.428 | 0.302 |
| RankSim | 0.714 | 0.567 | 0.889 | 0.829 | 0.802 | 0.510 | 0.652 | 0.081 | 0.128 | 0.043 | 0.047 | 0.077 | 0.176 | 0.127 |
| SVCCA | 0.400 | 0.583 | 0.473 | 0.779 | 0.472 | 0.207 | 0.286 | 0.287 | 0.221 | 0.317 | 0.084 | 0.295 | 0.481 | 0.392 |
| SoftCorr | 0.451 | 0.445 | 0.500 | 0.533 | 0.474 | 0.433 | 0.645 | 0.309 | 0.349 | 0.249 | 0.189 | 0.147 | 0.213 | 0.137 |
| UnifDiff | 0.167 | 0.181 | 0.221 | 0.176 | 0.214 | 0.167 | 0.167 | 1.000 | 0.551 | 0.489 | 0.735 | 0.577 | 1.000 | 1.000 |

**Table 19: Vision - Test 6: Monotonicity. Full results of the similarity measures for the monotonicity test.**
| Test | Augmentation |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Measure | Spearman | Conformity Rate |  |  |  |  |  |  |  |  |  |  |  |  |
| Architecture | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViT B32 | ViT L32 | RNet18 | RNet34 | RNet101 | VGG11 | VGG19 | ViT B32 | ViT L32 |
| 2nd-Cos | 0.550 | 0.775 | 0.922 | 1.000 | 1.000 | 1.000 | 1.000 | 0.150 | 0.087 | 0.053 | 0.000 | 0.000 | 0.000 | 0.000 |
| AlignCos | 0.517 | 0.627 | 0.525 | 0.933 | 0.115 | 1.000 | 1.000 | 0.155 | 0.135 | 0.200 | 0.100 | 0.315 | 0.000 | 0.000 |
| AngShape | 0.550 | 0.650 | 0.650 | 0.955 | 0.988 | 1.000 | 1.000 | 0.150 | 0.100 | 0.100 | 0.030 | 0.015 | 0.000 | 0.000 |
| CKA | 0.867 | 0.815 | 1.000 | 0.883 | 0.927 | 1.000 | 1.000 | 0.065 | 0.025 | 0.000 | 0.100 | 0.080 | 0.000 | 0.000 |
| ConcDiff | -0.783 | -0.050 | -0.140 | -0.247 | -0.274 | 0.650 | 1.000 | 0.800 | 0.537 | 0.540 | 0.685 | 0.570 | 0.100 | 0.000 |
| DistCorr | 0.967 | 0.794 | 0.997 | 0.883 | 0.932 | 1.000 | 1.000 | 0.050 | 0.022 | 0.003 | 0.100 | 0.077 | 0.000 | 0.000 |
| EOS | 0.883 | 0.963 | 0.970 | 1.000 | 1.000 | 1.000 | 1.000 | 0.100 | 0.050 | 0.043 | 0.000 | 0.000 | 0.000 | 0.000 |
| GULP | 0.533 | 0.475 | 0.638 | 1.000 | 1.000 | 1.000 | 1.000 | 0.300 | 0.300 | 0.200 | 0.000 | 0.000 | 0.000 | 0.000 |
| HardCorr | 0.007 | 0.526 | 0.759 | 0.908 | 0.735 | 1.000 | 1.000 | 0.390 | 0.150 | 0.110 | 0.085 | 0.070 | 0.000 | 0.000 |
| IMD | -0.003 | 0.025 | nan | -0.032 | 0.067 | 0.591 | 0.385 | 0.485 | 0.456 | nan | 0.520 | 0.495 | 0.135 | 0.458 |
| Jaccard | 0.550 | 0.650 | 0.750 | 1.000 | 1.000 | 1.000 | 1.000 | 0.150 | 0.100 | 0.090 | 0.000 | 0.000 | 0.000 | 0.000 |
| LinReg | 0.550 | 0.951 | 0.903 | 0.482 | 0.775 | 0.996 | 1.000 | 0.150 | 0.060 | 0.113 | 0.175 | 0.087 | 0.005 | 0.000 |
| MagDiff | -0.367 | 0.132 | 0.135 | 0.207 | 0.285 | 0.839 | 1.000 | 0.650 | 0.545 | 0.455 | 0.355 | 0.347 | 0.145 | 0.000 |
| OrthProc | 0.550 | 0.650 | 0.650 | 0.955 | 0.988 | 1.000 | 1.000 | 0.150 | 0.100 | 0.100 | 0.030 | 0.015 | 0.000 | 0.000 |
| PWCCA | 0.107 | 0.300 | 1.000 | 0.317 | 0.552 | 1.000 | 1.000 | 0.175 | 0.107 | 0.044 | 0.250 | 0.167 | 0.000 | 0.000 |
| PermProc | 0.203 | 0.605 | 0.392 | 0.690 | 0.145 | 0.712 | 1.000 | 0.370 | 0.295 | 0.388 | 0.290 | 0.412 | 0.300 | 0.000 |
| ProcDist | 0.550 | 0.425 | 0.391 | 0.483 | 0.667 | 0.712 | 1.000 | 0.150 | 0.200 | 0.210 | 0.200 | 0.195 | 0.300 | 0.000 |
| RSA | 0.967 | 0.724 | 0.876 | 0.580 | 0.660 | 1.000 | 1.000 | 0.050 | 0.057 | 0.028 | 0.195 | 0.095 | 0.000 | 0.000 |
| RSMDiff | -0.333 | -0.087 | -0.129 | 0.850 | 0.645 | 0.750 | 1.000 | 0.550 | 0.500 | 0.501 | 0.150 | 0.305 | 0.250 | 0.000 |
| RankSim | 0.550 | 0.650 | 0.675 | 1.000 | 1.000 | 1.000 | 1.000 | 0.150 | 0.100 | 0.098 | 0.000 | 0.000 | 0.000 | 0.000 |
| SVCCA | 0.203 | 0.266 | 0.526 | 0.423 | 0.399 | 0.868 | 0.610 | 0.280 | 0.420 | 0.233 | 0.250 | 0.310 | 0.140 | 0.220 |
| SoftCorr | 0.110 | 0.505 | 0.697 | 0.517 | 0.645 | 1.000 | 1.000 | 0.285 | 0.148 | 0.125 | 0.200 | 0.107 | 0.000 | 0.000 |
| UnifDiff | 0.183 | 0.204 | 0.552 | -0.303 | -0.061 | nan | nan | 0.350 | 0.325 | 0.188 | 0.615 | 0.505 | 0.000 | 0.000 |

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: * What happens with different architectures for the text modality (e.g. some LLMs)?
* What is "IN21k" in line 282? Please write out the full name instead of using the acronym, as it’s unclear without a reference to the actual name.
soundness: 3
strengths: * The paper is well-written and easy to follow, with experiments clearly presented and tasks well-defined. Figures and tables are self-explanatory, significantly enhancing the manuscript's readability and accessibility.

* The authors have provided the code, ensuring full reproducibility of results and facilitating the community's ability to expand and build upon the benchmark. This openness supports transparency and encourages collaborative advancement in the field.

* The problem addressed is highly relevant and important for the community, as it remains an open question which metric is best suited for different scenarios.
summary: This work presents a comprehensive benchmark for evaluating 23 representational similarity measures across six similarity tests, twelve different architectures, and six datasets from diverse domains. Additionally, the paper offers insights for future research and aims to democratize a unified framework that ensures fair comparisons across different studies.
weaknesses: * It’s unclear whether the similarities are calculated between two latent spaces (as suggested in formula 3, line 094) or between two individual representations (a single point), as written in line 095. Clarifying this distinction (and how each metric is adapted accordingly) could improve the reader's understanding. It seems to be between two latent spaces, but in Figure 2 (left), the metric is calculated between two points.

* When evaluating the text architectures:
  * Could you clarify the choice to test the same model with 25 different seeds, rather than exploring architectural variations like RoBERTa or ALBERT, which are available pretrained on Hugging Face? Including a wider variety of architectures might enhance the robustness and generalizability of the findings.
  * It would be helpful to either move the explanation of the choice to use the CLS token to the main manuscript or to reference the appendix. Additionally, you might consider testing alternative aggregation modalities, such as the mean or the last token, since these can yield different representations and may impact the results.

* The vision task may benefit from broader testing since only ImageNet with 100 subclasses is used. Expanding the evaluation to include the full ImageNet dataset or other widely-used datasets, such as ImageNet-1k, CIFAR-10, or CIFAR-100, could provide a more comprehensive assessment.

* The analysis could benefit from stronger takeaways. For instance, in lines 438-439, you suggest that the choice of measures should be task-specific, and in lines 453-455, you advocate for the development of best practices for using similarity metrics. However, these insights feel somewhat broad, and more concrete guidance could help readers understand how to apply the benchmark effectively.

* In line 413, it’s mentioned that some domain-specific trends can be identified, yet later (line 424), you note that no single measure consistently outperforms others across all tests, even within a single dataset. The first claim—that some metrics appear more suitable for specific modalities—is interesting and would benefit from a more in-depth discussion. Also, adding a summary table of these findings of the paper could help readers better understand the details.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 4
questions: - Should Eq. 3 map to R^N?
- It did not become clear initially what type of accuracy measures are considered; e.g. what task the model is trained to solved?
- I think coloring the labels of the similarity measures in Table 3 may be helpful for the reader (similar to the coloring in Fig. 3)
soundness: 3
strengths: - The challenge of understanding learned representations better, especially in an unsupervised way is highly relevant and of practical importance.
- Reproducible and extensive experiments covering a wide range of metrics, domains and tasks.
- The paper is overall well-written and easy to follow. The figures and findings are clearly presented.
summary: This paper presents the ReSi benchmark, a framework for assessing representational similarity measures across neural architectures from different domains. It features 23 similarity measures, twelve neural network architectures, and various datasets from graph, language, and vision. ReSi is proposed to support research on representation learning, reproducibility, and exploration of new approaches for comparing neural representations.
weaknesses: - I think the paper misses the opportunity to give the reader a better understanding of what drives the differences and inconsistent results of measuring similarity. Which types of measures perform well together and what feature of the representation is this driven by. Relevant aspects could be due to the way representations are normalized, what arithmetic function is used for computing the distance (e.g. dot-product, summed differences, etc.)
- Another angle to better understand similarity computations is from the point of interpretability, i.e. what features do the respective models pick up on and how do these change across models. I think this is an important and complementary direction to measuring different similarity scores and should be discussed (cf. [1,2,3]). 
- ReSi is presented as the “the first comprehensive benchmark for representational similarity” and can “yield rich insights” and a “theoretical understanding of representational similarity”. While I did find the general guidelines interesting and helpful, I think a deeper understanding of what causes these differences across metrics and domains is missing, and thus the above statements overstate the level of achievable insights from ReSi for me.


[1] Eberle et al. “Building and interpreting deep similarity models”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(3), 2020.

[2] Janizek et al. "Explaining explanations: Axiomatic feature interactions for deep networks", CoRR, 2020.

[3] Vasileiou et al. “Explaining Text Similarity in Transformer Models”, NAACL 2024.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 3
questions: Q1. Which layers are being used in the vision model experiments? I find that this information is not clearly provided in Sec. A.5 Vision.
soundness: 3
strengths: (S1) The goal of the paper is well-motivated and could be a valuable contribution to the community, since it is not clear how to compare representational similarity methods. 

(S2) The problem is quite challenging as it is hard to know exactly how two models differ, therefore the authors take reasonable steps to develop tasks in which they can approximately group models by differences in their performance or training method. 

(S3) The authors apply their benchmarking method on a comprehensive set of representational similarity methods in three popular domains in machine learning. 

(S4) The authors provide some interesting insights into differences between similarity methods and show that no single similarity method always performs the best.
summary: This paper proposes a suite of benchmarking tasks to test the quality of a representational similarity metric. The tasks are designed around training models such that they are different in one specific way that leads to a certain expectation for similarity. Specifically, the authors propose 6 tasks:

(T1): Correlating similarity vs accuracy. Models are trained with only varied seeds.
(T2): Correlating similarity to actual model outputs.  Models are trained with only varied seeds.
(T3): Label Randomization: Randomizing different percentages of the labels. Models are expected to be more similar within the same percentage of randomization.
(T4): Shortcut affinity: Models are trained with random "shortcuts" that leak the correct label. The shortcut that always leaks the correct label, sometimes leaks the correct label and never leaks the correct label. Models should be more similar within group. 
(T5): Augmentation: models are trained with different kinds of augmentations adn models are expected to be more similar within the same type of augmentations. 
(T6): Layer Monotonicity: The similarity between layers is expected to be related to the distance between the layers. 

They apply their benchmarking tasks to three domains, graphs, language and vision and test 23 different representational similarity methods. 

The authors find that no similarity metric is always the best according to these metrics, however, there are some that tend to perform better than others, especially within a specific sub-domain (graph, language, or vision).
weaknesses: (W1) Task 6 - I find that there are some careful considerations to use task 6 (see Q1). If the layers are chosen incorrectly, task 6 would have some issues for at least some vision models since skip connections could make neighboring layers dissimilar. Correcting/explaining this issue would shift my score to a 6. 

(W2) I found the depth of analysis to be somewhat limited. The authors could significantly strengthen the paper by providing some deeper analysis into a few similarity methods that explains how the similarity methods arrive at different similarity scores when comparing the same representations. For example, the authors show the impact of preprocessing by comparing two similarity methods that differ only in preprocessing: Orthogonal Procrustes (OrthProc) and Procrustes Size-and-Shape Distance (ProcDist). Can the authors analyze what the preprocessing does to the representation and how it changes the results of the method on the benchmarks? If this analysis and/or others are provided, I would consider improving the rating further. 

Minor ----
(W3) Section 3.1 should indicate more details of the tasks will be provided in Sec 3.5.

---

confidence: 4
contribution: 4
initial_rating: 8
presentation: 4
questions: 1. Can you please clarify notations at Figure 3? Does boxes and bars mean quantiles?
soundness: 4
strengths: 1. Novelty. To the best of my knowledge, authors have introduced the first representational similarity bechmark.
2. Inside the benchmark, several rarely used similarity measures were tested (like Jaccard similarity), however, they showed promising results.
3. An in-depth analysis is provided; cases where some similarity measures tend to work well as studied.
4. The benchmark is quite diverse: in includes datasets from spanning graph, language, and vision domains.
5. The language of the paper is fine and it's easy to follow.
6. Impact. The comparison of neural representational similarity measures is a long-standing problem in deep learning.
summary: The paper introduces ReSi, a comprehensive bechmark for comparing neural representations.
The benchmark consists of (i) six carefully designed tests for similarity measures, (ii) 23 similarity measures, (iii) twelvecneural network architectures, and (iv) six datasets, spanning over the graph, language, and vision domains.
The benchmark is extensible; future research can build on it and further expand it.
The source code is publicly available.
weaknesses: 1. Authors cite (Barannikov et. al, 2022.) several times as an example of representation comparison measure and experimental methodology, but don't include the proposed measure (RTD) into comparisons.
2. Stitching (Bansal, 2021) is not included intro the study.
3. Some important details of experiments are not disclosed: for example, CKA could be evaluated on the whole dataset or batches. Other similarity measures probably include hyperparameters as well.
4. The study might benefit from an Appendix with a brief description of all similarity measures under evaluation.

Bansal, Y., Nakkiran, P., & Barak, B. (2021). Revisiting model stitching to compare neural representations. Advances in neural information processing systems, 34, 225-236.

Meta-review: This paper presents ReSi, described as the first comprehensive benchmark for evaluating representational similarity measures, including 6 carefully designed tests, 23 similarity measures, 12 neural architectures, and 6 datasets across graph, language and vision domains. The key claims are that ReSi enables systematic evaluation and comparison of representational similarity measures, with experimental results showing that no single measure consistently outperforms others across all tests and domains. The paper's strengths include strong practical relevance for the field, extensive experimental validation across multiple domains, good reproducibility through public code, and clear presentation. Initial weaknesses included: lack of clarity around calculation methods (between latent spaces vs individual points), limited architecture variety in text domain, restricted vision dataset testing, and insufficient depth in analysis of why different measures perform differently. During rebuttal, the authors made substantial improvements by adding ALBERT and LLM architectures, including CIFAR-100 dataset, clarifying technical details, and expanding analysis sections. The review scores (6, 6, 6, 8) and thorough responses to reviewer concerns, particularly around experimental breadth and technical clarity, support accepting this paper

Decision: Accept (Poster)