Generalized Shape Metrics on Neural Representations
Year: 2022

Abstract:
Understanding the operation of biological and artificial networks remains a difficult and important challenge. To identify general principles, researchers are increasingly interested in surveying large collections of networks that are trained on, or biologically adapted to, similar tasks. A standardized set of analysis tools is now needed to identify how network-level covariates—such as architecture, anatomical brain region, and model organism—impact neural representations (hidden layer activations). Here, we provide a rigorous foundation for these analyses by defining a broad family of metric spaces that quantify representational dissimilarity. Using this framework, we modify existing representational similarity measures based on canonical correlation analysis and centered kernel alignment to satisfy the triangle inequality, formulate a novel metric that respects the inductive biases in convolutional layers, and identify approximate Euclidean embeddings that enable network representations to be incorporated into essentially any off-the-shelf machine learning method. We demonstrate these methods on large-scale datasets from biology (Allen Institute Brain Observatory) and deep learning (NAS-Bench-101). In doing so, we identify relationships between neural representations that are interpretable in terms of anatomical features and model performance.

heading: 1 Introduction
text: The extent to which different deep networks or neurobiological systems use equivalent representations in support of similar task demands is a topic of persistent interest in machine learning and neuroscience [1–3]. Several methods including linear regression [4, 5], canonical correlation analysis (CCA; [6, 7]), representational similarity analysis (RSA; [8]), and centered kernel alignment (CKA; [9]) have been used to quantify the similarity of hidden layer activation patterns. These measures are often interpreted on an ordinal scale and are employed to compare a limited number of networks—e.g., they can indicate whether networks A and B are more or less similar than networks A and C. While these comparisons have yielded many insights [4–12], the underlying methodologies have not been extended to systematic analyses spanning thousands of networks.
To unify existing approaches and enable more sophisticated analyses, we draw on ideas from statistical shape analysis [13–15] to develop dissimilarity measures that are proper metrics—i.e., measures that are symmetric and respect the triangle inequality. This enables several off-the-shelf methods with theoretical guarantees for classification (e.g. k-nearest neighbors, [16]) and clustering (e.g. hierarchical clustering [17]). Existing similarity measures can violate the triangle inequality, which complicates these downstream analyses [18–20]. However, we show that existing dissimilarity
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
measures can often be modified to satisfy the triangle inequality and viewed as special cases of the framework we outline. We also describe novel metrics within this broader family that are specialized to convolutional layers and have appealing properties for analyzing artificial networks.
Moreover, we show empirically that these metric spaces on neural representations can be embedded with low distortion into Euclidean spaces, enabling an even broader variety of previously unconsidered supervised and unsupervised analyses. For example, we can use neural representations as the inputs to linear or nonlinear regression models. We demonstrate this approach on neural representations in mouse visual cortex (Allen Brain Observatory; [21]) in order to predict each brain region’s anatomical hierarchy from its pattern of visual responses—i.e., predicting a feature of brain structure from function. We demonstrate a similar approach to analyze hidden layer representations in a database of 432K deep artificial networks (NAS-Bench-101; [22]) and find a surprising degree of correlation between early and deep layer representations.
Overall, we provide a theoretical grounding which explains why existing representational similarity measures are useful: they are often close to metric spaces, and can be modified to fulfill metric space axioms precisely. Further, we draw new conceptual connections between analyses of neural representations and established research areas [15, 23], utilize these insights to propose novel metrics, and demonstrate a general-purpose machine learning workflow that scales to datasets with thousands of networks.

heading: 2 Methods
text: This section outlines several workflows (Fig. 1) to analyze representations across large collections of networks. After briefly summarizing prior approaches (sec. 2.1), we cover background material on metric spaces and discuss their theoretical advantages over existing dissimilarity measures (sec. 2.2). We then present a class of metrics that capture these advantages (sec. 2.3) and cover a special case that is suited to convolutional layers (sec. 2.4). We then demonstrate the practical advantages of these methods in Section 3, and demonstrate empirically that Euclidean feature spaces can approximate the metric structure of neural representations, enabling a broad set of novel analyses.

heading: 2.1 Prior work and problem setup
text: Neural network representations are often summarized over a set of m reference inputs (e.g. test set images). Let Xi ∈ Rm×ni and Xj ∈ Rm×nj denote the responses of two networks (with ni and nj neurons, respectively) to a collection of these inputs. Quantifying the similarity between Xi and Xj is complicated by the fact that, while the m inputs are the same, there is no direct correspondence between the neurons. Even if ni = nj , the typical Frobenius inner product, 〈Xi,Xj〉 = Tr[X>i Xj ], and metric, ‖Xi−Xj‖ = 〈Xi−Xj ,Xi−Xj〉1/2, fail to capture the desired notion of dissimilarity. For instance, let Π denote some n × n permutation matrix and let Xi = XjΠ. Intuitively, we should consider Xi and Xj to be identical in this case since the ordering of neurons is arbitrary. Yet, clearly ‖Xi −Xj‖ 6= 0, except in very special cases.
One way to address this problem is to linearly regress over the neurons to predict Xi from Xj . Then, one can use the coefficient of determination (R2) as a measure of similarity [4, 5]. However, this similarity score is asymmetric—if one instead treats Xj as the dependent variable that is predicted from Xi, this will result in a different R2. Canonical correlation analysis (CCA; [6, 7]) and linear centered kernel alignment (linear CKA; [9, 24]) also search for linear correspondences between neurons, but have the advantage of producing symmetric scores. Representational similarity analysis (RSA; [8]) is yet another approach, which first computes an m×m matrix holding the dissimilarities between all pairs of representations for each network. These representational dissimilarity matrices (RDMs), are very similar to the m ×m kernel matrices computed and compared by CKA. RSA traditionally quantifies the similarity between two neural networks by computing Spearman’s rank correlation between their RDMs. A very recent paper by Shahbazi et al. [25], which was published while this manuscript was undergoing review, proposes to use the Riemannian metric between positive definite matrices instead of Spearman correlation. Similar to our results, this establishes a metric space that can be used to compare neural representations. Here, we leverage metric structure over shape spaces [13–15] instead of positive definite matrices, leading to complementary insights.
In summary, there are a diversity of methods that one can use to compare neural representations. Without a unifying theoretical framework it is unclear how to choose among them, use their outputs for downstream tasks, or generalize them to new domains.

heading: 2.2 Feature space mapping, metrics, and equivalence relations
text: Our first contribution will be to establish formal notions of distance (metrics) between neural representations. To accommodate the common scenario when the number of neurons varies across networks (i.e. when ni 6= nj), we first map the representations into a common feature space. For each set of representations, Xi, we suppose there is a mapping into a p-dimensional feature space, Xi 7→ Xφi , where X φ i ∈ Rm×p. In the special case where all networks have equal size, n1 = n2 = . . . = n, we can express the feature mapping as a single function φ : Rm×n 7→ Rm×p, so that Xφi = φ(Xi). When networks have dissimilar sizes, we can map the representations into a common dimension using, for example, PCA [6].
Next, we seek to establish metrics within the feature space, which are distance functions that satisfy:
Equivalence: d(Xφi ,X φ j ) = 0 ⇐⇒ X φ i ∼X φ j (1)
Symmetry: d(Xφi ,X φ j ) = d(X φ j ,X φ i ) (2)
Triangle Inequality: d(Xφi ,X φ j ) ≤ d(X φ i ,X φ k) + d(X φ k ,X φ j ) (3)
for all Xφi , X φ j , and X φ k in the feature space. The symbol ‘∼’ denotes an equivalence relation between two elements. That is, the expression Xφi ∼ X φ j means that “X φ i is equivalent to X φ j .” Formally, distance functions satisfying Eqs. (1) to (3) define a metric over a quotient space defined by the equivalence relation and a pseudometric over Rm×p (see Supplement A). Intuitively, by specifying different equivalence relations we can account for symmetries in network representations, such as permutations over arbitrarily labeled neurons (other options are discussed below in sec. 2.3).
Metrics quantify dissimilarity in a way that agrees with our intuitive notion of distance. For example, Eq. (2) ensures that the distance from Xφi to X φ j is the same as the distance from X φ j to X φ i . Linear regression is an approach that violates this condition: the similarity measured by R2 depends on which network is treated as the dependent variable.
Further, Eq. (3) ensures that distances are self-consistent in the sense that if two elements (Xφi and Xφj ) are both close to a third (X φ k ), then they are necessarily close to each other. Many machine learning models and algorithms rely on this triangle inequality condition. For example, in clustering, it ensures that if Xφi and X φ j are put into the same cluster as X φ k , then X φ i and X φ j cannot be too far apart, thus implying that they too can be clustered together. Intuitively, this establishes an appealing transitive relation for clustering, which can be violated when the triangle inequality fails to hold. Existing measures based on CCA, RSA, and CKA, are symmetric, but do not satisfy the triangle inequality. By modifying these approaches to satisfy the triangle inequality, we avoid potential pitfalls and can leverage theoretical guarantees on learning in proper metric spaces [16–20].

heading: 2.3 Generalized shape metrics and group invariance
text: In this section, we outline a new framework to quantify representational dissimilarity, which leverages a well-developed mathematical literature on shape spaces [13–15]. The key idea is to treat Xφi ∼X φ j if and only if there exists a linear transformation T within a set of allowable transformations G, such that Xφi = X φ j T . Although G only contains linear functions, nonlinear alignments between the raw representations can be achieved when the feature mappings Xi 7→ Xφi are chosen to be nonlinear. Much of shape analysis literature focuses on the special case where p = n and G is the special orthogonal group SO(n) = {R ∈ Rn×n | R>R = I,det(R) = 1}, meaning that Xφi and X φ j are equivalent if there is a n-dimensional rotation (without reflection) that relates them. Standard shape analysis further considers each Xφi to be a mean-centered ((X φ i ) >1 = 0) and normalized (‖Xφi ‖ = 1) version of the raw landmark locations held in Xi ∈ Rm×n (an assumption that we will relax). That is, the feature map φ : Rm×n 7→ Sm×n transforms the raw landmarks onto the hypersphere, denoted Sm×n, of m × n matrices with unit Frobenius norm. In this context, Xφi ∈ Sm×n is called a “pre-shape.” By removing rotations from a pre-shape, [Xφi ] = {S ∈ Sm×n | S ∼X φ i } for pre-shape X φ i , we recover its “shape.”
To quantify dissimilarity in neural representations, we generalize this notion of shape to include other feature mappings and alignments. The minimal distance within the feature space, after optimizing over alignments, defines a metric under suitable conditions (Fig. 2A). This results in a broad variety of generalized shape metrics (see also, ch. 18 of [15]), which fall into two categories as formalized by the pair of propositions below. Proofs are provided in Supplement B.
Proposition 1. Let Xφi ∈ Rm×p, and let G be a group of linear isometries on Rm×p. Then,
d(Xφi ,X φ j ) = min T∈G ‖Xφi −X φ j T ‖ (4)
defines a metric, where Xφi ∼X φ j if and only if there is a T ∈ G such that X φ i = X φ j T .
Proposition 2. Let Xφi ∈ Sm×p, and let G be a group of linear isometries on Sm×p. Then,
θ(Xφi ,X φ j ) = min
T∈G arccos
〈 Xφi ,X φ j T 〉
(5)
defines a metric, where Xφi ∼X φ j if and only if there is a T ∈ G such that X φ i = X φ j T .
Two key conditions appear in these propositions. First, G must be a group of functions. This means G is a set that contains the identity function, is closed under composition (T 1T 2 ∈ G for any T 1 ∈ G and T 2 ∈ G), and whose elements are invertible by other members of the set (if T ∈ G then T−1 ∈ G).
Second, every T ∈ G must be an isometry, meaning that ‖Xφi −X φ j ‖ = ‖X φ i T −X φ j T ‖ for all T ∈ G and all elements of the feature space. On Rm×p and Sm×p, all linear isometries are orthogonal transformations. Further, the set of orthogonal transformations, O(p) = {Q ∈ Rp×p : Q>Q = I}, defines a well-known group. Thus, the condition that G is a group of isometries is equivalent to G being a subgroup of O(p)—i.e., a subset of O(p) satisfying the group axioms. Intuitively, by requiring G to be a group of functions, we ensure that the alignment procedure is symmetric—i.e. it is equivalent to transform Xφi to match X φ j , or transform the latter to match the former. Further, by requiring each T ∈ G to be an isometry, we ensure that the underlying metric (Euclidean distance for Proposition 1; angular distance for Proposition 2) preserves its key properties.
Together, these propositions define a broad class of metrics as we enumerate below. For simplicity, we assume that ni = nj = n in the examples below, with the understanding that a PCA or zero-padding preprocessing step has been performed in the case of dissimilar network sizes. This enables us to express the metrics as functions of the raw activations, i.e. functions Rm×n × Rm×n 7→ R+.
Permutation invariance The most stringent notion of representational similarity is to demand that neurons are one-to-one matched across networks. If we set the feature map to be the identity function, i.e., Xφi = Xi for all i, then:
dP(Xi,Xj) = min Π∈P(n)
‖Xi −XjΠ‖ (6)
defines a metric by Proposition 1 since the set of permutation matrices, P(n), is a subgroup of O(n). To evaluate this metric we must optimize over the set of neuron permutations to align the two networks. This can be reformulated (see Supplement C) as a fundamental problem in combinatorial optimization known as the linear assignment problem [26]. Exploiting an algorithm due to Jonker and Volgenant [27, 28] we can solve this problem in O(n3) time. The overall runtime for evaluating Eq. (6) is O(mn2 + n3), since we must evaluate X>i Xj to formulate the assignment problem.
Rotation invariance Let C = Im − (1/m)11> denote an m×m centering matrix, and consider the feature mapping φ1 which mean-centers the columns, φ1(Xi) = X φ1 i = CXi. Then,
d1(Xi,Xj) = min Q∈O
‖Xφ1i −X φ1 j Q‖ (7)
defines a metric by Proposition 1, and is equivalent to the Procrustes size-and-shape distance with reflections [15]. Further, by Proposition 2,
θ1(Xi,Xj) = min Q∈O
arccos 〈Xφ1i ,X φ1 j Q〉
‖Xφ1i ‖‖X φ1 j ‖
(8)
defines another metric, and is closely related to the Riemannian distance on Kendall’s shape space [15]. To evaluate Eqs. (7) and (8), we must optimize over the set of orthogonal matrices to find the best alignment. This also maps onto a fundamental optimization problem known as the orthogonal Procrustes problem [29, 30], which can be solved in closed form inO(n3) time. As in the permutationinvariant metric described above, the overall runtime is O(mn2 + n3).
Linear invariance Consider a partial whitening transformation, parameterized by 0 ≤ α ≤ 1:
Xφα = CX(αIn + (1− α)(X>CX)−1/2) (9)
Note that X>CX is the empirical covariance matrix of X . Thus, when α = 0, Eq. (9) corresponds to ZCA whitening [31], which intuitively removes invertible linear transformations from the representations. Thus, when α = 0 the metric outlined below treats Xi ∼ Xj if there exists an affine transformation that relates them: Xi = XjW + b for some W ∈ Rn×n and b ∈ Rn. When α = 1, Eq. (9) reduces to the mean-centering feature map used above.
Using orthogonal alignments within this feature space leads to a metric that is related to CCA. First, let ρ1 ≥ . . . ≥ ρn ≥ 0 denote the singular values of (Xφαi )>(X φα j )/‖X φα i ‖‖X φα j ‖. One can show that
θα(Xi,Xj) = min Q∈O
arccos 〈Xφαi ,X φα j Q〉
‖Xφαi ‖‖X φα j ‖
= arccos( ∑ ` ρ`) , (10)
and we can see from Proposition 2 that this defines a metric for any 0 ≤ α ≤ 1. When α = 0, the values ρ1, . . . , ρn are proportional to the canonical correlation coefficients, with 1/n being the factor of proportionality. When α > 0, these values can be viewed as ridge regularized canonical correlation coefficients [32]. See Supplement C for further details. Past works [6, 7] have used the average canonical correlation as a measure of representational similarity. When α = 0, the average canonical correlation is given by ∑ ` ρ` = cos θ0(Xi,Xj). Thus, if we apply arccos(·) to the average canonical correlation, we modify the calculation to produce a proper metric (see Fig. 4A). Since the covariance is often ill-conditioned or singular in practice, setting α > 0 to regularize the calculation is also typically necessary.
Nonlinear invariances We discuss feature maps that enable nonlinear notions of equivalence, and which relate to kernel CCA [33] and CKA [9], in Supplement C.

heading: 2.4 Metrics for convolutional layers
text: In deep networks for image processing, each convolutional layer produces a h × w × c array of activations, whose axes respectively correspond to image height, image width, and channels (number of convolutional filters). If stride-1 circular convolutions are used, then applying a circular shift along either spatial dimension produces the same shift in the layer’s output. It is natural to reflect this property, known as translation equivariance [23], in the equivalence relation on layer representations. Supposing that the feature map preserves the shape of the activation tensor, we have Xφk ∈ Rm×h×w×c for neural networks indexed by k ∈ 1, . . . ,K. Letting S(n) denote the group of n-dimensional circular shifts (a subgroup of the permutation group) and ‘⊗’ denote the Kronecker product, we propose:
Xφi ∼X φ j ⇐⇒ vec(X φ i ) = (I ⊗ S1 ⊗ S2 ⊗Q)vec(X φ j ) (11)
for some S1 ∈ S(h), S2 ∈ S(w), Q ∈ O(c), as the desired equivalence relation. This relation allows for orthogonal invariance across the channel dimension but only shift invariance across the spatial dimensions. The mixed product property of Kronecker products, (A⊗B)(C ⊗D) = AB ⊗CD, ensures that the overall transformation maintains the group structure and remains an isometry. Figure 2B uses a toy dataset (stacked MNIST digits) to show that this metric is sensitive to differences in spatial activation patterns, but insensitive to coherent spatial translations across channels. In contrast, metrics that ignore the convolutional structure (as in past work [6, 9]) treat very different spatial patterns as identical representations.
Evaluating Eq. (11) requires optimizing over spatial shifts in conjuction with solving a Procrustes alignment. If we fit the shifts by an exhaustive brute-force search, the overall runtime isO(mh2w2c2+ hwc3), which is costly if this calculation is repeated across a large collection of networks. In practice, we observe that the optimal shift parameters are typically close to zero (Fig. 3A). This motivates the more stringent equivalence relation:
Xφi ∼X φ j ⇐⇒ vec(X φ i ) = (I ⊗ I ⊗ I ⊗Q)vec(X φ j ) for some Q ∈ Q, (12)
which has a more manageable runtime of O(mhwc2 + c3). To evaluate the metrics implied by Eq. (12), we can simply reshape each Xφk from a (m× h× w × c) tensor into a (mhw × c) matrix and apply the Procrustes alignment procedure as done above for previous metrics. In contrast, the “flattened metric” in Fig. 2B reshapes the features into a (m × hwc) matrix, resulting in a more computationally expensive alignment that runs in O(mh2w2c2 + h3w3c3) time.

heading: 2.5 How large of a sample size is needed?
text: An important issue, particularly in neurobiological applications, is to determine the number of network inputs, m, and neurons, n, that one needs to accurately infer the distance between two network representations [12]. Reasoning about these questions rigorously requires a probabilistic perspective of neural representational similarity, which is missing from current literature and which we outline in Supplement D for generalized shape metrics. Intuitively, looser equivalence relations are achieved by having more flexible alignment operations (e.g. nonlinear instead of linear alignments). Thus, looser equivalence relations require more sampled inputs to prevent overfitting. Figure 3B-C show that this intuition holds in practice for data from deep convolutional networks. Metrics with looser equivalence relations—the “flattened” metric in panel B, or e.g. the linear metric in panel C—converge slower to a stable estimate as m is increased.

heading: 2.6 Modeling approaches and conceptual insights
text: Generalized shape metrics facilitate several new modeling approaches and conceptual perspectives. For example, a collection of representations from K neural networks can, in certain cases, be interpreted and visualized asK points on a smooth manifold (see Fig. 1). This holds rigorously due to the quotient manifold theorem [34] so long as G is not a finite set (e.g. corresponding to permutation) and all matrices are full rank in the feature space. This geometric intuition can be made even stronger when G corresponds to a connected manifold, such as SO(p). In this case, it can be shown that the geodesic distance between two neural representations coincides with the metrics we defined in Propositions 1 and 2 (see Supplement C, and [15]). This result extends the well-documented manifold structure of Kendall’s shape space [35].
Viewing neural representations as points on a manifold is not a purely theoretical exercise—several models can be adapted to manifold-valued data (e.g. principal geodesic analysis [36] provides a generalization of PCA), and additional adaptions are an area of active research [37]. However, there is generally no simple connection between these curved geometries and the flat geometries of Euclidean or Hilbert spaces [38].1 Unfortunately, the majority of off-the-shelf machine learning tools are incompatible with the former and require the latter. Thus, we can resort to a heuristic approach: the set of K representations can be embedded into a Euclidean space that approximately preserves the pairwise shape distances. One possibility, employed widely in shape analysis, is to embed points in the tangent space of the manifold at a reference point [41, 42]. Another approach, which we demonstrate below with favorable results, is to optimize the vector embedding directly via multi-dimensional scaling [43, 44].

heading: 3 Applications and Results
text: We analyzed two large-scale public datasets spanning neuroscience (Allen Brain Observatory, ABO; Neuropixels - visual coding experiment; [21]) and deep learning (NAS-Bench-101; [22]). We constructed the ABO dataset by pooling recorded neurons from K = 48 anatomically defined brain regions across all sessions; each Xk ∈ Rm×n was a dimensionally reduced matrix holding the neural responses (summarized by n = 100 principal components) to m = 1600 movie frames (120 second clip, “natural movie three”). The full NAS-Bench-101 dataset contains 423,624 architectures; however, we analyze a subset of K = 2000 networks for simplicity. In this application each Xk ∈ Rm×n is a representation from a specific network layer, with (m,n) ∈ {(322 × 105, 128), (162 × 105, 256), (82 × 105, 512), (105, 512)}. Here, n corresponds to the number of channels and m is the product of the number of test set images (105) and the height and width dimensions of the convolutional layer—i.e., we use equivalence relation in Eq. (12) to evaluate dissimilarity.
1However, see [39] for a conjectured relationship and [40] for a result in the special case of 2D shapes.
Triangle inequality violations can occur in practice when using existing methods. As mentioned above, a dissimilarity measure based on the mean canonical correlation, 1− ∑ ` ρ`, has been used in past work [7, 10]. We refer to this as the “linear heuristic.” A slight reformulation of this calculation, arccos ( ∑ ` ρ`), produces a metric that satisfies the triangle inequality (see Eq. (10)). Figure 4A compares these calculations as a function of the average (regularized) canonical correlation: one can see that arccos(·) is approximately linear when the mean correlation is near zero, but highly nonlinear when the mean correlation is near one. Thus, we reasoned that triangle inequality violations are more likely to occur when K is large and when many network representations are close to each other. Both ABO and NAS-Bench-101 datasets satisfy these conditions, and in both cases we observed triangle inequality violations by the linear heuristic with full regularization (α = 1): 17/1128 network pairs in the ABO dataset had at least one triangle inequality violation, while 10128/100000 randomly sampled network pairs contained violations in the NAS-Bench-101 Stem layer dataset. We also examined a standard version of RSA that quantifies similarity via Spearman’s rank correlation coefficient [8]. Similar to the results above, we observed violations in 14/1128 pairs of networks in the ABO dataset.
Overall, these results suggest that generalized shape metrics correct for triangle inequality violations that do occur in practice. Depending on the dataset, these violations may be rare (~1% occurrence in ABO) or relatively common (~10% in the Stem layer of NAS-Bench-101). These differences can produce quantitative discrepancies in downstream analyses. For example, the dendrograms produced by hierarchical clustering differ depending on whether one uses the linear heuristic or the shape distance (~85.1% dendrogram similarity as quantified by the method in [45]; see Fig. 4B).
Neural representation metric spaces can be approximated by Euclidean spaces. Having established that neural representations can be viewed as elements in a metric space, it is natural to ask if this metric space is, loosely speaking, “close to” a Euclidean space. We used standard multidimensional scaling methods (SMACOF, [43]; implementation in [46]) to obtain a set of embedded vectors, yi ∈ RL, for which θ1(X φ i ,X φ j ) ≈ ‖yi−yj‖ for i, j ∈ 1, . . . ,K. The embedding dimension L is a user-defined hyperparameter. This problem admits multiple formulations and optimization strategies [44], which could be systematically explored in future work. Our simple approach already yields promising results: we find that moderate embedding dimensions (L ≈ 20) is sufficient to produce high-quality embeddings. We quantify the embedding distortions multiplicatively [47]:
max ( θ1(X φ i ,X φ j )/‖yi − yj‖; ‖yi − yj‖/θ1(X φ i ,X φ j ) )
(13)
for each pair of networks i, j ∈ 1, . . .K. Plotting the distortions as a function of L (Fig. 4C), we see that they rapidly decrease, such that 95% of pairwise distances are distorted by, at most, ~5% (ABO data) or 10% (NAS-Bench-101) for sufficiently large L. Past work [10] has used multidimensional scaling heuristically to visualize collections of network representations in L = 2 dimensions. Our results here suggest that such a small value of L, while being amenable to visualization, results in a highly distorted embedding. It is noteworthy that the situation improves dramatically when
L is even modestly increased. While we cannot easily visualize these higher-dimensional vector embeddings, we can use them as features for downstream modeling tasks. This is well-motivated as an approximation to performing model inference in the true metric space that characterizes neural representations [47].
Anatomical structure and hierarchy is reflected in ABO representations. We can now collect the L-dimensional vector embeddings of K network representations into a matrix Z ∈ RK×L. The results in Fig. 4C imply that the distance between any two rows, ‖zi − zj‖, closely reflects the distance between network representations i and j in shape space. We applied PCA to Z to visualize the K = 48 brain regions and found that anatomically related brain regions indeed were closer together in the embedded space (Fig. 5A): cortical and sub-cortical regions are separated along PC 1, and different layers of the same region (e.g. layers 2/3, 4, 5, and 6a of VISp) are clustered together. As expected from Fig. 4C, performing multidimensional scaling directly to a lowdimensional space (L = 2, as done in [10]) results in a qualitatively different outcome with distorted geometry (see Supplement E). Additionally, we used Z to fit an ensembled kernel regressor to predict an anatomical hierarchy score (defined in [48]) from the embedded vectors (Fig. 5B). Overall, these results demonstrate that the geometry of the learned embedding is scientifically interpretable and can be exploited for novel analyses, such as nonlinear regression. To our knowledge, the fine scale anatomical parcellation used here is novel in the context of representational similarity studies.
NAS-Bench-101 representations show persistent structure across layers. Since we collected representations across five layers in each deep network, the embedded representation vectors form a set of five K ×L matrices, {Z1,Z2,Z3,Z4,Z5}. We aligned these embeddings by rotations in RL via Procrustes analysis, and then performed PCA to visualize the K = 2000 network representations from each layer in a common low-dimensional space. We observe that many features of the global structure are remarkably well-preserved—two networks that are close together in the Stack1 layer are assigned similar colors in Fig. 5C, and are likely to be close together in the other four layers. This preservation of representational similarity across layers suggests that even early layers contain signatures of network performance, which we expect to be present in the AvgPool layer. Indeed, when we fit ridge and RBF kernel ridge regressors to predict test set accuracy from representation embeddings, we see that even early layers support moderately good predictions (Fig. 5D). This is particularly surprising for the Stem layer. This is the first layer in each network, and its architecture is identical for all networks. Thus, the differences that are detected in the Stem layer result only from
differences in backpropagated gradients. Again, these results demonstrate the ability of generalized shape metrics to incorporate neural representations into analyses with greater scale (K corresponding to thousands of networks) and complexity (nonlinear kernel regression) than has been previously explored.

heading: 4 Conclusion and Limitations
text: We demonstrated how to ground analyses of neural representations in proper metric spaces. By doing so, we capture a number of theoretical advantages [16–20]. Further, we suggest new practical modeling approaches, such as using Euclidean embeddings to approximate the representational metric spaces. An important limitation of our work, as well as the past works we build upon, is the possibility that representational geometry is only loosely tied to higher-level algorithmic principles of network function [10]. On the other hand, analyses of representational geometry may provide insight into lower-level implementational principles [49]. Further, these analyses are highly scalable, as we demonstrated by analyzing thousands of networks—a much larger scale than is typically considered.
We used simple metrics (extensions of regularized CCA) in these analyses, but metrics that account for nonlinear transformations across neural representations are also possible as we document in Supplement C. The utility of these nonlinear extensions remains under-investigated and it is possible that currently popular linear methods are insufficient to capture structures of interest. For example, the topology of neural representations has received substantial interest in recent years [50–53]. Generalized shape metrics do not directly capture these topological features, and future work could consider developing new metrics that do so. A variety of recent developments in topological data analysis may be useful towards this end [54–56].
Finally, several of the metrics we described can be viewed as geodesic distances on Riemannian manifolds [35]. Future work would ideally exploit methods that are rigorously adapted to such manifolds, which are being actively developed [37]. Nonetheless, we found that optimized Euclidean embeddings, while only approximate, provide a practical off-the-shelf solution for large-scale surveys of neural representations.

heading: Acknowledgments
text: We thank Ian Dryden (Florida International University), Søren Hauberg (Technical University of Denmark), and Nina Miolane (UC Santa Barbara) for fruitful discussions. A.H.W. was supported by the National Institutes of Health BRAIN initiative (1F32MH122998-01), and the Wu Tsai Stanford Neurosciences Institute Interdisciplinary Scholar Program. E. K. was supported by the Wu Tsai Stanford Neurosciences Institute Interdisciplinary Graduate Fellows Program. S.W.L. was supported by grants from the Simons Collaboration on the Global Brain (SCGB 697092) and the NIH BRAIN Initiative (U19NS113201 and R01NS113119).

Review summaries:
This paper develops metrics for comparing different (yet presumably related) representations. The emphasis is on representations generated by artificial or biological neural networks. The paper relates this development to previously proposed approaches. Experimental results on recordings of neuronal activity in the mouse brain and representations generated by different layers of different deep neural networks are presented.

The current manuscript presented a unifying framework to analyze the representation similarity between neural networks. The authors first established that the distance metrics between neural representations should satisfy equivalence, symmetry and triangle inequality. And the authors showed that a simply modification of canonical correlation analysis would satisfy these reequipment. Next, the authors identified the transformation matrix that best conform the two representation spaces under various conditions. The current framework revealed the hierarchy of multiple brain areas and visualized the representation structure of a deep neural network.  

The paper is concerned with issues that are fundamental to basic research on understanding natural and artificial neural networks. Such research often involves measuring the similarity of neural representations (patterns of neural activity). The authors propose desirable attributes for the methods used for such comparisons, e.g. that the similarity function should define a metric metric space: it should satisfy the constraints of equivariance, symmetry and the triangle inequality. They show that many commonly used methods do not satisfy these constraints but can be easily modified to do so. They also construct a metric especially suited to convolutional layers, which traditionally have not been straightforwardly analyzed by existing methods. The authors also discuss practical issues of general importance: They discuss what sample size is required for such metrics to be accurate/robust, which is highly important for neuroscience research where it can be difficult to obtain large sample sizes. They also talk about dimensionality reduction, showing that approximating neural embeddings with low dimensional Euclidean spaces, while suitable for visualization, can highly distort the embedding. They validate their new metrics by showing that it can recover known structure in large scale datasets.