GULP : a prediction-based metric between representations
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Comparing the representations learned by different neural networks has recently emerged as a key tool to understand various architectures and ultimately optimizethem.
In this work, we introduce GULP , a family of distance measures betweenrepresentations that is explicitly motivated by downstream predictive tasks.
Byconstruction, GULP provides uniform control over the difference in predictionperformance between two representations, with respect to regularized linear prediction tasks.
Moreover, it satisﬁes several desirable structural properties, suchas the triangle inequality and invariance under orthogonal transformations andthus lends itself to data embedding and visualization.
We extensively evaluate
GULP relative to other methods and demonstrate that it correctly differentiatesbetween architecture families, converges over the course of training, and captures generalization performance on downstream linear tasks.
Introduction
The spectacular success of deep neural networks (DNN) witnessed over the past decade has beenlargely attributed to their ability to generate good representations of the data [BCV13] .
But whatmakes a representation good?
Answering this question is a necessary step towards a principledtheory of DNN design.
This fundamental question calls for a metric over representations as a basicprimitive.
Indeed, embedding representations into a metric space enables comparison, modiﬁcationsand ultimately optimization of DNN architectures [LTQ + 18]; see Figure 1.
In light of the practical impact of a meaningful metric over representations, this question hasrecently garnered signiﬁcant attention, leading to a myriad of propositions such as CCA , CKA , and
P ROCRUSTES .
Their relative pros and cons are currently the subject of a lively debate [DDS21,
DHN + 22] whose resolution calls for a theoretically grounded notion of metric.
Our contributions.
In this work, we deﬁne a new family of metrics 1 , called GULP , over the spaceof representations.
Our construction rests on a functional notion of what makes two representationssimilar: namely, that two representations are similar if
and only if they are equally useful as inputs todownstream, linear transfer learning tasks.
This idea is partially inspired by feature-based transferlearning, in which simple models adapt pretrained representations, such as Inceptionv3 [SVI + 16],
CLIP [RKH + 21], and ELMo [PNI + 18], for speciﬁc tasks [RASC14]; indeed, this is a key use forpretrained representations.
Moreover, our application of linear transfer learning is reminiscent oflinear probes , which were introduced by [AB17] as a tool to compare internal layers of a DNN in termsof prediction accuracy.
Linear probes play a central role in the literature on hidden representations.
They have been used not only to study the information captured by hidden representations [RBH20], but also to themselves deﬁne desiderata of distances between representations [DDS21].
However,previous applications of linear probing required hand-selecting the task on which prediction accuracyis measured, whereas our GULP distance provides a uniform bound over all norm-bounded tasks.
We establish various theoretical properties of the GULP pseudo-metric, including the triangle inequality (Thm 2), sample complexity (Thm 3), and vanishing cases.
In particular, we show that akin tothe P ROCRUSTES pseudo-metric, GULP is invariant under orthogonal transformations (Thm 1) andvanishes precisely when the two representations are related by an orthogonal transformation (Thm 2).
In turn, we use GULP to produce low-dimensional embeddings of various DNNs that provide newinsights on the relationship between various architectures (Figures 1, 5, and 6).
Moreover, in Figure 7,we showcase a numerical experiment to demonstrate that the GULP distance between two independentnetworks decreases during training on the same dataset.
Related work.
This contribution is part of a growing body of work that aims at providing toolsto understand and quantify the metric space of representations [RGYSD17, MRB18, KNLH19,
AB17, ALM17, CLR + 18, LC00, LV15, LYC + 15, LLL + 19, Mig19, STHH17, WHG + 18, DDS21,
DHN + 22, CKMK22].
Several of these measures, such as SVCCA [RGYSD17] and PWCCA [MRB18],are based on a classical canonical correlation analysis ( CCA ) from multivariate analysis [And84].
More recently, centered kernel alignment CKA [CSTEK01, CMR12, KNLH19, DHN + 22] hasemerged as a popular measure; see Section 2 for more details on these methods.
The orthogonal procrustes metric (P ROCRUSTES ) is a classical tool of shape analysis [DM16] to compute thedistance between labelled point clouds.
Though not as conspicuous as CKA -based methods in thecontext of DNN representations, it was recently presented under a favorable light in [DDS21].
Various desirable properties of a similarity measure between representations have been put forward.
These include structural properties such as invariance or equivariance [LC00, KNLH19], as well assanity checks such as speciﬁcity against random initialization [DDS21], for example.
Such desideratacan serve as diagnostics for existing similarity measures, but fall short of providing concrete designguidelines.
Outline The rest of the paper proceeds as follows.
Section 2 lays out the derivation of GULP , aswell as important theoretical properties: conditions under which it is zero, and limiting cases in termsof the regularization parameter  , demonstrating that it interpolates between CCA and a version of
CKA .
Section 3 establishes concentration results for the ﬁnite-sample version, justifying its use inpractice.
In Section 4 we validate GULP through extensive experiments.
Finally, we conclude in
Section 5.
The GULP distance
As stated in the introduction, the goal of this paper is to develop a pseudo-metric over the space ofrepresentations of a given dataset.
Unlike previous approaches, which work with ﬁnite datasets, wetake a statistical perspective and formulate the population version of our problem.
We defer statisticalquestions arising from ﬁnite sample size to Section 3.
Let X 2 R d be a random input with distribution P X and let f : R d ! R k denote a representationmap , such as a trained DNN.
The random vector f ( X ) 2 R k is the representation of X by f .
Weassume throughout that a representation map is centered and normalized, so that E [ f ( X )]
= 0 and
E k f ( X ) k 2 = 1 .
In particular, this normalization allows us to identify representation maps  , thatare related by ( x ) = a  ( x ) + b , P X -a.s.
for a 2 R and b 2 R d , down to a single representation of X , which is a well-known requirement for distances between representations [KNLH19, Sec. 2.3].
We are now in a position to deﬁne the GULP 2 distance between representations; the terminology“distance" is justiﬁed in Theorem 2.
To that end, let  : R d ! R k and : R d ! R ` be tworepresentation maps, where ` may differ from k .
Let ( X, Y ) 2 R d ⇥
R be a random pair and let⌘ ( x ) =
E [ Y | X = x ] denote the regression function of Y onto X .
Moreover, for any  > 0 , let  denote the population ridge regression solution given by
[Equation]
and similarly for   with respect to ( · ) .
Since we use squared error, these only depend thedistribution of Y through the regression function ⌘ .
Deﬁnition 1.
Fix  > 0 .
The GULP distance between representations  ( X ) and ( X ) is given by
[Equation]
where the supremum is taken over all regression functions ⌘ such that k ⌘ k L 2 ( P X )  1 .
The GULP distance measures the discrepancy between the prediction of an optimal ridge regressionestimator based on  , and its counterpart based on , uniformly over all regression tasks.
While thisnotion of distance is intuitive and motivated by a clear regression task, it is unclear how to compute ita priori .
The next proposition provides an equivalent formulation of GULP , which is amenable toaccurate and efﬁcient estimation; see Section 3.
It is based on the following covariance matrices:
[Equation]
We implicitly used the centering assumption in the above deﬁnition, and the normalization condition implies that the covariance matrices have unit trace.
Throughout, we assume these matrices are invertible, which is without loss of generality by projecting onto the image of the representation map.
We also deﬁne the regularized inverses:
[Equation]
as well as the cross-covariance matrices ⌃  and ⌃  as follows:
[Equation]
Proposition 1.
Fix   0 .
The GULP distance between representations  ( X ) and ( X ) satisﬁes
[Equation]
Proof.
See Appendix A.1.
2.1 Structural properties
In this section, we show that GULP is invariant under orthogonal transformations and that it is a validmetric on the space of representations.
We begin by establishing a third characterization of GULP thatis useful for the purposes of this section;the proof can be found in Appendix A.1.
Lemma 1.
Fix   0 .
The GULP distance d  (  , ) between the representations  ( X ) and ( X ) satisﬁes
[Equation]
where X 0 is an independent copy of X .
We are now in a position to state our main structural results.
We begin with a key invariance result.
Theorem 1.
Fix   0 .
The GULP distance d  (  , ) between the representations  ( X ) 2 R k and ( X ) 2 R ` is invariant under orthogonal transformations: for any orthogonal transformations
U : R k !
R k and V : R ` ! R ` , it holds
[Equation]
Proof.
We slightly abuse notation by identifying any orthogonal transformation W to a matrix W such that W ( x ) = W · x .
Note that for any representation map, we have ⌃ W  f = W ⌃ f W > and
[Equation]
Hence, using Lemma 1, we get that
[Equation]
where we used the fact that U > U = I k and V > V =
I ` .
Next, we show that GULP satisﬁes the axioms of a metric.
Theorem 2.
Fix  > 0 .
The GULP distance d  (  , ) satisﬁes the axioms of a pseudometric, namelyfor all representation maps  , , ' , it holds
[Equation]
Moreover, d  (  , ) =
if and only if k = ` and there exists an orthogonal transformation U suchthat  ( X ) =
U ( X ) a.s.
Proof.
Lemma 1 provides an isometric embedding of representations f 7!
f ( X ) ⌃   f f ( X 0 ) into the
Hilbert space L 2 ( P ⌦ 2 X ) .
It readily yields that d  is a pseudometric.
It remains to identify for which , it holds that d  (  , ) = 0 .
The “easy” direction follows from the invariance property of Theorem 1: if  and satisfy  ( X ) =
U ( X ) almost surely, then d  (  , ) =
d  ( U , ) = 0 .
We sketch the proof of the other direction,and defer the full proof to Appendix A.2.
Deﬁne ˜
=
( ⌃  +  I )  1 / 2  and ˜ = ( ⌃ +  I )  1 / 2 .
By Lemma 1, the condition that d  (  , ) = 0 is equivalent to ˜  ( X ) >
˜  ( X 0 ) =
˜ ( X ) >
˜ ( X ) almostsurely over X, X 0 .
So if d  (  , ) = 0 , then we can leverage a classical fact that the Gram matrix ofa set of vectors determines the vectors up to an isometry [HJ12], to prove that there is an orthogonaltransformation U 2 R k ⇥ k such that ˜  ( X ) =
U ˜ ( X ) almost surely over X .
Finally, via analyzing ahomogeneous Sylvester equation, this implies that  ( X ) =
U ( X ) almost surely.
Note that when  = 0 , the conclusion of this theorem fails to hold: d 0 still satisﬁes the axioms of apseudo-distance, but the cases for which d 0 (  , ) = 0 are different.
This point is illustrated in thenext section where we establish that d 0 is the CCA distance commonly employed in the literature.
2.2 Comparison with CCA , ridge- CCA and CKA
Throughout this section, we assume that k = ` for simplicity.
Ridge-
CCA .
Our distance is most closely related to ridge-
CCA , introduced by [Vin76] as a regularized version of Canonical Covariance Analysis ( CCA ) when the covariance matrices ⌃  or ⌃are close to singular.
More speciﬁcally, for any   0 , deﬁne the matrix C  :=
⌃   ⌃  ⌃   ⌃  ;the ridge- CCA similarity measure is deﬁned as ⇢   CCA = tr( C  ) .
Hence, we readily see from
Proposition 1 that GULP and ridge-
CCA are describing the same geometry over representations.
To seethis, recall that Lemma 1 provides an isometric embedding f 7! f ( X ) ⌃   f f ( X 0 ) of representationmaps into L 2 ( P ⌦ 2 X ) .
While GULP is the distance on this Hilbert space, ridge-
CCA is the inner product.
Ridge-
CCA was brieﬂy considered in the seminal work [KNLH19] but discarded because of (i) its lackof interpretability and (ii) the absence of a rule to select  .
We argue that in fact, our prediction-drivenderivation of GULP gives a clear and compelling interpretation of this geometry (as well as suggestsseveral extensions; see Section 5).
Moreover, we show that tunability of  is, in fact, a desirablefeature that allows to represent the space of representations at various resolutions, giving variouslevels of information; for example, in Figure 6, higher  leads to a coarser clustering structure.
CCA .
Due to the connection with ridge- CCA , our GULP distance is related to (unregularized) CCA when  = 0 .
Speciﬁcally, deﬁning C :=
⌃  1  ⌃  ⌃  1 ⌃  , the mean-squared- CCA similarity measure is given by (see [Eat07, Def. 10.2]):
[Equation]
where X is an independent copy of X 0 ; the last identity can be checked directly.
From Lemma 1 itcan be seen that our GULP distance d 0 (  , ) with  = 0 is a linear transformation of ⇢ CCA .
It can be checked that ⇢ CCA takes values in [0 , 1] , which has led researchers to simply propose 1  ⇢ CCAas a dissimilarity measure.
Interestingly, this choice turns out to produce a valid (squared) metric, i.e.,a dissimilarity measure that satisﬁes the triangle inequality.
Indeed, we get that
[Equation]
⇤ where K ( u, v ) =
u > v is the linear kernel over R d and ˜  :=
⌃  1 / 2   (where ˜ and ˜  are thewhitened versions of and  respectively).
These identities have two consequences: (i) we seefrom Lemma 1 that d CCA corresponds to the GULP distance with  =
up to a scaling factor and(ii)
d CCA is a valid pseudometric on the space of representations, since we just exhibited an isometry
T : ˜ f 7!
K ( ˜ f
( X ) , ˜ f
( X 0 )) with L 2 ( P ⌦ 2 X ) .
We show in Appendix A.2 that d CCA (  , ) = {+
+}
iff( X ) =
A  ( X ) a.s. for some matrix A .
Note that the invariance of ⇢ CCA to linear transformationswas previously known and criticized in [KNLH19] as arguably too strong.
CKA .
In fact, thanks to the additional structure of the Hilbert space L 2 ( P ⌦ 2 X ) , the d CCA distance comes with an inner product
[Equation]
This observation allows us to connect CCA with CKA , another measure of similarity between distributions that is borrowed from classical literature on kernel methods [CSTEK01, CMR12] and that wasrecently made popular by [KNLH19].
Under our normalization assumptions, CKA is a measure ofsimilarity given by
[Equation]
where k T k 2 CCA = h T, T i CCA and ] denotes the angle in the geometry induced by h · , · i CCA . In turn, 152 d
CKA is chosen as d 2 CKA = 1  ⇢ CKA , which does not yield a pseudometric.
This observation highlightstwo major differences between CCA and CKA : the ﬁrst measures inner products and works withwhitened representations, while the second measures angles and works with raw representations.
Asillustrated in the experimental section 4 as well as in [DDS21], this additional whitening step appearsto be detrimental to the overall qualities of this distance measure.
The fact that GULP with  = 0 recovers d CCA (i.e. d 20 = 2 kd 2 CCA ) is illustrated in Figure 2.
As shown,although GULP has a roughly monotone relationship with CKA , they remain quite different.
Although CKA and GULP are related for large  , their relationship is not linear due to the difference in normalization.
Appendix B.2 contains more details and comparisons, including a surprisingly strong correlation between GULP and P ROCRUSTES for some values of  .
Plug-in estimation of GULP
In practice, the distribution P X of X is unknown, so we cannot compute the population version of
GULP exactly.
Instead, we have access to a sample X 1 , . . .
X n i.i.d.
⇠ P X .
In all of the experiments ofthis paper, we approximate GULP with the following plug-in estimator:
[Equation]
whereˆ ⌃  = 1 n n
[Equation]
and ˆ ⌃  = 1 n n X i = [Equation]
[Equation]
X
[Equation]
X
[Equation]
are the empirical covariance and cross-covariance matrices, andˆ ⌃   =
(ˆ ⌃  +  I )  1 , and ˆ ⌃   = (ˆ ⌃
[Equation]
[Equation]
) are the empirical inverse regularized covariance matrices.
To justify our use of the plug-in estimator,we prove concentration around the population GULP distance as n goes to inﬁnity.
Theorem 3.
Assume that k  ( X ) k 2 , k ( X ) k 2  1 almost surely.
Then, for any  2 (0 , 1) ,  > 0 ,with probability at least 1   the plug-in estimator ˆ d 2  ,n satisﬁes
[Equation]
We defer the proof of this theorem to Appendix A.3.
At a high-level, we ﬁrst show that the inverseregularized covariance matrices, ( ⌃  +  I )  1 and ( ⌃ +  I )  1 , are well-approximated in operatornorm, so the expectation of the plug-in estimator is close to the population distance.
We then apply
McDiarmid’s inequality to show that the plug-in estimator concentrates around its expectation.
Notethat the boundedness conditions on the representations are here to simplify technical arguments byappealing simply to McDiarmid’s inequality; these can be presumably be relaxed to weaker tailconditions at the cost of more involved arguments.
Figure 3 supports our theoretical result by showing convergence on pairs of networks on the ImageNetdataset.
See Appendix B.3 for more details.
Experiments
We evaluate our distance in a variety of empirical settings, comparing to CCA , CKA , the classical
P ROCRUSTES method from shape analysis, and a variant of CCA known as projection-weighted CCA( PWCCA ); see [DDS21, Sec. 2] for deﬁnitions.
imator as n !
We plot relative error | ˆ d 2  ,n  d 2  , 10000 | /d 2  , 10000 averaged over pairs of ImageNet DNNs.
We plot Spearman’s ⇢ between the differences in predictions by  -regularized linear regression, and the different distances.
Results are averaged over 10 trials.
4. GULP captures generalization performance by linear predictors
The GULP distance is motivated by how differently linear predictors using the representations  andgeneralize.
In this section, we demonstrate that GULP indeed captures downstream generalizationperformance by linear predictors.
We consider the representation maps  1 , . . .
,  m given by m = 44pretrained image classiﬁcation architectures on the ImageNet dataset µ (see Appendix B.4).
For eachpair of representations, we estimate the CKA , CCA , PWCCA , and GULP distances, using the plug-inestimators on 10,000 images, sufﬁcient to guarantee good convergence (see Figure 3).
We then draw n = 5,000 images from the dataset X 1 , . . .
, X n ⇠ µ , and assign a random label
Y k ⇠ N (0 , 1) to each one.
For each representation i 2 [ m ] , we ﬁt a  -regularized least-squares linearregression to the training data { ( X k , Y k ) } k 2 [ n ] , which gives a coefﬁcient vector   ,i .
Finally, foreach 1  i < j  m , we estimate the distance ⌧ ij = E X ⇠ µ [(  >  ,i  i ( X )   >  ,j  j ( X )) 2 ] betweenthe predictions with representations
i and  j , by taking the empirical average over 3000 samples ina test set.
In Figure 4, we plot Spearman’s ⇢ rank correlation between ⌧ and each of the distances
GULP , CKA , CCA , PWCCA , viewed as vectors with  m entries, one for each pair of networks.
Noticethat for each  , the distance that attains the best correlation is the GULP distance with that  .
Thisindicates that while GULP is a measure of distance that holds uniformly over prediction tasks, itretains its meaning in the context of a single prediction task.
4. GULP distances cluster together networks with similar architectures
We are interested in how GULP can be used to compare networks of different architectures trained onthe same task.
We begin by comparing fully-connected ReLU networks of widths ranging from 100to 1,000 and depths ranging from 1 to 10, trained on the MNIST handwritten digit database.
Everyarchitecture is retrained four times from different initializations (see Appendix B.1).
We input all
MNIST training set images into each network, save their representations at the ﬁnal hidden layer, andcompute CKA , P ROCRUSTES , and GULP distances between all pairs of representations.
Figure 5 shows Multi-Dimensional Scaling (MDS) embeddings of the distances between all MNISTnetworks, color coded by width and depth.
For GULP with  = 10  6 , the networks are largelyorganized according to rank of the feature matrix: networks of large width and small depth, oneswhose representations have the largest rank, are the most different, as evidenced by the halo of pointsin the MDS plots.
This outcome conﬁrms that CCA simply measures rank [KNLH19].
However, for
P ROCRUSTES and GULP with  = 10  2 , networks become clustered by their depth, as evidencedby the striations in the MDS embeddings (plot colored by depth).
Furthermore, networks of thesame depth look most similar at large widths, as shown by the red centerline in the MDS embedding(plot colored by width), implying that as width increases networks converge to a shared limitingrepresentation.
Finally, GULP with  = 1 closely resembles CKA and roughly organizes networksby depth.
A takeaway is that GULP with  resembles P ROCRUSTES and CKA , and captures intrinsiccharacteristics such as width and depth.
Next, we show how distances between ﬁnal-layer representations allow us to cluster pretrainednetworks with more complex architectures and, in turn, draw comparisons between them.
[-c i e n t n e t _ b 
-]
[-r e g n e t _ y _ 
-]
To that end, we study 36 state-of-the-art models on the ImageNet Object Localization Challenge, of which the ﬁvemajor groups are ResNets, Vision Transformers (ViTs), EfﬁcientNets, ConvNeXts, and MobileNets(see Appendix B.1).
We compute the baseline distances between every pair of representations using 10,000 training images,and visualize them using a two-dimensional t-SNE embedding (Figure 6, top row).
In the bottom rowof Figure 6 we plot the dendogram resulting from a hierarchical clustering of the networks based ontheir distances.
As seen from the embeddings, when  increases, the GULP distance separates the
ResNet architectures (blue) from the EfﬁcientNet and ConvNeXt convolutional networks (orangeand red).
For all metrics, the Vision Transformers (purple) are close in distance to the ConvNeXtmodels (red), consistent with the fact that ConvNeXts were designed to achieve competitive ImageNetaccuracy compared to ViTs [LMW + 22].
Compared to other distances, GULP with large  is ableto more compactly cluster ResNets and convolutional networks separately.
In Appendix B.4 wefurther quantify the compactness of clusterings under each distance metric by computing the standarddeviation of distances within each cluster.
4.3 Network representations converge in GULP distance during training
So far, we have used GULP to compare static networks taken as a blackbox representation maps.
Nowwe use GULP to examine how representation maps evolve over the course of training.
To that end,we independently train 16 Resnet18 architectures on the CIFAR10 dataset [KH + 09] for 50 epochs.
Figure 7 tracks the distance (averaged over all network pairs) at each epoch.
As shown, other distances change very little or even brieﬂy increase over the course of training.
For GULP with small  , the previous sections have demonstrated that our distance captures ﬁne- grained differences between representations; here too, it accentuates differences in representationsmid-training (visible around epoch 25).
However, as  increases, the GULP distance differentiatesless between representations, and smoothly decreases over the course of training, thus indicating thatit captures intrinsic properties of the representations rather than artifacts due to random seeds.
4.4 Sensitivity versus speciﬁcity of GULP
In Appendix B.6, we reproduce the experiments of [DDS21].
Our distance compares favorably tobaselines and correlates with measures of a DNN’s functional behavior.
It achieves the speciﬁcity of
CCA and PWCCA to random initializations, and improves the sensitivity of CKA to out-of-distributionperformance.
Conclusion
In this paper, we have deﬁned a family of distances for comparing learned representations in terms oftheir worst-case performance gap over all  -regularized regression tasks.
We proved convergenceof the ﬁnite-sample estimator of this distance, quantiﬁed its relationship to existing notions suchas CCA , ridge- CCA , and CKA , and demonstrated promising performance in a variety of empiricalsettings, including the ability to distinguish between network architectures and to capture performancedifferences on regression tasks.
Further studying extensions beyond linear transfer learning could provide a rich direction for futurework.
In fact, preliminary experiments reported in Appendix B.7 indicate that, compared to section 4.1, GULP fails to predict generalization performance when the downstream task shifts from linear tologistic prediction.
This suggests extending GULP to a uniform bound over other downstreampredictive tasks, such as logistic regression, multi-class classiﬁcation, or kernel ridge regression.
Although GULP under kernel ridge regression has a closed form using the kernel trick 3 , GULP forlogistic regression does not have a closed form.
This brings additional computational questions ofinterest that are beyond the scope of this work.
can be easily derived vanilla ridge regression as studied