On Affine Homotopy between Language Encoders
Authors: Robin Chan
Reda Boumasmoud
Anej Svete
Yuxin Ren
Qipeng Guo
Zhijing Jin
Shauli Ravfogel
Mrinmaya Sachan
Bernhard Schölkopf
Mennatallah El-Assady
Ryan Cotterell
Venue: NeurIPS 2024 poster (2024)
Keywords: language encoders
neural network similarity
metric spaces
homotopy
Primary area: natural_language_processing

Abstract: Pre-trained language encoders---functions that represent text as vectors---are an integral component of many NLP tasks.
    We tackle a natural question in language encoder analysis: What does it mean for two encoders to be similar?
    We contend that a faithful measure of similarity needs to be \emph{intrinsic}, that is, task-independent, yet still be informative of \emph{extrinsic} similarity---the performance on downstream tasks.
    It is common to consider two encoders similar if they are \emph{homotopic}, i.e., if they can be aligned through some transformation.
    In this spirit, we study the properties of \emph{affine} alignment of language encoders and its implications on extrinsic similarity.
    We find that while affine alignment is fundamentally an asymmetric notion of similarity, it is still informative of extrinsic similarity.
    We confirm this on datasets of natural language representations.
    Beyond providing useful bounds on extrinsic similarity, affine intrinsic similarity also allows us to begin uncovering the structure of the space of pre-trained encoders by defining an order over them.

On Affine Homotopy between Language Encoders
Robin S. M. Chan'
Reda Boumasmoud' Anej Svete'
Yuxin Ren
Qipeng Guo
Zhijing Jin 1.4
Shauli Ravfogel' Mrinmaya Sachan
Bernhard Schölkopf
Mennatallah El-Assady' Ryan Cotterell'
ETH Zürich
Tsinghua University Fudan University
*Max Plank Institute for Intelligent Systems
Abstract
Pre-trained language encoders functions that represent text as vectors are an integral component of many NLP tasks. We tackle a natural question in language encoder analysis: What does it mean for two encoders to be similar? We contend that faithful measure of similarity needs to be intrinsic, that is, task-independent, yet still be informative of extrinsic similarity tasks. It is common to consider two encoders similar if they are homotopic, i.e., if they can be aligned through some transformation In this spirit, we study the properties of affine alignment of language encoders and its implications similarity. We find that while affine alignment fundamentally an asymmetric notion of similarity, it is still informative of extrinsic similarity. We confirm this on datasets natural language representations. Beyond providing useful bounds extrinsic similarity, affine intrinsic similarity also allows to begin uncover- ing the structure of the space of pre-trained encoders by defining an order over them.
performance on downstream
extrinsic
Introduction
A common paradigm in modern natural language processing (NLP) is to pre-train language encoder on a large swathe of natural language text. Then, a specific model is fit (fine-tune) using the language encoder as the representation function of the text. More formally, a language encoder is function h: E RDD i.e., a function that maps a string over an alphabet to dinte--amensional vector. Now, consider sentiment analysis as informative example of a task. Suppose our goal is to classify string y E* as one of three polarities = (0, @ @). Then, the probability of y exhibiting a specific polarity is often given by log-linear model, e.g., the probability of @ is
p
softmax (Eh(y) +
where E R3xD, b 3R3 and softmax R N-1, Empirically, using pre-trained encoder h. leads to significantly better classifier performance than training a log-linear model from scratch.
In the context of the widespread deployment of language encoders, this paper tackles natural question: Given two language encoders h and g. how can we judge to what extent they are similar? studies have shown that even small variations in
This question is of practical importance-receet the random seed used for training can result significant performance differences on downstream tasks between models with the same architecture [13] 35 In this case, say that two such language encoders exhibit an extrinsic difference, i.c., the difference between two encoders manifests itself downstream task. However, also seek an intrinsic notion
when considering their performance
Hommtopy, from the Greek ôạيóę (homo: same) and vóπoς (topos; place), refers continuous transforma- tion between functions or shapes, showing they can be deformed into one another without breaking or tearing.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

of similarity between two language encoders, i.e., a notion of similarity that is independent of any particular downstream task. Moreover, we may hope that a good notion of intrinsic similarity would allow us to construct notion of extrinsic similarity that holds for all downstream tasks.
Existing work studies language encoder similarity by evaluating whether two encoders produce similar representations for finite dataset of strings [3, 20) 22 42 inter alia], often by analyzing whether the representation sets can be approximately linearty aligned [22 27]. More formally, two encoders are considered similar if there exists a matrix A such that (zy ~ A ggy) holds for strings y some finite set D Σ* This assumes that examining finitely many outputs provides sufficient insight into encoder behavior. In contrast, we set out to study the relationships between language encoders, i.e., functions, themselves. This decision, rather than being just a technicality, allows us to derive richer understanding of encoder relationships, revealing properties and insights that remain obscured under conventional finite-set analysis. Concretely, we ask what notions of similarity between encoders one could consider and what they imply for their relationships.
The main contributions of the paper are of theoretical nature. We first define an (extended) metric space on language encoders. We then extend this notion to account for transformations in broad framework of S-homotopy for a of transformations S, where g is S-homotopic h if g can be transformed into h through some transformation in S. As a concrete application of the framework, we study affine homotopy- the similarity of and 1o g for affine transformations tb. The notion of in- trinsic similarity induced by such one-sided alignment is not symmetric and can be seen as the cost of transforming g into h. Nevertheless, we show informative of extrinsic similarity: If one encoder can be affinely mapped to another, we can guarantee that it also performs similarly on downstream tasks. We confirm this empirically by studying the intrinsic and extrinsic similarities of various pre- trained encoders, where we observe positive correlation between intrinsic and extrinsic similarity. Beyond measuring similarity, homotopy also allows us to define a form of hierarchy on the space of encoders, elucidating structure in which some encoders are more informative than others. Such an or- der is also suggested by our experiments, where we find that certain encoders are easier to map to than others which shows in the rank of the learned representations and affects their transfer learning ability,
2 Language Encoders
Let Σ be an alphabet a finite, non-empty set of symbols and EOS & E distinguished end-of- string symbol. With > def U n we denote the Kleene closure of Σ, the set of all strings y. A language encoder is a function h: Σ* - V def R that maps strings to real vectors We write Ey def 5 for the RRvvecoor space of language encoders, and E, Ey for its sub-vector space of bounded encoders.
def (h Ev ((E*) ) is bounded) c
There are two common ways that language encoders are created (7]. The first is through autoregressive language modeling. A language model (LM) is a probability distribution over > LMs are defined through the multiplication of conditional probability distributions pa (yt y/<r)
5* Autoregressive
(1) Pa (EOS 3)
I
Pa(ve y=c), t-1
(2)
where each (.(. |ye)) is distribution over Σ u [BOS) parametrized by a language encoder h:
Pa(U6 V=e)) softmax E h(y<:))
where E E R()++1)) An autoregressive LM provides a simple manner to learn a language encoder from a dataset of strings D {") } by minimizing D's negative log- ligeliiel We may also learn a language encoder through masked language modeling (MLM), which defines the conditional probabilities based on both sides of the masked symbol's 's context
(y yec y def
softmax (EaemaxxxEtt (yyee [MASK] o y>x))
We discuss related work in more detail in App- In principle, one could relax the replace H²" with any finite dimensional vector space. 'In the following, we assume language model tightness to the effect that we can assume that LMs produce valid probability distributions over * [15].

---

Maximizing the log-likelhhood of corpus under a language model derived from language encoder with a gradient-basee algorithm only requires h to be differentiable function of its parameters. Once a language encoder has been trained on a (large) corpus, its representations can be used on more fine-grae mee-ggained NL.P tasks such as classification. The rationale for such transfer learning that representations h. (y) stemming from a performant language model also contain information useful for other downstream tasks on natural language. An NLP practitioner might then implement task-specific transformation of h (y). To tackle the problem that the tasks of interest are often less resource-abundant and to keep the training costs low, task-specific transformations are usually simple, often in the form of linear transformations of h (y), in Eq.
Measuring the Alignment of Langauge Encoders
We begin by introducing measures of affine alignment and hemi-metrics on yy.
3.1 Preliminaries on Hemi-Metric Spaces
Language encoders compute representations for the infinitely many strings in Σ*, In general, these representations might diverge towards sn, making it necessary talk about unbounded encoders, where it is convenient to allow distances and norms to take extended real numbers as values
Definition 3.1. An extended metric on set X is a map d: X - R2 such that
E d(a,v) 0 iff x Vi
(Identity)
b. V2,8,2E d(x,y) < d(z, 2) + d(z,8); c. Yx,yE d(z,y) = z).
(Triangle Inequality)
(Symmetry)
Similarly, an extended norm is a map will consider maps that do not satisfy the symmetry axiom. Lawvere 25 notes that symmetry artificial and unnecessary for many of the main theorems involving metric spaces. In such situations, the quantity d(z, y) can interpreted as the cost going from æ to y. Occasionally, we want to capture that it costs more to go from x to y than to return, making asymmetry desirable.
X - R4 that satisfies the norm axioms. Moreover, we
Definition 3.2. A hemi-metric or Lawvere-metric on a set X a map d: X R, such that
a. d(r, x) 0, b. d(x,2) < d(z, y) d(y,2) for all x, 2, X.
One of our main contributions is a formalization of measuring how far a language encoder h is from the set of all possible transformations of another encoder g for example, from all affine transformations of g. For this, we lift a hemi-metric over elements x E X to subsets of X, a crucial for the rest of the paper.
Definition 3.3. Let (X, d) be a hemi-metric space. For non-empty E, E' X, we define
d'(FF, sup inf d(x.y). FEE yEE'
(5)
The map d= is called the Hausdorff-Hoare map and is hemi-metric on P(X))( the power set of X. When E is singleton set (x), we will, with a slight abuse of notation, write d (z, E') to mean d² (1x), E"), defined as y d(x, y)
We next introduce the hemi-metric recipe. It tells us how one can define hemi-metric set X by embedding X into the power set of another space where a hemi-metric already exists. After X is embedded, one use the Hausdorff Hoare map based on the hemi-metric from Y to define hemi metrim on through the images x e X.
*R is the set of non non-negative real numbers along with the "value" co. assumed to be above all reals. We adopt the following conventions: on -0-0 0== 0- 0; 00 + - oo oo, R.o for every R 'A basic example of hemi-metric space is the pair (IE, da), where da 8) = max(# - y. 0).
AAdiiioonl properties of dft are discussed in Lem. D.I
3

---

Remark 3.1 (Hemi-Metric Recipe), Let X be set, (Y,d) a hemi-metric space. and S: X P(Y))Ø), - E, a function that assigns an x E subset E, E P(Y))22). Using Lem. D.1 we can construct a hemi-metric on X with d (z. v) ot d d Ey) = (S(x), SYv),, and an extended pseudo-metric (a symmetric hemi-metric) with d= ("(x, v) max(dy (z, v). d (v: x)).
Remark 3.1 introduces a general recipe for defining hemi-metric spaces on function spaces- subsets of extended-metrie space.
topological spaces whose elements are functions from This naturally applies to the study of encoders and their transformations, which we call S-homotopy, i.e. two encoders are S-homotopic if one can be S-deformed into the other. In this case, the set En for h E Ey corresponds to the set of encoders that h can be transformed into with mappings in S. We could, for example, take S as the set of all continuous maps, smooth maps, or multi-layer perceptrons. Our following discussion of affine maps, i.e., where S = AV) is extrinsically motivated but can be understood as a specific instance of the more general framework of S-homotopy.
3.2 A Norm and a Distance on Ey
The hemi meti-mmtric recipe first requires us to define (hemi-)meeric the individual elements. Given that all norms on the Revector space V are equivalent [24] Proposition 2.2, SXII], we fix in this paper
a norm lv on V. We introduce the maps || Ilo Ey R and dx (-.)): Ev x Ey R_:
h
sup |h(y)lv yEΣ*
(6) and
de (h, g) = 1h- gl
where || Il is an extended norm on Ey and (Ev, is a complete extended metric space g
Let GL(VV be the set of invertible Dx D matrices. We write Il-llv GL(V) - E for the subordinate view V affine space 10 and set (V) for the group of affine transformations of V. An affine transformation yb V a map V ~ Av b, for some invertible A GL(V) and b V. We call 4/iin def "A the linear part of s and 4: v - v b its translation part. We denote with T c Af((V) the subgroup of translations.
matrix norm, i.e., ||Ay S Ivvv By abuse of language,
AAvyy
Note that there is a natural left action of Af)( on εy, i.e., Af) x Ev - εv, h - và o h
3.3 Affine Alignment of Language Encoders
We now use the general recipe from Remark 3.I for affine alignment of language encoders- affinely mapping from one encoder another. For subset S = (V) we define
dg dg g) def (h,
inf ||
g|اo
(8a)
Ves
hill del dO h),
(8b)
where S (h) (s 8 S}. In the notation of the hemi-metric recipe from Remark 3.1 we set
X = Y = Ev (we align an encoder with another encoder) and d = dz,, the uniform convergence distance (cf. 3.2). Further, we take S s Aff(V)
VV and define S: Ey - P (Ev).. h
S (h) del (s h s E S}. In words, ds(h, (h, g) captures the notion of how well the encoder g can be S- transformed into h. This is commonly called the alignment of g with h. ds (h, g) does not, however, necessarily tell us anything about how well h can be S-transformed into g, resulting in asymmetry.
Remark 3.2. dxæ(y) defined in Eq. (Bap is not a metric on εy 12 Further, when S = Aff(V), the map inf (.f.///AAAV) 11 h v/ o gll is trivially zero by Cor D.
AA metric space is complete if every Cauchy sequence (a sequence where the distance between terms eventually becomes arbitrarily small) converges to a point within the space.
9This follows immediately from the fact that] | lv is norm and from the completeness of V.
10hhi amounts to "forgetting" the special role played by the zero vector.
X such that e = x for E X, where e the identity element G, and 91 (g2 x) = (91g2) )- a for all 91, 92 E G and x e X.
AA left action of a group G on a set
a map : G
See App. D.2 for derivation.

---

In the case of affine isometries Iso(V) = fe e Aff(V) 4'mm E O(V)) we show that the pair (Ev, d(s(()) constitutes an extended pseudo-metric space.
Proposition 3.1. The pair (Ev, d(so(v)) is an extended pseudo-metric space.
4 Intrinsic Affine Homotopy
The notion affine alignment allows us to introduce homotopic relations on Ev. We first derive the affine intrinsic preorder ž Aal on the space of encoders 13
Lemma 4.1. Let (X, d) be a hemi-metric space. The relation (æ: Za y iff d(x, y) = 0) is a preorder and it will be called the specialization ordering of d.
Proof. Goubault-Larrecq 17 Proposition 6.1.8).
Definition 4.1 (Intrinsic Affine Preorder). For encoders h. 8 E Ev, define the relation h Aff 9 iff (An O.
(9)
Lemma 4.2. The relation Zan is preorder on Ev.
Proof. Follows from ( h. < Viallv dAn(v)(h. g). see App. D.3
Intuitively, Aff captures the order of encoders such that higher-positioned encoders in the order can be S-transformed to the snwer-positioned ones. To derive the implications of A we introduce the notion of an encoder rank.
Definition 4.2 (Encoder Rank). For any h Ey let the encoder rank be rank(h) d dimg (Vr.). where Vn is the subvector space generated by the image of h. When rank(h) dimp (V), h is full rank encoder, else is rank deficient.
Theorem 4.1. For h, 9 Ev, we have
h AAll 9 < h (T o g) for some vp AII(V)
(10)
where, Th the orthogonal projection of V onto Vh. In particular, if dan(v) (h, g) then rank(h) rank(g). If addtion, we know rank(g) rank(h). then g must by an affine transfor- mation of g, i.e., h = 2b g for some v/ E Aff(V).
This allows to state our first notion of language encoder similarity: intrinsic affine homotopy.
Definition 4.3 (Exact Intrinsic Affine Homotopy). We say that two encoders h, g Ev are exactly intrinsically affinely homotopic and write h AH g if
dA g) 0 and rank(h) = rank(g).
(11)
For any h. E Ey, one can easily show that
h 2H 9 > (g AA h and h Aa g) <
H. (h, (h,g) 0,
Af((V)
(12)
which implies that Arf equivalence relation the of language encoders &v. Intuitively, two encoders h and are exactly intrinsically affinely homotopic, this means that both can be affinely mapped h. well as the other way around.
5 Extrinsic Homotopy
In $4 we explore methods for assessing how similar two language encoders are without reference to any downstream tasks. Here, extend our discussion to the extrinsic homotopy of language encoders. Since language encoders are primarily used to generate representations for downstream susks-such in transfer learning, illustrated by the sentiment analysis example in $ we argue that the key criterion the similarity of two encoders lies in how closely we can align predictions stemming from their representations
13 All left-out proofs can be found in App. D.2
reflexive and transitive relation on X.
The proofs of all claims in this section can be found in App. D.3
5

---

Principle 5.1 (Extrinsic Homotopy). Two language encoders h and g are extrinsically homotopic if we can guarantee similar performance on any downstream task and g might be used for.
The rest of the section formalizes this intuitive notion and describes its relationship with intrinsic affine homotopy. Let W be the vector space TR- and set AV, W) as the set of affine maps from V to W 16 We define Eو def Map(L', A and Ew Map ( W). Lastly, we formalize the notion of a transfer learning task as constructing a classifier that uses a language encoder's string representations. Particularly, we set Vx be the family of log-linear models follows
VN: Ey - P(EAx-:)(11), h h h softmax (Aff (Affy,((h)),
(13)
where v is the map
Aff Ev P(EW))()), h fvo DE Aff(V,
(14)
and softmaxx: INN- N-1 is defined for X R4, NY and e [M] as
softmaxx (x) exp (AI,,)
(15)
2m exp (Atr.)
Remark 5.1. Each p softmaxx (h(()) can be seen "probability distribution" " over [N]
(x(h) fp(a y): [N] - [0, = softmax, o +(((y))) | vb Aff W)).
(16)
Through our standard recipe from Remark 3.1 we can define the following hemi-metrics on Ey.
Definition 5.1. For any encoders h. g E Ev. we define
('alv.w) (h.g) edt d. (Aff (h), Affy d (vv. (h, del de (V,(h),
(17a)
(17b)
Notice that we use d rather than d in Def. [.1 since we are interested in how closely we can bring h and g when we affinely transform both of them-this corresponds to independently affinely transforming the encoders for the same transfer learning task. In particular, Eq. (17b) measures how different two encoders are on any transfer learning task, formalizing the notion of extrinsic homotopy (ef. Principle 5.1), captured by the following definition.
Definition 5.2 (Extrinsic Affine Preorder). An encoder h E Ey is exactly extrinsically homotopic g Ey if dii (v.
Analogously to Def. 4.1 we use (VV,)) (h, g) to define preorder.
Definition 5.3 (Extrinsic Affine Preorder). For two encoders h. g e Ev, we define the relation
h Ext 9 iff dvv.,)(hh.)) = 0.
(18)
Lemma 5.1. The relation h Exxt 9 is preorder on Ey.
(h. We now relate d d" (h.g. g) and (Vvv,a(h,, g) from Def. 5.1 and daif(v) g) from $4
Lemma 5.2. Let h Ev. We have
1. There exists a constant c(2) > O such that for any up E A))(V,W)
d (sofmmaaaa h), VN(g)) & (A))livolll
2. dlt (Vv.s)( (..)))mnv...
"Given affine map f: V W, there is unique linear map A fin L(V. W) and b W such that for every E we have f(v) A v v b.
The subscript oc in d and de.,w is used to insist on that considering the supremum distance dog A N- and W, respectively.
Exact extrinsic homotopy is asymmetric.

---

3. d) =
d AH(VWw) (h. (h.g) = = d (VV.A) (h. g) = 0.
Lem. 5.2 shows that Exx is finer than Afll- This means that the affine intrinsic preorder is contained in the extrinsic preorder, i.c., h Aail g = h Exx g- Lastly, we can show that di dvvv,a) (h. g) is upper bounded by the intrinsic hemi-metric d'A An(VY
Theorem 5.1 (e-Intrinsic = ((c) Extrrnxxinss,), Let h E &y be two encoders. Then, (ivv.a)(h, & c(A) d' (e
6 Linear Alignment Methods for Finite Representation Sets
$$ and introdduee ways of comparing language encoders as functions, which holistically charac- terizes relationships between them. We now address more practical concern: Given language encoders h and 9. how can we approximate their similarity in practice? Rather than comparing compare them over a finite In=1 We combine )'s representations given by h and g into matrices H, G E R where we denote Hy, h (ÿ) and Gy. (z). We can approximate the notions of similarity from $ by optimizing over the affine maps Aff(V) (for example, using gradient descent). Particularly, we approximate intrinsic similarity as
(y): > R with g (y) >* IeD over the entire > [9
of strings y (y("))
d dAm(v) (H, G)
inf max H y. (EAR(V) WE
v Gy
(19)
and extrinsic similarity for some task task-specific fixed el as
(H, G) der
inf max |softmax(( a/ Hy softmax(1/ G. Gy,.ll.. (FA(WW) wEy
(20)
Unfortunately, the max over y makes the optimization in Eqs. (19) and (0) difficult. For simplicity, commonly used linear alignment methods, which review for completeness.
Orthogonal Procrustes Problem. Rather than optimizing the infinity norm over Y as Eqs. 19 and (20), the orthogonal Procrustes problem finds the orthogonal transformation minimizing the Frobenius norm 34 by solving argmin Aeo(V) |H-AGE LG Given the singular-val value decomposition
H G UEVT, the optimum is achieved by UV
20 Since the argmin is over O(V), this defines
an extended pseudo-metric space by Prop. 3.1
Canonical Correlation Analysis (CCA). CCA 20 is a linear aligmment method that finds the matrices A, that project H and G into subspaces maximizing their canonical correlation. Let A., and B be the jth column vectors of A and B, respectively. The formulation is as follows
Pg sup corr HH..,,,,,, GB.g) s.t. Viej HA, 1 HA.,, Visj GB GB.,, (21)
B.J
The representation similarity is measured in terms of the goodness of CCA fit, e.g., the mean squared CCA correlation RoCA Ei / D. We can reformulate the CCA objective in Eq. 21
inf A,B
1 1 A H -
G
s.t. (A (ATH)(A TH) ( GB TJ==I.
=I
(22)
Given the singular-van value decomposition H G UEVT, the solution of Eq. (22 is (Á, B) ((HH))- (G V), where (HH and (GGT)-1 whitening transforms of U and V. Assuming the data is whitened during pre- pre-processing, CCA corresponds to linear alignment under an orthogonality constraint, equivalent to the orthogonal Procrustes problem; see also App. E
CCA Extensions.
inplection-weighee CCA (PWCCA) 30 also finds alignment matrices with CCA but applies weighting to correlation values ps report the goodness of fit. Given the canonical vectors Á, PWCCA reports Fpw DD aspps Σ, a4 where a; = Σ, KA H.
For simplicity, we assume that h and g both map R
20e App. for the derivation. 21 PCCA extensions beyond PWCCA are dicussed in App.
7

---

Laver
Laver
100
475
10-
201-
SST-2
10
3007
300
2013
100
145
Max Ervor Noraa.
100
101-
CoLA
420
302
3014
1263
10.
301-
303-
323-
MNLI
570
103
1
Figure I: Asymmetry between ELECTRA (E), RoBERTa (R), and MULTIBERT encoders (M1-M25) across layers. For each pair of the encoders M and M we generate training set embeddings H(), H E x for SST-2, COLA, and MNLI. We then fit H to HO) with an affine map and report the goodness of fit through the max error L2 norm, i.e., an approximation of d(H(), H()) on row and column of the grid. Full results across GLUE tasks are shown in Figure
Non-Alignment Methods. While explicitly (linearly) aligning representations, CKA 22 eval- uates the kernel similarity between representations. CKA computes the normalized Hilhert-Schmidt independence (18) between centered kernel matrices K and KG where K.H k(H,,, H,.), and KG = k(GG,. G3 for a kernel function k, i... tr Linear CKA, where k(H,, Hy,) H. Hj., commonly used.
KH ("TKK"""""KKKKKKK"K"
Experiments
We now explore the practical implications of our theoretical results. We conduct experiments ELECTRA (6), ROBERTA 28 and the 25 MULTIBERT (35] encoders, which are architecturally
identical to BERT-BASE II models pre-trained with different seeds. We report results the training
sets two GLUE benchmark classification tasks: SST-2 (38 and MRPC [4]. When reporting
and de. from Eq. 19) and Eq. 20 we use the L2 norm for simplicity and approximate dy(V,A) as
((V.) (H, G) sup inf max ||softmax(( Hy.)) - softmax(ú G2 (23) ('A (eA BEy
The experimental setup and compute resources are further described in App.
The Intrinsic 'Preorder' of Encoders. We first investigate whether the asymmetry of dam(v) is measurable in the finite alphabet encoder representations. Figure shoowss distinet vertical lines for both tasks indicating that there are encoders that are consistently easier to affinely map to (-M)). This seems to be rather independent of which encoder map from (M-)). We further that this trend is task-independent for early layers but diverges for later layers.
The Influence Encoder Rank Deficiency. role in affine mappability; exact affine homotopy is only achievable between equal-rank encoders ?2 With this mind, we return to our findings from Figure to evaluate whether the observed differences
discussed in $ the encoder rank plays a pivotal
We provide additional experiments the role of the encoder rank in App.

---

SST-22
MIRPC
CoL.A
RTE
MNLI
QQP
0.30
0.55
0.38
10.3.9
0.30
0.06
0.224
11,000
1.489
0.30-
0.25
0.20
0.52
0.30
0123
3.64
0.25
0.28
0.70
0.03
0.20
0.17
0.10
0.10-
0.20
0.21
3.01
0.15
0.20
0:00
1000
0.00
0.30
0.20
0.03
0.05
0.05
B.06
0.30
0:15
0.05
0.10
Figure 2: For ELECTRA (E), RoBERTa (R), and MULTIBERTs (MI-M25), we plot extrinsic (dg)
against intrinsic similarity (dAm(V)) across GLUE tasks. We group the points by how well we can map to each encoder ("M), and display the median, well as the first and third quartiles as vertical and horizontal lines. We additionally show the linear regression from daa(y) to dg-
between encoders can be attributed to difference in measurable rank. Due to the inaccuracies of computing the rank numerically, we approximate the encoder rank using the rank to precision e as the number of representation matrix singular values larger than some TR 23 We find statistically
significant (p-value 0.05) rank correlation with the median intrinsic distance d'm(v) when mapping to the corresponding encoder for RTE (p = 0.312), MRPC (p 0.609), and QQP (p 0.389). We find no statistically significant correlations with the median distance when mapping from from the corre- sponding encoder. This difference in encoder ranks could, therefore, partially cxplain the previously observed differences affine mappability as some encoders seem learn lower-rank representations.
A Linear Bound on Extrinsic Similarity. Lem. 5.2 derives relationship between affine intrinsic and extrinsic similarity. To evaluate strength in practice, we measure Spearman's 's Rank Correlation (p) and Pearson Correlation (PCC) between intrinsic measures introduced in S and the extrinsic measures de and di (VVAY PCC measures the strength and direction of a linear relationship between two random variables, whereas Spearman's p additionally evaluates the variables' monotonic association. de is computed by training linear classifier e E AAf((V on the final MULTIBERT layer for each task. Further, we report (v..)) as the maximum L2 loss for a large number of randomly generated classifiers e on the final laver of each MULTIBERT encoder. We generate show significant, large linear correlation linear, non-alignment method- does not visualizes the linear relationship explicitly
100 such classifiers for a range of GLUE datasets 25 Table
prevalent in all linear alignment methods, whereas CKA
capture extrinsic behavior as faithfully. Further, Figure for all considered GLUE datasets.
8 Discussion
We set out to explore homotopic relationships between language encoders, augmenting existing work the similarity of finite representation by holistically studying encoder functions. In particular, the general framework of S-homotopy allows us to study any functional relationship between encoders, enabling the exploration of many types of encoder relationships. As first step in this direction and a concrete example, explorre affine homotopy, discussing what it means to be able align as they allow us to measure a notion of (asymmetric) distance between point-an encoder-and the set of all affine transformations of another encoder. Lem. 5.2 then connects the intrinsic,
models with affine transformations. Here, Hausdorfl-Hoare maps prove useful,
FFlloowin Press et al. [31]. choose singular value of the x D representation matrix and 6p the float machine epsilon. We note that rank to precision and the recovered correlation may depend on the chosen e.
di Cp max(N D) for 5, where is the largest
TTe generation process is described in App.
25Thee computational expense of computing dv(v.a) restricts this analysis to limited set of classifiers. depending the alphabet size. See App. B for discussion.

---

Intrinsic Measure
dAA(V)
Orth. Procrustes
Roca
PWCCA
Linear CKA
Lin. Alignment-Basse
Yes
Yes
Yes
Yes.
No
dAm(()
p
0.080
0.095
0.172
0.016
0.088
PCC
0.545*
0.937
0.932
0.970
0.231 *
SST-2
0.621
0.157
0.071
0.231
0.295
dN (V(VA)
PCC
0.723
0.539
0,457*
0.566*
0.320
0.309*
0.250
-0.001
0.220)
0.214*
dv
PCC
0.707*
0.697
0.733
0.743*
0.241
MRPC
d dv(V
0.231
0.025
0.178
0.059
0,030
PCC
0.790*
0.755*
0.879
0.875#
0.174
0.534
0.053
0.037
0.308
0.185*
dv
PCC
0.570*
0.401 *
0.429*
0,250*
0.078
RTE
iu (VVA)
0.234*
0.317
-0.147*
0.338
0.240
PCC
0.718*
0.870
0.778
0.780
0.205
dv
0.196*
0.006
0.040
0.185*
0.165*
PCC 0.204*
0.529
0.553*
0.550*
0.215
CoLA
0.348
0.078
0.133*
0.340
0.380
((V.))
PCC
0.429*
0.664
0.318
0.786*
0.513
Table Spearman's 's Rank Correlation Coefficient (p) and Pearson's 's Correlation Coefficient (PCC) between intrinsic measures introduced $6 and the extrinsic similarities dy and du VVV,A) across various GLUE datasets. indicates p-value 0.01 (assuming independence).
task-independent, similarly to extrinsic similarity-the similarity performance on downstream tasks. Concretely, it derives a linear relationship between the intrinsic and extrinsic dissimilarity for any fixed affine transformation v (i.e., a fixed downstream task). Thm. 5.1 discusses a stronger bound, namely the worst-case extrinsic dissimilarity among all downstream linear classifiers, i.e. among all possible tasks. Further, by accounting for the asymmetries of encoder relationships, we augment the work on similarity in proper metric spaces [ 36 42
Although encoders may not be affinely related in practice, empirical evidence in 7 suggests that notions of affine order still surface (cf. Tab. [ Fig. 2) particularly as differently initialized BERTs exhibit variations in downstream task performance [29]. While other similarity measures, such as those used in seed specificity tests [12), are designed to remain invariant to initialization changes, our results indicate that intrinsic affine homotopy appropriately sensitive to them. This sensitivity raises new questions about the landscape of pre-trained encoders; seen in Fig. T asymmetry in intrinsic affine similarity among similarly pre-trained encoders impacts downstream performance, corroborated by Lem. 5.2 and empirical results in Tab. Differences in representation ranks may partly explain this sommetry-mmapinng between artificially generated rank-deficienn encoders yields mostly symmetric affine distances (cf. Fig. 3). Another explanation might be that easy-to-leary encoders might be approximately linear combinations of others, making them easy to map to but not necessarily from. Overall, findings highlight the need account for directionality in encoder similarity measures to address the asymmetry inherent this problem.
9
Conclusion
We discuss the structure of the space of language encoder in the framework of 5-homotopy the notion of aligning encoders with a chosen set of functions. formalize affine alignment between encoders and show that it provides upper bounds on the differences in performance on downstream tasks. Experiments show our notion of intrinsic affine homotopy to consistently predictive of downstream task behavior while revealing an asymmetric order in the space of encoders.
10

---

Broader Impact
This paper presents foundational research about the similarity of language encoders. To the best of our knowledge, there are no ethical or negative societal implications to this work.
Acknowledgements
Ryan Cotterell acknowledges support from the Swiss National Science Foundation (SNSF) as part of the "The Forgotten Role of Inductive Bias in Interpretability" project. Anej Svete is supported by the ETH AI Center Doctoral Fellowship. Robin Chan acknowledges support from FYAYC. We thank Raphaël Baur and Furui Cheng for helpful discussions and reviews of the current manuscript.
References
[1] Yamini Bansal, Preetum Nakkiran, and Boaz Barak. 2021. Revisiting model stitching to compare neural representations In Advances in Neural Information Processing Systems, volume 34, pages 225-236. Curran Associates, Inc.
[2] Saaid Baraty, Dan A. Simovici, and Catalin Zara. 2011. The impact of triangular inequality violations on medoid-based clustering] In Foundations of Intelligent Systems, pages 280-289 Berlin, Heidelberg. Springer Berlin Heidelberg.
[3] Enric Boix-Adsera, Hannah Lawrence, George Stepaniants, and Philippe Rigollet. 2022. Gulp: prediction-basee metric between representations In Advances in Neural Information Processing Systems, volume 35, pages 7115-7127. Curran Associates, Inc.
[4] Dmitri Burago, Yuri Burago, and Sergei Ivanov. 2001. Course Metric Geometry, Mathematical Society, Providence, RI.
American
[5] C. Chang, W. Liao, Y. Chen, and L. Liou. 2016. A mathematical theory for clustering in metric spaces IEEE Transactions on Network Science and Engineering, 3(01)22-16
[6] Kevin Clark, Minh- Minh-Thang Luong, Quoe V. Le, and Christopher D. Manning. 2020. ELECTRA] Pre-training text encoders as discriminators rather than generators) In International Conference on Learning Representations.
[7] Ryan Cotterell, Anej Svete, Clara Meister, Tianyu Liu, and Li Du. 2023. Formal aspects of language modeling arXiv preprint arXiv:2311.04329.
[8] Adrián Csiszárik, Péter Körösi-Szabó, Ákos Matszangosz, Gergely Papp. and Dániel Varga. 2021. Similarity and matching of neural network representations' Advances in Neural Information Processing Systems, volume 34, pages 5656-5668 Curran Associates, Inc.
[9] Alexander D' Amour, Katherine A. Heller, Dan I. Moldovan, Ben Adlam, Babak Alipanahi. Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yi-An Ma, Cory Y. McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan. Christopher Nielson, Thomas F. Osborne, Rajiv Raman, Kim Ramasamy, Rory Sayres, Jessica Schrouff, Martin G. Seneviratne, Shannon Sequeira, Harini Suresh, Victor Veitch, Max Vladymyrov, Xuezhi Wang, Kellie Webster, Steve Yadlowsky, Taedong Yun, Xiaohua Zhai, and D. Sculley. 2020. (Underspecification presents challenges for credibility in modern machine learning Journal of Machine Learning Research, 23:226:1-226:6
[10] Sanjoy Dasgupta and Philip M. Long. 2005, Performance guarantees for hierarchical clustering Journal of Computer and System Sciences, 70(4):555-569, Special Issue on COLT 2002.
[11] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre training of deep bidirectional transformers for language understanding In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational I Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 4171- 4186, Minneapolis, Minnesota. Association for Computational Linguistics.
11

---

[12] Frances Ding, Sean-Stamislas Denain, and Jacob Steinhardt. 2021. Grounding representation similarity through statistical testing In Advances in Neural Information Processing Systems, volume 34, pages 1556-1568. Curran Associates, Inc.
[13]
Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannanch Hajishirzi, and Noah Smith. 2020. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping arXiv preprint a:Xi::2002.063355.
[14] William B.
Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases] In Proceedings of the Third International Workshop on Paraphrasing (IWP2005).
[15] Li Du, Lucas Torroba Hennigen, Tiago Pimentel, Clara Meister, Jason Eisner, and Ryan Cotterell. 2023. [ measure-theoretic characterization of tight language models) Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), pages 9744-9775 9770, Toronto, Canada. Association for Computational Linguistics.
[16] Bolin Gao and Lacra Pavel. 2017. On the properties of the softmax function with application in game theory and reinforcement learning arXiv preprint arXiv:1704.0000 170400085, 1704.00805.
[17] Jean Goubault-Larreq, 2013, Non-Hausdorff Topology and Domain Theory: Selected Topics in Point-Set Topology New Mathematical Monographs. Cambridge University Press.
[18] Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. 2005. Mcasuring statistical dependence with Hilbert- norms In Algorithmic Learning Theory, pages 63-77, Berlin, Heidelherg. Springer Berlin Heidelherg.
[19] William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2016. Diachronic word embeddings
reveal statistical laws of semantic change In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 1489-1501, Berlin, Germany. Association for Computational Linguistics.
[20]
David Roi Hardoon, Sándor Szedmák, and John Shawe-Taylor 2004. Canonical correlation analysis: An overview with application to learning methods Neural Computation, 16:2639 2664.
[21] Max Klabunde, Tobias Schumacher, Markus Strohmaier, and Florian Lemmerich. 2023. Simi larity of neural network models: A survey of functional and representational measures arxiv preprint arXiv:2305.00329.
[22] Simon Korblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. 2019. Siiillriity of neural network representations revisited, In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 3519-3529.
PMLR.
[23] Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini. 2008. [Representational similarity analysis connecting the branches of systems neuroscience Frontiers in Systems Neuroscience,
[24] Serge Lang. 2002. Algebra Springer New York.
[25] W. Lawvere. 2002. Metric spaces. generalized logic. and closed categories. Theory and Applications of Categories No. 1 (2002) pp 1-37.
[26] Karel Lenc and Andrea Vedaldi. 2019. Understanding image representations by measuring their equivariance and equivalence International Journal of Computer Vision, 127(5)
[27] Li, Yixuan and Yosinski, Jason and Clune, Jeff and Lipson, Hod and Hopcroft, John. 2015. Convergent learning: Do different neural networks learn the same representations? In Pro- ceedings of the 1st International Workshop Feature Extraction: Modern Questions and Challenges at NIPS 2015, volume 44 of Proceedings of Machine Learning Research, pages 196 212, Montreal, Canada. PMLR.
[28] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach| ArXiv, abs9.9
12

---

[29] R. Thomas McCoy, Junghyun Min, and Tal Linzen. 2020. BERTs of feather do not generalize together. Large variability in generalization across models with similar test set performance In Proceedings of the Third BlackboxNI.P Workshop on Analyzing and Interpreting Neural Networks for NL.P, pages 217-227, Online. Association for Computational Linguistics.
[30] Ari Morcos, Maithra Raghu, and Samy Bengio. 2018. Insights on representational similarity in neural networks with canonical correlation In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc.
[31] William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. 2007. Numerical Recipes: The Art of Scientific Computing 3rd edition. Cambridge University Press.
[32] Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. 2017. Sveca Singular vector canonical correlation analysis for deep learning dynamics and interpretability In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
[33] Yuxin Ren, Qipeng Guo, Zhijing Jin, Shauli Ravfogel, Mrinmaya Sachan, Bernhard Schölkopf. and Ryan Cotterell. 2023, All roads lead to Rome? Exploring the invariance of transformers representations) arXiv preprint arXiv:2305.
[34] Peter H. Schönemann. 1966. A generalized solution of the orthogonal procrustes problem Psychometrika, 3101111-10.
[35] Thibault Sellam, Steve Yadlowsky, Jason Wei, Naomi Saphra, Alexander D' Amour, Tal Linzen, Jasmijn Bastings, Iulia Ture, Jacob Fisenstein, Dipanjan Das, et al. 2022. TThe MultiERRTs BERT reproductions for robustness analysis In International Conference on Learning Repre- sentations. OpenReview.net.
[36] Mahdiyar Shahbazi, Ali Shirali, Hamid Aghajan, and Hamed Nili. 2021. Using distance on the Riemannian manifold to compare representations in brain and in models, Neurolmage, 139:118271.
[37] C.G. Small. 2012. The Statistical Theory of Shape Springer Series in Statistics. Springer New York.
[38] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts 2013. Recursive deep models for semantic compositionality over a sentiment treebank In Proceedings of the 2013 Conference on Empirical Methods in Nat- ural Language Processing, pages 1631-1642, Seattle, Washington, USA. Association for Computational Linguisties.
[39] Hrishikesh D. Vinod. 1976. Canonnical ridge and econometrics of joint production Journal of Econometrics, 4:147-166.
[40] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. GLUE: A multi-task benchmark and analysis platform for natural language understanding In International Conference on Learning Representations.
[41] Fei Wang and Jimeng Sun. 2015. Survey on distance metric learning and dimensionality reduction in data mining Data Mining and Knowledge Discovery, 29(2) 564.
[42] Alex H. Williams, Erin Kunz, Simon Kornblith, and Scott Linderman. 2021. Generalized shape metrics on neural representations In Advances in Neural Information Processing Systems, volume 34, pages 4738-4750. Curran Associates, Inc.
[43] Peter N. Yianilos. 1993. Data structures and algorithms for nearest neighbor search in general metric spaces) In Proceedings of the Fourth Annual ACM-SIAM Symposium Discrete Algorithms, SODA '93, page 31 311-321, USA. Society for Industrial and Applied Mathematics.
[44] Ruiqi Zhong, Dhruba Ghosh, Dan Klein, and Jacob Steinhardt. 2021. Are larger pretrained language models uniformly better? Comparing performance at the instance level In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3813-3821 Online. Association for Computational Linguistics.
13

---

A
Notation
Symbol
Meaning
Introduced
D
Size of string representations.
V
D-dimensional -evvector space.
R
R+ U (co) The set (1...., N) for N E N.
Def
[M
N
A A-1
The - 1-dimensional probability simplex. The vector space of language encoders.
&y
&
The subspace of bounded language encoders. The set of (invertible) affine transformations on V. The group of all invertible linear maps on to itself.
AT(())
GL(V) O(V)
The orthogonal group of V; the group of norm-presee comm-reseerving linear maps on
3.3
d
A general (hemi-)
Def. 3.2
d
The 3o (uniform convergence) norm on &v. The Hausdorff-Hoon map of d. The symmetrized Hausdorff Hoare map. The Hausdorff-Hoose map where the sets in the arguments are computed by applying all transformations two input elements.
$.2
d
Der. 3.3
ae sm
Def. 3.3
d'
Eq.
S to the
ds
One-sided affine alignment measure over S. The S-homotopy norm of an encoder.
Eq.
Eg. 8a
A preorder.
An equivalence relation.
Table 2: summary of notation used in the paper.
B Limitations
In this section, address some of work's limitations.
Non-Linear Encoder Relationships. This work focuses on affine similarity between encoders. As find and discuss in $4 with the example of MULTIBERTs, language encoder representations may generally not be exactly affinely related. Nevertheless, understanding the affine-homotopy relationships &y still helps us make conclusions about practical findings in
Linear Classifiers. Our work provides precise theoretical guarantees on the performance of linear classifiers applied to affinely-related encoders. In practice, task fine-tuning take the form of more complex models, such as re-training entire pre-trained models. This work does not cover such more complex fine-tuning techniques.
Numerical Approximations. To bridge our theoretical findings on affine homotopy relationships in yy with their practical implementations in $7 we concede several approximations. For instance, while d2t (V) is valuable analysis, optimizing Eq. (3 directly is computationally challenging and requires costly approximations. Similarly. in computing intrinsic distances across all representation layers in Fig. I we optimize for mean squared error (MSE) and evaluate the maximum loss instead of optimizing for it directly, which serves as approximation of d that results in more stable optimization given computational constraints. Finally, we address numerical inaccuracies encountered during singular value decomposition (SVD) computations in 7 which we mitigate by tuning the rank according to precision
C Additional Related Work
In this section, complement discussion in $6 and $8 with additional related work.
14

---

Representational and Functional Similarity. Our work is related to the ongoing efforts to quantify the similarity between neural networks. Much related work discusses similarity measures in terms of the invariance properties of neural networks [2 22 inter alia]; see Klabunde et al. [2]] for a recent comprehensive survey. Notably, Klabunde al. [21] compile various representational 32 39, infer alia] and functional ways to measure similarity, which are related to our notions of intrinsic and extrinsic homotopy. respectively. Whereas our notion of intrinsic affine homotopy fits into the class of linear alignment-based measures [22 27 42 inter alia] as described in $6 the notion of extrinsic similarity fits into the broader category performance-basee functional measures [ 8 26]. Most relevantly, Boix-Adsera et al. [3] propose the GULP metric that provides bound on the expected prediction dissimilarity for
19 23 27
ridge regression.
Similarity Mcasures as Metrics, A line of work draws from statistical shape analysis 37 to
motivate the development of similarity measures that are that conform to axioms of valid metrics
3,36 42]. Learning within proper metric spaces provides certain theoretical guarantees 2 S
43 For example, Williams et al. [42] derive two families of generalized shape metrics, modifying existing dissimilarity measures to ensure they meet metric criteria. Notably, one of these generalized shape metries based on linear regression over the group of linear isometries, similar to the approach derived for encoder maps in Prop. 3.1
Understanding Similarity of Language Encoders. Finally, several previous works characterize the landscape of language encoders and their sensitivity to slight changes to the pre-training or fine-tuning procedure [9] 33 44] This prompted multi-seed releases of encoders such as BERT 35 4 that are frequently used for robustness or sensitivity analysis [2] 33 similar to the one presented in this work.
D Addenda on Affine Homotopy
In this section, we provide additional derivations and proof' complementing the discussion in
$
D.1 Preliminaries on Hemi-Metric Spaces
Definition D.1. Let (X, d) be hemi-metric space. The open ball B(x,e) E) of center and radius <> O, is the set (y E d(z, < cj. The open balls form a base for the open ball topology 26
Lemma 4.1. Let (X, d) be a hemi-metric space. The relation (x Za y iff d(x, y) = 0) is a preorder and it will be called the specialization ordering of d.
Example D.1. An example of specialization ordering the prefix ordering of strings predix More precisely, for any y E Σ", we define dy:* (y, y to be zero if is a prefix of y and 2 otherwise, where n is the length of the longest prefix of y that is also a prefix of y'. Then (Σ* dy:s) is a hemi-metric space whose specialization ordering is Epefix
Lemma D.1. Let (X, d) be hemi-metric space.
1. The set e X d2' F) 0) is exactly the closure of Fr in the open ball topology.
2. For any x, x' E X, we have the inequality d= E) & d(z,x') d'd E). of is metric. then d' E) is 1-Lipschitz from (X, d) R...
3. Let = P(X) be any space of non-empty subsets of X. The Hausdorff-Hoare map dl is hemi-metric on 2. Its specialization ordering ZH is given by E. Zzzt E' E cl(F') iff cll E)) = cl(F').
Proof. See Haabault-Larreey
17 Lemma 6.1.11, 1. Proposition 6.2.16 & Lemma 7.5.1].
This, by definition, is the topology generated by all open balls.
A reflexive and transitive relation on X.
Defined by y Spreis y if y is prefix s/"
29 Here, the closure is with respect to the topology defined by d.
15

---

D.2 Additional Derivations: Affine Alignment Measures
Remark 3.2. dxF(V) defined in Eq. (8a) is not a metric on vy Further, when Af((V). the map inf (. 11 h g x trivially zero by Cor. D.
Proof. see that d'ar() not a metric, consider the following two encoders: g = |g e, where e V is any fixed vector, and h any map from Σ* to the ball B(Ov, 1) of radius one. In such case, we have dAmrv g) sD Even on the space of bounded encoders 31 daf(()) is not metric. We provide the following counter-example: Let h any rank encoder, e.g., h can be any map that sends the first It strings to the basis of V. Let A be a non-invertible linear map of V and set A(h). Then clearly dan(v)((, = O, but d'm(v) (g, h) can not be zero for dimensionality reasons (see Thm.
Lemma D.2 (Hausdorff Distance). Let E, E' c Ev. The map
d% SUR (E,""EEEE)) max(d E"), (E, EE) = sup (h, E) (h, E¹)
(24)
hefy
is an extended pseudo-metric ((Ev)MO).
Proof. It follows readily from Lem. D.1 See also Burago et al. 4 $7.3.1].
For any affine subgroup c Aff(V). let S(h) (v(h) | / S). then follows immediately from sh"(h, g) def d sm (S(h), is extended pseudo-metric εv. Lem. D.2 that the map d= s9"
Lemma D.3. For any h.g d Ev. any vp (V) and any non-empty S = Ev, have ds( h g) = do-is(h,gg)
(25)
In particular, diso(v) h g) diso(v) (h.g).
Proof. Lem. D.3 follows by definition die (1b h vb g) dh, 1h up g).
Proposition 3.1. The pair (Ev, diso(v)) is an extended pseudo-metric space.
Proof. Using Lem. D.3 one can show that diso(v) (h, g) dH, sym (Tso Tso(g)), where Iso(h) def { h: Iso( J]. The proposition follows then from Lem. D.2
For any the Aff(V) and any h Ey, then have
1b h Es 4> h E En |//100 00.
Lemma D.4.
1. If E Eg, then
h [so(V)
|
Th:
where The denotes the radius of h, which define as the radius of the minimum enclosing ball of the set h(E.), and the || [[so(V) norm is defined as in Eq.
2. For any & Aff(V) and a subset Aff(V) normalized by va and containing T. Then
1&b his 1 [|llss
where the || I|s norm is defined as in Eq.
Proof.
SScce App. D.2 for derivation.
Recall & (h Ev h(xx*) is bounded).
30e set S is normalized by if - E S for all & S.
16

---

1. & T be the translation moving the center of the ball enclosing h(E*) to the center Dy Hence
(V)
$ | f ht
th
Now observe that for any other isometry v * t, then Tyjc = The The ball B(v,, || o h| clearly contains all points in ve o h(E*), hence by definition of the radius Teich we must have | htll co E The which finishes the proof of I.
2. Write db = Otin t, with t E T. We then have
|| o fells = inf 6E9
|
inf V Tin
do tmn tcch)
oeS
Es
Note that Φ - vhin o oo Viin o is by definition a bijection of S, hence
|/ h
inf
(inn( h)ll
scS
inf sup |thnn ((db h)(y)) Iv
dES yeΣ*
< ||vin//v inf sup 1( (y))ll oES yes
= =||vnn/lve v |h|s.
Corollary D.1. Let S = T such that infées |/ lv g/ for all h.g g Es.
33
Then, de ds g) def infe 1 o eb
Proof. Note that ds( g) < ||riilll ds ds(h, g). which follows from Lem. D.4 Hence
ds dh inf ds( h, .)
weS
< inf Viuallv (h
WES
D.3 Proofs: Intrinsic Affine Homotopy
Lemma 4.2. The relation Zail is preorder on Ey.
Proof. Since d dam(v) (1b h g) & Vviiallv (An(v)(h, g) (see Lem. D.4
dAa(v)(h, g) <> (h, 0.
Therefore, the relation AA# is the specialization ordering of the hemi-metric d% AfrVyy
Theorem 4.1. For h, g Ev, we have
h ZAa h= (/ o g) for some vp Aff(V)
(10)
where, Th is the orthogonal projection V V onto Vn. particular, if danty) g) = then rank(h) rank(g). addrion, we know rank(g) rank(h). then g must by an affine transfor- mation of i.e., h. = 1/ 9 for some 1o Aff(V).
Proof.
1. Recall from $3.2 that Ey is complete with respect the metric d.. The condition d(h, A)(V) 0 simply means that there exists dn Aff(V) such that lim Un h in εv, in other words E Aff i.c., h lies the closure of Aff(g) in Ev.
This is. for example, the case if S is group and there exists o S such that |dnn lv L. Aff(F).
17

---

Let BB C Σ" such that h(Bb) is basis for V4. Therefore, there exists > 0 such that any family
(Ny)y E II B(h(y).c c)
yEBn
has rank dimæ (iim)(VV....... This shows that there exists z 1 such that for any n 2> N one has h On gloo < c, and
rank(Ion o g(v)) y E Bb)) rank(h).
Which implies in partieular
dim dimR (Vg) i.e., rank h rank
(26)
2. If rank(g) = rank(h) (h) = D, then lim On = Φ, where á is the affine map given by g - hly) for y E Bn. Indeed, for any v = L Agb V, we have
| (w) < h th glo
Σ
Dul < c/h Un o Iely
(Eh
for some constant e >> 0, since all norms on V are equivalent. Hence, lim 10 -0-llllV= 0, which shows the claim. Accordingly, we must have 6 e og- h.
Now we can prove Eq. 110
3 >. Given that |h T on gll
h On o gll gl we also have lim To on og h.
Thh + n|on T Note that Write x' for the orthogonal projection V and set Th,n
lim Thnn = Thh Accordingly,
lim Un g) h, n-
h.n" where vin Th.n In h. From this, deduce that
dAm[v) (h, TA g) = 0.
Now applying 2. yields h d(T og) for some on Aff(V)), h p(TT o g) where Φ dy ① Th Aff(V).
3 <. Assume now that h @(Tn g) for some & AT((). Then h lim Th (g),
where Th. + The which shows the desired implication.
D.4 Proofs: Extrinsic Homotopy
Lemma 5.2. Let h. Ev. We have
1. There exists a constant c(A) > 0 such that any up E Aff(VWW)
(soottsstta Vx) & ((A)
2. dlt (iv,)) g) < c(A)d (h.
3. dA g) di datIV.w (h.
(vv.aa(hh.9 -h 00 d
Proof.
1. Clearly,
d 2 (softmaaas o h), VN(g))
& (()) h, Affy:w(g))
< c(A)
inf | v gllx,
= c(A) inf 1w[h - - go
WEAH(()
= -clAlleehllddssssshhh.
close enough to l(Bb))
18

---

where, the first inequality follows from the fact that softmax, is c())) Lipschitz for some constant that depends on [16] Proposition
2. & 3. are are immediate consequences of I.
Theorem 5.1 (e-Intrinsic = O(e)-Extrinsie). Let h. E Ev be two encoders. Then,
di & c()) (h.g).
Proof. Let Aff(V, W). There exists linear map A: V W and ov GL(V), such that v = A o and |A| = Accordingly, Lem. 5.2 yields
d (softmaxxx h). Vv < c(A) ( Affr.w(0))
(28a)
c(A)daR())(óO h.
(28b)
< c(1) sup (Wy o h,
(28c)
UVEAl(())
c ((())xmm(v)) (h
(28d)
Therefore di (VV, (h, c(A) dk (
E Addenda on Linear Alignment Methods for Finite Representation Sets
Linear Regression A common way to evaluate the similarity of two representation matrices H e RNxD and G e RNDD through linear regression. Linear regression finds the matrix À E DxD that minimizes the least squares error:
= argmin G - H-H = H AcRDxD
(29)
Let H QHRH and G = QGRa be the QR-decomposition of H and G, respectively. The goodness of fit commonly evaluated through the R-squared value R2 i.e., as the proportion of variance in G explained by the fit:
G
HÁ 2
1Q H
R
(30)
G
To derive Eq. (30), consider the fitted value G
G = HÁ H(H 'H
(31a)
QuRa (R OF RR Q
(31b)
QHQAG.
(31c)
The residuals are therefore
|G G 2 = tr((G - (G
(32a)
tr((G - tr((G-G)TG)
(32b, residuals orthogonal to fitted values)
tr(G - (
(32c)
G
(32d)
With this, compute the coefficient of determination as
G
G
|G
Q
|Q
R2
(33)
G
2
G
G
19

---

Orthogonal Procrustes Problem. Let G E RND and H RR NxD representation matrices. In the orthogonal Procrustes problem, we seek find the orthogonal matrix A that best maps to G:
argmin H - F. AcO(V)
(34)
Since
|G HA tr((G - (G- -G- HA))
= -tr(G G) - trtGGGHAH HA) - tr(A +
= -|G + |
an equivalent objective to Eq. (4
argmax AH, G)p AEO(V)
Let UEVT be the singular- value decomposition of H G. then
Å = argmax AH, F)F
(35a)
A=O(V)
argmax A, GH ) AcO(V)
(35b)
argmax A, UEV! ' AEO(V)
(35c)
argmax UT AV, Σ)F
(35d)
A=O(V)
where U AV is product of orthogonal matrices, and, therefore, orthogonal. Since Σ is diagonal, Eq. (35d maximized by UT ATÁV- which means that Å UVT.
Canonical Correlation Analysis.
We can rewrite the CCA objective from Eq. 21 as
max Lr B A,B
s.t. (A (ATH)(ATH)) (B"G)(BTG) = I,
(36)
which, by definition of the Frobenius norm, is equivalent Eq. (2). Let MHG HG Mμн HHT, MGG = TGT, and let UΣVT MHG be the singular-value decomposition of MHG- One can show that the optimum of Eq. 22 is found at (Á, ВB = (M U, M V). Because A B G, U, and are by definition orthogonal, we see that CCA first whitens the representations
(H, G) through (M M and then orthogonally transforms them. This provides the intuition behind close relationship between CCA and the Orthogonal Procrustes problem: For pre-whitened representation matrices, CCA (Eq, (22h) is equivalent to solving the Orthogonal Procrustes problem
(Eq. (34) ). To see this, let WH and WG be whitening transforms for H and G. respectively. Then, Eq. 22 is equivalent to
min |A WWWH B 3 A.BeO(V)
(37)
such that
(AWH = AAT I, (BWGG)BeGG] BBT =I.
(38a)
(38b)
Therefore, derive
min ||A WATWHH-B W A,BEO(V)
B = min
|A | [|| A
W
AB =0(()
(39a, A O(V))
min CcO(V)
W
(39b, C AB O(V))
which is equivalent to solving the Orthogonal Procrustes problem (Eq. on the whitened matrices W and G.G.
20

---

F Experimental Setup
In this section, we provide additional details about the setup and compute resources of the experiments in $7 To generate embeddings, we used the open-sourced code by Ren et al. [33 Further, for
Orthogonal Procrustes, CCA, PWCCA, and Linear CKA, we use the open source implementation by Ding et al. [2] Our complete code is added as supplementary material.
Models and Datasets. We first extract the D 768 dimensional training representa- tions for SST-2, MRPC, RTE. CoLA, MNLI, and QQP across all 12 layers of ELECTRA [6]. ROBERTA (28), and the 25 MULTIBERT 355 models from HuggingFace The models and the MRPC dataset are licensed under Apache License 2. O. The SST-2 dataset is licensed under the Creative Commons CCO: Public Domain license. The RTE dataset licensed under the CC BY 3. 0.0 license. The CoLA dataset is licensed under the CC BY-SA 4. 0 license. The MNLI dataset is li- censed under the General Public License (GPL). THE QQP dataset is licensed under custom non-commercial license 36 The dataset statistics are shown in Tab. 3 We note that for all experiments, MNLI and QQP were shortened to the first 10K training samples due to computational limitations.
Dataset
Task
Train Dataset Size
Domain
SST-2
Sentiment Analysis
67K
Movic reviews
MRPC
Paraphrase Detection
3,7K
News Mixed
RTE
Textual Entailment
2.5K
CoLA
Linguistic Acceptability
8.5K
Miscellaneous
MNLI
Natural Language Inference
393K
Multi-Genre
QQP
Paraphrase Detection
364K
Social QA
Table 3: Statistics for the used GLUE benchmark [40] datasets.
Hyperparameters. Each experiment was run using RiemannSGD" as an optimizer as it initially produced the best convergence when computing our affine similarity measures, Further, to account for convergence artifacts, we ran the intrinsic similarity computation optimizations each experiment for learning rates [1E-4, 1E-3, 1B-2, 1E-1] and extrinsic computations for [1E-3, 1E-2, 2F-2] and report the best result. When training the task-specific linear probing classifier x/ for dy entropy loss, RiemannSGD and optimize over the learning rates [1E-2, 1E-1, 2E-1, 4E-1]. For the computation of Hausdorff- Hoare map d!*, we fixed Ir of 1E-3 to save compute resources, this Ir generally leads to the best convergence in previous experiments. We used a batch size 64 and let optimization run for 20 epochs, keeping other parameters at default. For reproducibility, set the initial seed to 42 during training.
37
the cross-
Generating Random Affine Maps. For the last experiment, we generate random affine maps. To approximate d' we sample the matrix entries of the affine map from (O 1). We then additionally normalize the transformed representation matrix this leads better convergence. To approximate d (v.)).) we fit linear probe on H 100 sets of randomly generated class labels, for the embeddings of each task. The predictions of that probe then become what G affinely maps to. In both cases, the seeds are set ascendingly from 0.
Compute Resources. We compute the embeddings on a single A100-40GB GPU, which took around two hours. All other experiments were run on 8-32 CPU cores, each with GB of memory. Computing extrinsic distances between 600 model combinations across both datasets usually takes 2-3 hours on 8 cores, whereas intrinsic computation is more costly, and can run up to 4 hours. Note our approximation of Hause maps (cf. Eq. (23 across all models is significantly more costly due to our sampling approach and can take up to 72 hours to compute on 32 cores for large datasets such as SST-2, and up to 12 hours for MRPC. The resources needed for initially failed experiments do not significantly exceed the reported compute.
36 "httss//uuggingffccccm co.goooocccc///ooccco/oo/o.ooco.
https//www..uurr.ccmmbbuuuu
butps//gihnbb.cm//eoop/eeees
21

---

MRPC
SST-2
768
768
(ful))
(fall)
Max Error Norm
0.0150
614
614
(80%)
(80%)
0.0125
461
461
(60%)
(60%)
307
307
0,0100
(40%)
(40%)
154
1.5
(20%)
(20%)
307
461
(20%)
(40%)
(60%) (80%)
(full)
(20%)
(40%)
(60%)
(80%)
(full)
.M.
M
Figure 3: The effect of artificial rank deficiency averaged across MULTIBERTs. For each pair of embeddings ( and HQ) from MUTLIBERTs M and M we generate additional rank- deficient encoders H and H with X,Y E (20%, .... 90%} of the full rank through SVD X%
Y%
truncation. We compute d( Y H x%% for each pair of possible rank-deficiency and finally report the median across all SUL.TIBERTS on row X and column Y on the grid. We additionally show row-, and column medians.
G Additional Experimental Results
The Influence of Encoder Rank Deficiency. Thm. we discuss how the relative rank of encoders influences their affine alignment and derive the equivalence relation ~A# conditioned on equal rank between encoders. To test the effect of unequal rank on affine alignment in an isolated setup, artificially construct reduced-rank encoders through singular value decomposition (SVD) truncation. In Figure 3e expectedly find a trend in how the encoder rank influences affine mappability. We additionally highlight that the computed distances clear differences when mapping to ("M),, rather than from (M") an encoder. Finally, we note the trend in the diagonal indicating that mapping between encoders of the same rank becomes easier between lower-rank encoders.
rather symmetric, with no
22

---

Layer
Layer
Layer
Laver
303-
142
1423
34.0
SST-22
3419
CoLA
400
M/
570
145-4
342-
101-
301-
3015
345
MNLI
Max Eiror Norm
135
35
30.0
N40
345
30.00
1415
522
165
3.5
30.0
1015
125
3000
10x
360
309
142-
1429
QQP
3455
142
1455
1455
103
315
RTE
302
011-
101
101
411
MRPC
33-
50
10
35
300 340
355
145
V06
365
35
300
526
M85
300
M05
365
MM
AM
MM
Figure Asymmetry between ELECTRA (E), RoBERTa (R), and MULTIBERT encoders (M1-M25) across layers. For each pair of the encoders M and M6), we generate training set embeddings H() HD) x for the GLUE tasks SST-2, CoLA, MNLI, QQP, RTE, and MRPC. We then fit H to HD) with an affine map and report the goodness of fit through the max error L2 norm, i.c., an approximation of H HO) on row and column j of the grid.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the claims and contributions. The claims match the theoretical and experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We highlight limitations in the main text and add more general points in
App. B
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
24

---

Answer: [Yes]
Justification: If not in the main text, all proofs along with their assumptions are in App. D and App. We additionally provide short proof sketches in the main text.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We describe all used models, procedures as well as parameters, seeds, and required compute resources either in the main text or in App. F
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
25

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: Code pulls data from GLUE, is runnable, and submitted as supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (httpss:/niis.co
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ths //nips.cc/pub cwtpublccggule desggiddss//oemmmm setssummmissiiisoo sww.mmossssssooiooooooooooooooci oonolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Again, if not listed in the main text, it is listed in App.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We provide significance indications in the form of paluue to all correlation computations and fitted regressions.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
26

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes)
Justification: Listed App.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httppp////errrsss.c////////tctat
Answer: [Yes]
Justification:
The authors do not foresee any ethical implications to this work.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Discussed the final, "Broader Impact", section in the main body.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
27

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the original paper that produced each re- re-used code snippet and dataset. We provide licenses for all models and datasets used in App. Fl
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paper swi thcode. com/ coomdatasets. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
28

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not do crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not do crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "eCkvG2XWBP", "reviewer": "Reviewer_Jeko", "summary": "This paper presents theoretical analysis on \"intrinsic alignment\" of two or more pretrained language encoders (e.g. BERT trained with various seeds). The work proposes computing the intrinsic alignment between two encoders by first defining a algebraic metric space in which these two encoders exist, and then looking at the affine homotopic transformations that are possible in the given space. The primary motivation of the work is to set theoretical bounds on the similarity of two given encoders, and for the intrinsic alignment to have a strong (positive) correlation to extrinsic alignment (in simple terms, whether two encoders produce similar outputs for similar inputs). The work claims that this is important to have an \"elementary understanding\" of these encoders, and that having these theoretical guarantees can help derive a richer set of properties of the relationship between encoders.\n\nThe work first proposes the methodology to compute these alignments, and then conducts experiments on 25 multiBERT encoders, which are BERT models trained with varying seeds. Two GLUE classification tasks are used, SST-2 and MRPC. The results show that that such an alignment can indeed be computed, and that there is a positive correlation between the intrinsic and extrinsic alignments.", "questions": "- What are some of these \"richer\" properties that intrinsic alignment can help discover?\n- Can these be run across encoders with different architectures?\n- The paper mentions that one advantage of intrinsic alignment is the possibility to define an order over the encoders. What does this imply?\n- I am wondering if these alignment measures can somehow be used to improve pre-training; lets say if we can compute the similarity between a well trained model and one that is in the training loop, and make training decisions based on how close/far the alignment is. Do you envision any such use of the proposed metric?\n- Is there a way to convert the alignment metric into a measure of \"badness\" for a given encoder? I envision it would be useful to weed out \"bad\" models than to just compare \"good\" ones.", "limitations": "Authors have addressed limitations adequately.", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The work shown in the paper approaches some of the nuances of modern LM training (randomization, seeds etc) from a theoretical perspective, which can not only help solidify our understanding but also give us concrete bounds and limitations when these models are trained\n- The results and discussion emerging from the analysis seems sound, and something that can potentially advance our understanding of these encoders further", "weaknesses": "- While the paper makes some effort to depict the practical value of the underlying method, it falls short of giving actual examples. For instance, the paper mentions that deriving intrinsic alignment can help discover \"richer\" properties - but for a reader (that does not necessarily have theoretically expertise), it is difficult to see what these richer properties look like\n- There is also very little discussion of practical aspects of running this analysis; The appendix mentions that intrinsic alignment is more expensive to compute than extrinsic alignment; given that the aim of the work is to set upper bounds, why not just compute the extrinsic scores? I was also unable to find any discussion on whether different kinds of encoders can be compared (which is easy to do for downstream tasks).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough review and the helpful feedback and ideas they provided! We are happy to hear that our theoretical results as well as practical implications are appreciated. In the following, we address the open points and concerns:\n\n- **“It is difficult to see what the richer properties are, but for a reader (that does not necessarily have theoretically expertise), it is difficult to see what these richer properties look like”**: What is meant by “richer properties” in the introduction refers to our much more general theoretical results and the empirical implications that result from formalizing and studying encoders as what they are – functions between countably infinite sets. For instance, our derivation for Remark 3.2 finds that only one-sided affine alignment (Eq. 9.2) is non-trivial, and further, defining hemi-metric spaces allows us to derive proper mathematical preorders and equivalence relations on the space of encoders, which (e.g., Thm. 4.1) has concrete empirical implications (cf. §7, “The Influence of encoder Rank Deficiency”). Further, our formalization allows us to explicitly construct the linear bounds of extrinsic dissimilarity not just for a fixed classifier (Lemma 5.1.1), but for the “worst-case dissimilarity” (cf. Thm. 5.1) -- this highly practical, as it shows our intrinsic similarity measure to be indicative of how an encoder may perform across downstream tasks in the transfer learning setting. Still, we acknowledge the density of the theoretical sections, and as per recommendation by this reviewer and reviewer 6x9V we plan to create a distilled overview paragraph of the theoretical contributions in a next revision to appeal to the broader ML community. \n---\n- **“There is also very little discussion of practical aspects of running this analysis, The appendix mentions that intrinsic alignment is more expensive to compute than extrinsic alignment; given that the aim of the work is to set upper bounds, why not just compute the extrinsic scores?”**: The practicality comes exactly *from* the fact that we show that the intrinsic similarity measures can be already indicative of task performance – not just for a fixed task (Lemma 5.1.1.), but for the worst case task across all tasks (see Thm 5.1., as well as §8; “Implications of §5”). In other words, we show that computing our intrinsic measures of similarity may already indicate the overall utility of an encoder for *any* downstream task, thus potentially eliminating the need to do task-specific evaluation of extrinsic scores.\n---\n- **“Lacking a discussion of what kinds of encoders can be compared”, “Can these be run across encoders with different architectures?”**: Generally, we do not make any assumptions about the structure of the encoder in the derivation of our theoretical results, resulting in wide generalizability of our method. In other words, our method can be applied to measure the (intrinsic) similarity between *any* two encoders and evaluate how this affects how (extrinsically) similar they can be in terms of their output probabilities.\n---\n- **“What is the benefit of defining an order over encoders”, “alignment metric into a measure of \"badness\" for a given encoder”**: Purely mathematically speaking, a proper order over a set is a powerful structure that allows us to define proper equivalence relations over encoders. Beyond equivalence, the order between encoders (cf. Lemma 4.1.) indicates that we may be able to exactly affinely map from one image into the image of a lower-rank encoder, whereas the inverse does not hold. A higher-rank encoder is therefore more powerful, and intrinsic affine homotopy is indicative of this. We find this rank deficiency to surface empirically (cf. §7, “The Influence of Encoder Rank Deficiency”). By Lemma 5.1.1., this also strongly affects extrinsic similarity, i.e., how closely we can affinely reach the output probabilities of an encoder for a specific task from another encoder – yielding a measure of the “goodness”/”badness” of an encoder, in your words. \n---\n- **“these alignment measures can somehow be used to improve pre-training”**: this is an interesting suggestion, especially in light of the nuances in fine-tuning that have shown to have large effects on downstream task performance of the pretrained encoders [1]. We can add this as possible future work or motivation, thank you for the suggestion.\n\nWe again thank the reviewer for the insightful suggestions and hope that our clarifications address their concerns about the overall contributions of our work!\n\n---\n\n[1] Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. 2020. Fine-tuning pretrained language models: Weight initializations, data orders, and earlystopping.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y0ZDiTnQYE", "reviewer": "Reviewer_6x9V", "summary": "The paper aims to formally define, derive and then analyse similarity between pretrained language encoders, focusing on aspects of intrinsic (task-independent) similarity and extrinsic (task performance-oriented) similarity. The paper is mostly of theoretical nature, aiming to properly and formally define the studied aspects of similarity and then aiming to propose the idea of transformations in an (affine) homotopic framework. The work makes a step towards more formal studies of representation similarity within language encoders (and probably decoder-style LLMs in future work).\n\nThe main non-theoretical finding, based on experiments with MultiBERT models (i.e., BERT models trained from different random seeds) is that there exists (as expected) a correlation between the defined intrinsic and extrinsic notions of similarity.", "questions": "- Beyond page limit, is there any other reason why only two tasks (and very simple ones) are targeted for the main experiments? \n\n- Have the authors considered running the similarity analyses also with BERT models having different hyper-parameters? Can we expect the affine homotopy properties to hold for such models? What implications might this finding have?\n\n- I am missing the reason why certain encoders end up being more informative than others. How do you define 'being informative' in this context? Why is it important?\n\n- Would it be possible to derive similar measures for decoder-only LMs in the future?", "limitations": "Limitations are (to most part) properly discussed in Appendix A. Some additional reflection on the current limitations of experiments (and practical aspects of the work) might be necessary and useful.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper provides a fresh perspective on the question of similarity between language encoders (and language models) in general, aiming to rigorously formalise and derive different properties associated with intrinsic and extrinsic similarity.\n\n- The paper is quite dense but admirably well-written given its largely formal and 'math-heavy' content. I see it also as a potentially very didactic piece of work which could inspire additional work in this space.\n\n- Related work, limitations, implications of the key results (both from the theoretical as well as from the more practical perspectives) are all very comprehensive and nicely structured.", "weaknesses": "- The work is heavily focused on theory and theoretical contributions; this means that its more practical findings are the weaker part of the work and the experimental setup and results are a bit underwhelming:\na) The only studied architecture is the BERT architecture, and the work just aims to quantify correlation between intrinsic and extrinsic similarity for MultiBERT models (which just use different random seeds). As far as I am concerned, a positive correlation between intrinsic and extrinsic similarity is very much expected for this group of models.\nb) The work should ideally study other encoder architectures and aim to establish how the notion of similarity changes over the spectrum of 'expected model distance' and what implications it might have. Speaking of the 'expected model distance', what one could/should study here is:\n-- models of the same architecture starting from the same seed where different checkpoints were taken\n-- models of the same architecture with different random seeds (this is the only thing studied in the paper atm)\n-- models of similar architecture but not completely the same (e.g., BERT vs RoBERTa)\n-- completely different architectures (e.g., BERT vs tELECTRA)\nc) The work also focuses on very basic GLUE tasks which also limits the generalisability of the main findings, and additional experiments over tasks of different complexity are required here to fully trust the core ideas (which would also increase the impact of the work substantially imo).\n\n- While mathematical rigor in the paper is very useful, the paper would contribute from a short (sub)section aiming to properly 'distill' the key take-home messages in a plain(er) language so that it also becomes more obvious how future work could contribute from the more theoretical insights (e.g., can the derived measures be used for computing representation similarity in general)? That would be a very useful addition to the work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for taking the time to read our work, as well as for their thorough review and helpful feedback. We are very happy to hear the positive comments about our formalization of the problem and our writing. We address the open points and concerns raised in the review below:\n\n- **\"The work should ideally study other encoder architectures and aim to establish how the notion of similarity changes over the spectrum of 'expected model distance' and what implications it might have\"** We fully agree that such study across encoder architectures and additional tasks would complement our current experiments and we plan to add this for the next revision of the paper – thank you for the suggestion!\n\n---\n\n- **\"The only studied architecture is the BERT architecture [...] which just use different random seeds\"**: Although we fully agree with the point raised in b) to get a broader coverage of results by further studying different encoder architectures and running experiments across more datasets (which was solely limited by page limits), we would like to restate the motivation and significance of the found results mentioned in §1. Namely, as found in the original paper and as seen in the experiments in §6, MultiBERTs may behave very differently among themselves. More concretely, MultiBERTs have been shown to produce embeddings that yield significantly different downstream task performance (Dodge et al. 2021 [1], Sellam et al. 2022, [2]) and produce representation matrices of different ranks to precision $\\epsilon$ (cf. §6). These are important attributes of encoders that we aim to capture in our notion of similarity, which are often disregarded (as discussed in §8). As such, MultiBERTs offer themselves well to investigate which task-independent (intrinsic) relation between the encoders best captures extrinsic dissimilarity in practice – the subject of our study in §7.\n\n---\n\n- **“A positive correlation between intrinsic and extrinsic similarity is very much expected for this group of models”**: our derivation exactly shows that specifically our notions of intrinsic similarity will be correlated with the extrinsic similarity **independent** of the considered group of models. The linear upper bound on the extrinsic dissimilarity by our notion of intrinsic similarity does not make any assumptions about the nature of the encoder, and we therefore expect such correlations to hold across encoder families. \n\n---\n\n- **“Have the authors considered running the similarity analyses also with BERT models having different hyper-parameters?”** As mentioned above, we did not make assumptions about the underlying encoder structure to derive the upper bound on the extrinsic similarity. Although we expect gaps in similarity to become more prominent, we do not expect these to affect the correlation between intrinsic and extrinsic similarity. \n\n---\n\n- **“I am missing the reason why certain encoders end up being more informative than others.”** We refer to the fact that in theory (cf. Thm. 4.1) higher-rank encoders are more powerful, as we can exactly affinely map from their image into the image of a lower-rank encoder, whereas the inverse does not hold. We also find this to surface empirically (cf. §7, “The Influence of Encoder Rank Deficiency”), and it has a significant impact on the affine mappability between the representation spaces (intrinsic similarity), and, as a result, task performance and the extrinsic similarity. \n\n---\n\n- **“Would it be possible to derive similar measures for decoder-only LMs in the future”** Although our method discusses encoder functions motivated by their usage for transfer learning, we do not in principle make assumptions about the nature of the function. In that sense, decoder representations may be evaluated equivalently. \n\nWe again thank you for the insightful suggestions and hope that our clarifications address any concerns about the overall contributions of our work.\n\n---\n\n[1] Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. 2020. Fine-tuning pretrained language models: Weight initializations, data orders, and earlystopping.\n\n[2] Thibault Sellam, Steve Yadlowsky, Jason Wei, Naomi Saphra, Alexander D’Amour, Tal Linzen, Jasmijn Bastings, Iulia Turc, Jacob Eisenstein, Dipanjan Das, et al. \"The MultiBERTs: BERT reproductions for robustness analysis.\" In ICLR, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wqGdcX8F2o", "reviewer": "Reviewer_nK2G", "summary": "In this paper, the authors study a nature question \"What does it mean for two encoders to be similar\" and proposed an intrinsic measure of similarity that aligns with extrinsic performance on downstream tasks. It introduces the concept of affine alignment and explores its properties and implications for understanding encoder relationships.\nThe main contributions are as follows:\n1. The authors first define an metric space on encoders.\n2. The authors extend the definition to account for transformations in a broad framework of $S$-homotopy for a set of transformations $S$.\n3. As a concrete application of the framework, the authors study affine homotopy—the similarity for affine transformations.", "questions": "1. Please explain clearly the motivation why we need Hemi-metrics, I think maybe from both sides of topological view and machine learning view.\n2. Please explain the connection between intrinsic homotopy and extrinsic homotopy. I think it can help us understand theorem 5.1 better.", "limitations": "See the weaknesses.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The idea of the paper is quite novelty. Homotopy is a very important tool in algebraic topology. The application of homotopy theory in machine learning is very attractive.\n2. The authors first define an (extended) metric space on encoders and then extend this definition to account for transformations in a broad  framework of $S$-homotopy for a set of transformations $S$.", "weaknesses": "1. The motivation for introducing affine Hemi-Metrics to measure the similarity of two encoders is not so clear to me.\n2. The definition of Extrinsic Homotopy seems not like a mathematical definition since authors does not claim that how to quantify the performance.\n3. More experiments are needed for supporting the claim in the paper.\n4. Such method seems not practical.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough review and helpful feedback. We are pleased to read that you appreciate the novelty of the approach and the breadth of the ideas introduced in the paper. Below, we address the specific concerns raised in the review:\n\n\n1. **“Elaborate your Motivation of using Hemi-Metrics”**: Although some previous work has explored measuring similarity in proper metric spaces, we posit that purely from a machine learning perspective, there is an asymmetry to the problem of measuring the similarity between encoders. More concretely, when we assess the similarity of two encoders in terms of how closely we can affinely map their classification probabilities on a task, we find this to be asymmetric theoretically as well as practically. Some encoders may be more powerful than others (e.g., think of a lower-rank encoder, such as the ones generated in Appendix G). A symmetric distance, therefore, is not practical as it does not capture this phenomenon. This motivates using Hemi-Metrics.\n\n---\n\n2. **“Extrinsic Homotopy does not seem like a mathematical definition”**: We agree that Definition 5.1 is somewhat informal. We will explicitly state this in the next revision – thank you for pointing this out. We note, however, that we provide a formal mathematical definition of extrinsic homotopy in Lemma 5.2 similar to intrinsic affine homotopy in Def. 4.2.\n\n---\n\n3. **“More experiments are needed to support the claim”**: We would like to highlight that most theoretical results/claims (Thm. 4.1, Lemma 5.1, and Thm. 5.1) were evaluated empirically across a large number of encoders that exhibit significant differences in properties and downstream task behavior (cf. §6, [1], [2]). Still, in light of this and other reviews, we acknowledge that an evaluation across more datasets may help support the claim, which we will include for a next revision – we thank the reviewer for the suggestion!\n\n---\n\n4. **“Such method seems not practical”**: Although we acknowledge the gap between some properties derived in §2-5 with the experimental results (see “Limitations” Appendix A), our derivation of intrinsic similarity measures that upper bound extrinsic dissimilarity (Lemma 5.1, Def. 5.2) is purely practically driven. Namely, we show that our measure of intrinsic similarity is indicative of how an encoder may perform across downstream tasks in the transfer learning setting. Further, our discussion of the algebraic properties of affine homotopy gives us a rich theoretical foundation useful to understand phenomena we observe empirically (cf. §7-8), such as:\n\n- The theoretical upper linear bounds of intrinsic similarity on task performance (Lemma 5.1, Def. 5.2) surface empirically, and therefore show us how our intrinsic measures of similarity can indicate downstream task performance similarity across arbitrary tasks. This is of significant practical interest for encoders’ utility in transfer learning, as this may reduce the need to do task-specific evaluation, as motivated in §1.\n\n- Our proof about how affine mappability is affected by the encoder rank surfaces in our experiments §7 - “The Influence of Encoder Rank deficiency” as well as in our additional experiments in Appendix G.\n\n---\nQuestions\n----\n\n\n1. See point 1. above\n\n---\n\n2. **“Explain the connection between intrinsic homotopy and extrinsic homotopy [Thm. 5.1]”**: We prove in Lemma 5.1 that for some fixed linear classifier $\\psi’$, the extrinsic dissimilarity (i.e., how closely we can map the output probabilities of representations of encoder h with the ones from encoder g) is linearly bounded by the intrinsic similarity measure (Eq. 9b). Theorem 5.1 makes a stronger statement and states that the Haussdorff-Hoare variant of the intrinsic distance upper bounds the extrinsic dissimilarity over all possible linear classifiers. This shows that our methods of measuring intrinsic homotopy linearly upper bounds, and is therefore indicative of extrinsic homotopy. The practical implications of this are significant, as this shows we can derive encoder similarity measures that are indicative of their downstream task behavior, without the need for task-specific evaluation.\n\nWe again thank you for the insightful suggestions and hope that our clarifications address your concerns about the overall contributions of our work.\n\n---\n\n[1] Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. 2020. Fine-tuning pretrained language models: Weight initializations, data orders, and earlystopping.\n\n[2] Thibault Sellam, Steve Yadlowsky, Jason Wei, Naomi Saphra, Alexander D’Amour, Tal Linzen, Jasmijn Bastings, Iulia Turc, Jacob Eisenstein, Dipanjan Das, et al. \"The MultiBERTs: BERT reproductions for robustness analysis.\" In ICLR, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lFAAsqKlvZ", "reviewer": "Reviewer_sF8t", "summary": "The paper introduces formal mathematical concepts of intrinsic and extrinsic homotopy between language encoders. The first concept compares the behavior of two encoders on a concrete dataset, while the second concept compares them independently from the concrete dataset. The paper also demonstrates how to apply these concepts to measure the difference between different versions of the pre-trained BERT model.\n\nIn more detail, chapter by chapter:\n\n- The 1st chapter introduces the problem of comparison of text encoders.\n- The 2nd chapter is devoted to the discussion of the formal definition of a language encoder. The authors define a language encoder $\\mathbf{h}$ very generally: as a function from all possible strings of some alphabet $\\Sigma$ to a $d$-dimensional vector space $V$. They mention some ways to train an encoder in practice; however, for the rest of the theoretical part of the paper, $\\mathbf{h}$ remains general enough and is not required to be implemented through a neural network.\n- The 3rd chapter discusses the formalism of hemi-metrics and introduces uniform convergence norms on the space of language encoders. The uniform convergence norm allows the authors to introduce an affine alignment measure that shows how aligned two encoders are with each other.\n- The 4th and 5th chapters introduce the notion of Intrinsic and Extrinsic Affine Homotopy on encoders. It is based on notions of affine alignment measure from the previous chapter, and encoder rank.\n- The 6th chapter is devoted to adapting the notions above to the real-life scenario, where we can't check the encoder performance on all possible strings of our alphabet. Finally, asymmetric extrinsic and intrinsic similarity measures, that are usable in practice, are derived.\n- In the 7th chapter the authors measure an extrinsic and intrinsic similarity between pre-trained BERT models, all trained with similar hyper-parameters but differing in random weight initialization and shuffling of training data (see \"The MultiBERTs: BERT Reproductions for Robustness Analysis\" paper). They show that these pre-trained BERT models can be differentiated from each other by the methods, introduced in the paper. They also provide a table of the correlations between their notions of similarity and some previously introduced notions of similarity between encoders. All experiments were performed using SST-2 and MRPC datasets.\n- Finally, in the 8th chapter they discuss relations between extrinsic and intrinsic dissimilarities, their asymmetry, and the finding that some BERTs are more similar to other BERTs by affine similarity in one direction, but not necessarily in another.", "questions": "Questions:\n\n- Why is your mathematical tool of checking, whether two encoders are similar to each other (line 153), called \"Affine homotopy\"? What is the connection to the notion of homotopy that we know from algebraic geometry?\n- You found that some BERTs are more similar to other BERTs by affine similarity in one direction, but not in another, and that this fact says something about the \"universality\" of BERTs, that are easier to map \"from\". Can you please elaborate on this conclusion about their universality?\n- By which machine learning algorithm is $\\mathbf{Ð}$ approximated and how?\n- What was the point of defining a dataset-independent way to measure the difference between encoders, if, starting from section 6, everything, in essence, became dataset-dependent again? (and of course, all experiments were dataset-depended as well)\n- What is the purpose of the Figure 2? What can it tell about MULTIBERT encoders number $1, ... , 25$?\n- In general, your paper doesn't explain what is the exact difference between, let's say, MULTIBERT encoder number $1$ and number $2$ (except that they are both BERTs with different weights init, etc). So what's the point of caring about their comparison in the first place? What useful conclusion can we make from such a comparison?\n- What is the purpose of Table 1? If I understand correctly, it shows a correlation between different similarity measures, but what conclusion should we make from it?\n- The most torturing question: what are $E_x$ and $E_y$, introduced in line 103, but never explained? I had to assume that they were just some neighborhoods of $x$ and $y$ to somehow get through the rest of the paper, but I'm not sure it's true.\n\nSuggestions:\n- I would suggest experimentally measuring the similarity of BERT with encoders of different natures, e.g. AlBERT, ELECTRA, and maybe even some LSTM-based encoders. Maybe the encoders of similar architecture (or trained on similar data) will be more similar by your similarity measures, than the encoders of very different architecture (or trained on very different data)? Or maybe not?\n- I would suggest using more datasets for any further experiments.\n- You heavily cite the \"Metrics, quasi-metrics, hemi-metrics\" chapter from the book called \"Non-Hausdorff Topology and Domain Theory\". However, this book is not well-known outside of the professional mathematical community. So, I would suggest explaining the conceptions from the book in more detail and providing some examples of hemi-metrics that would show why one wants to use hemi-metric instead of symmetric metric.\n- Finally, I would suggest turning this work into a **journal paper** (after a revision) and merging Appendices D and B (\"Additional Related Work\") with the main part of the paper to make it more readable.\n\n---\n\nTypos and presentation issues:\n\n- In 1st page, footnote №4, typo: \n> In principle, one could relax the replace R^d with any finite-dimensional vector space\n- In line 63, you wrote: \n> There are two common ways that language encoders are created. The first is through autoregressive language modeling.\n\nAnd the second one? Did you mean Masked Language Modelling?\n- In line 119: typo: \n> Let GL(V) We write\n- In line 119, you introduced GL(V) without definition. I've supposed it's a General Linear Group for the rest of the paper.", "limitations": "The authors addressed the limitations of the theoretical part of their paper. I'd suggest adding the limitations of the experimental section (one architecture, a few datasets) as well (see above).", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "- The paper introduces a novel way to compare the language encoders;\n- An interesting and non-trivial mathematical tools are utilized.", "weaknesses": "First of all, it looks like this paper is not a good fit for this venue by its very nature: \n- It is unclear what is the contribution of this work to the field of Neural Information Processing Systems. The paper introduced many novel concepts, but most of these concepts, starting from the definition of the language encoder itself, are very general, so their connection with neural networks looks far-fetched. The usefulness and purpose of the experiments are also unclear (see \"Questions\" section).\n- 9-page format is too small for this type of scientific work; authors had to overuse the Appendices to fit their work into limited space. When I was reading the paper, I had to constantly go around in circles, moving between Appendix D (\"Addenda on Affine Homotopy\") and the main part of the paper.\n\nSecond, there are some general problems with the research and text:\n- The experimental section in general is very limited. Only various versions of the BERT models are compared with each other, and only two datasets are considered. In line 235 there is a note that experiments show some task-independency, but how can we say anything about task-independency when we have only two tasks on hands?\n- The conclusion and findings of the paper are also unclear (see \"Questions\" section).\n- The paper is generally hard to read and understand. As was told above, this problem is partially connected with the unsuitable format; however, it is also connected with typos (see below) and lack of needed definitions (see \"Questions\" section).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read and review our work – we are grateful for all the suggestions. Further, we are happy to hear that our usage of novel non-trivial mathematical tools for the analysis of the space of encoders is appreciated. Below, we address the concerns raised in the review:\n\n- **“It is unclear what is the contribution of this work to the field of NeurIPS”**: Although we recognize that our usage of non-standard mathematical tools for NeurIPS, this is an attempt to more rigorously characterize the space of encoders as what they are – functions between countably infinite sets – and derive formal bounds that have concrete practical implications on downstream transfer learning behavior – and to that end, such tools are required. We further note that this paper follows a line of work, published at NeurIPS, on the similarity of neural representations (e.g., Boix-Adsera et al. (2022), Ding et al. (2021)) with extensions to the formalization of the problem as well as the objective of providing guarantees for extrinsic, downstream task behavior.\n\n---\n\n- **format, “hard to read and understand”**: We retain only the relevant interim results in the main text that contribute to the main results of the paper, leaving the proofs to interested readers in the appendix. We acknowledge the resulting density of §2-5 and would use any additional space to add examples and proof outlines to support better understandability for the broader ML community.\n\n---\n\n- **“Only various versions of the BERT models are compared with each other”, “What's the point of caring about [MultiBERTs’] comparison in the first place?”**. \nAs found in the original paper and as seen in the experiments in §6, MultiBERTs may behave very differently among themselves. More concretely, MultiBERTs show vastly different downstream task performance (Sellam et al. 2022) and produce representation matrices of different ranks to precision $\\epsilon$ (cf. §6). These are important attributes of encoders that we aim to capture in our notion of similarity, which are often disregarded (as discussed in §8). As such, MultiBERTs offer themselves well to investigate which task-independent (intrinsic) relation between the encoders best captures extrinsic dissimilarity in practice – the subject of our study in §7. Still, we acknowledge that a study across different architectures would yield broader comparisons, and we plan on including such a study as well as more extensive dataset coverage to complement our current experimental section.\n\n---\n\n- **Connection to homotopy in algebraic geometry**: In classical algebraic topology, homotopy describes the continuous deformation of one function into another, typically parameterized by the interval $[0, 1]$. However, in this paper, we consider the concept of homotopy by using a broader set of parameterizations, denoted by $S$. This allows for more flexible transformations suited to specific classification problems. For instance, $S$ could be an affine space $Aff(V), Aff(V,W)$ or others, enabling more adaptable frameworks for continuous transformations.\n\n---\n\n- **“Universality of BERT”**: From Theorem 4.1, we derive the influence of the encoder rank on affine mappability. We confirm that affine mappability is strongly influenced by the encoder rank in our experiments in Appendix G. This leads to the conclusion discussed in §7, where we find that some encoders seem to learn lower rank representation matrices (to precision $\\epsilon$), and this correlate significantly with how easily one can affinely map to that encoder, which influences their concrete utility for transfer learning by Lemma 5.1.1 and Def. 5.2. Considerations about the asymmetry are discussed separately in §8. \n\n---\n\n- **“How is Ð approximated”**: Ð is implemented as a gradient descent over affine maps (as mentioned in §6), where the loss is simply the max loss over all strings in the dataset. The gradient over the max is handled through subgradients in standard pytorch.\n\n---\n\n- **“Task-dependence of experiments”**: This is a valid concern and a limitation of the work, which we partially acknowledge in the “Limitations” section. We note, however, that the formalism of a language encoder is by definition a map between countably infinite sets (strings to vectors), and our work makes a first attempt to derive properties of such a construction, which, evidently, also have practical implications for the encoders' use in transfer learning (cf. Discussion and §7). The approximation to the finite-string setting both shows the applicability of theoretical results on real datasets and allows us to draw comparisons to existing methods in representational similarity.\n\n---\n\n- **“Purpose of Figure 2”**: As described in §3, Figure 2 plots intrinsic and extrinsic similarity, and, on a larger scale, shows an empirical approximation to the theoretical linear bound derived in Lemma 5.1. This does not tell us much about the encoders individually, but may act as a visual aid in seeing the empirical implications of the connections between our notions of intrinsic and extrinsic similarity.\n\n---\n\n- **“Purpose of Table 1”**: Correct. It shows the correlation between affine intrinsic similarity measures (cols) and extrinsic measures of similarity (rows). As in the experimental setup in Ding et al. (2021), this measures how strongly differences in extrinsic similarity are picked up by the intrinsic similarity measures. We find that Ð (as well as other linear alignment methods) tend to be strongly (and significantly) indicative of the extrinsic, downstream behavior, as theorized by the upper bound in Lemma 5.1.1.\n\n---\n\n- **“What are Ex and Ey”**: equivalent to the set E defined in Def. 3.3, Ex and Ey are non-empty subsets of X, and Y, respectively. We will add a comment, thanks!\n\nWe again thank the reviewer for their suggestions and hope that we could address most concerns and open questions about the soundness and overall contribution of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents theoretical analysis on "intrinsic alignment" of two or more pretrained language encoders. As pointed out by the reviewers the theoretical analysis in the paper is very solid and well presented. This kind of work is very valuable for the community to understand the neural encoders and improve them further. A weakness that all the reviewers mentioned is a lack of thorough experiments. For the camera ready version of the paper, I highly encourage the authors to add more experiments to study other encoders as promised in the rebuttal. This will make the paper stronger.
Decision: Accept (poster)