Stable Anisotropic Regularization
Authors: William Rudman
Carsten Eickhoff
Venue: ICLR 2024 poster (2024)
Keywords: isotropy
LLMs
outlier dimensions
Primary area: representation learning for computer vision, audio, language, and other modalities
TL;DR: We propose I-STAR: IsoScore*-based STable Anisotropic Regularization and show that decreasing isotropy in model representations improves downstream performance.

Abstract: Given the success of Large Language Models (LLMs), there has been considerable interest in studying the properties of model activations. The literature overwhelmingly agrees that LLM representations are dominated by a few ``outlier dimensions'' with exceedingly high variance and magnitude. Several studies in Natural Language Processing (NLP) have sought to mitigate the impact of such outlier dimensions and force LLMs to be isotropic (i.e., have uniform variance across all dimensions in embedding space). Isotropy is thought to be a desirable property for LLMs that improves model performance and more closely aligns textual representations with human intuition. However, many claims regarding isotropy in NLP have been based on the average cosine similarity of embeddings, which has recently been shown to be a flawed measure of isotropy. In this paper, we propose I-STAR: IsoScore$^{\star}$-based STable Anisotropic Regularization, a novel regularization method that can be used to increase or decrease levels of isotropy in embedding space during training. I-STAR uses IsoScore$^{\star}$, the first accurate measure of isotropy that is both differentiable and stable on mini-batch computations. In contrast to several previous works, we find that \textit{decreasing} isotropy in contextualized embeddings improves performance on the majority of tasks and models considered in this paper.

Published as a conference paper at ICLR 2024
STABLE ANISOTROPIC REGULARIZATION
William Rudman
Carsten Eickhoff
Department of Computer Science
School of Medicine
Brown University
University of Tübingen
wi william wallian_rudmaan rudman@brown edu
wazsten..i.khooffeuee tuebbiiigne de
ABSTRACT
Given the success of Large Language Models (LLMs), there has been considerable interest studying the properties of model activations. The literature overwhelm- ingly agrees that L.M representations are dominated by few "outlier dimensions" with exceedingly high variance and magnitude. Several studies in Natural Language Processing (NLP) have sought to mitigate the impact of such outlier dimensions and force LLMs to be isotropic (i.e., have uniform variance across all dimensions in embedding space). Isotropy is thought to be a desirable property for LLMs that improves model performance and more closely aligns textual representations with human intuition. However, many claims regarding isotropy in NLP have been based on the average cosine similarity of embeddings, which has recently been shown to be a flawed measure of isotropy. In this paper, we propose I-STAR: IsoScore" -based STable Anisotropic Regularization, novel regularization method that can increase or decrease levels of isotropy in embedding space during training. I-STAR uses IsoScore', the first accurate measure of isotropy that is both differ- entiable and stable on mini-batch computations. In contrast to several previous works, find that decreasing isotropy in contextualized embeddings improves performance on most tasks and models considered in this paper. !
INTRODUCTION
Several previous works have investigated the role of isotropy in Large Language Model (LLM) representations (Rudman et al. 2022). A distribution is isotropic the variance of the data uniform and the data dimensions are uncorrelated. In practice, a distribution is isotropic when its covariance matrix is proportional to the identity matrix. Studies have found that representations from LLMs, such BERT or GPT-2, lack the property of isotropy and that contextualized word embeddings dominated by a few "rogue" " or "outlier" dimensions (Timkey and van Schijndel| Kovaleva et al. 2021 Several previous works have argued that anisotropy, i.e., the lack of isotropy, is detrimental to LLM embeddings as it I) forces representations to occupy a "narrow cone" in space (Ethayarajh 2019 Cai al. 20221: ) 2) obscures linguistic information, thereby limiting the expressive power of the embeddings (Gao et al. 2019 Zhang et al., 2020 Mickus et al. 2019), and; 3) hinders performance on a variety of downstream tasks (Kovaleva al. 202 Bis al. 2021 Timkey and van Schijndel 2021). However, some recent works have challenged previously held conceptions about isotropy, arguing that current methods of measuring isotropy are fundamentally flawed (Rudman al.) 2022, Rajaee and Pilehvar 2021a). To address these concerns, [Rudman et al. (2022) propose IsoScore, an accurate and robust method for measuring isotropy based on the covariance matrix of a distribution. Although IsoScore is effective method for measuring isotropy, we demonstrate that IsoScore is neither differentiable nor stable when the number of points in a given sample is small. Therefore, IsoScore cannot serve effective model regularizer.
Given the recent criticism of methods for measuring isotropy, a reassessment of previously accepted theories of isotropy in LLMs is needed. This paper aims to determine the relationship between isotropy and model performance on various language models and fine-tuning tasks. We first propose IsoScore'. method for measuring isotropy that is 1) fully differentiable, 2) incorporates classical techniques for covariance estimation to create stable isotropy estimates for mini-batch data, and 3) approximates IsoScore for large sample sizes. We then use I to develop I-STAR; IsoScore* -based STable
Cooe: http:///gihhbb.oomobbiicccchhhhh wiihehhhwwwpeeehhhoheeccee

---

Published as a conference paper at ICLR 2024
X xt
[=1
11 22
IsoSoure" (.x))
LISTAR : Log M1 X1-boSooree &R. Es,1
Classilier
Embed
Input
I
T2
Tn
LOE
Hidden States
Figure I: Forward pass of our I-STAR loss function. Let be the token embeddings in a mini- batch let X = Ui_ let Lis, be the shrinkage covariance matrix for epoch and
at layer& / E {1, 2..,, n). let ( E (0, 1) be the shrinkage parameter. I-STAR loss is weighted sum between cross-entropy loss, LCE, and IsoScore" (X, ss,) where À the decreasing isotropy in representations, and positive values of À encourage isotropy.
Negative values of correspond to
Anisotropic Regularization. I-STAR is flexible way to adjust isotropy in model representations during training or fine-tuning, In contrast to works that use "flawed" measures of isotropy, we find that using I-STAR to decrease isotropy in embedding space, i.e., making representations more anisotropic, tends to improve downstream performance across three different LLMs and nine different fine-tuning tasks. Our finding that anisotropic representations perform better downstream tasks is aligned with literature outside of NLP that argues anisotropy is a natural by-product of stochastic gradient descent, where anisotropy helps models escape local minima in the loss landscape and, thus, generalize better to unseen data (Zhu et al. 2018). Additionally, our findings are supported by well-established body of literature arguing that lower intrinsic dimensionality of network representations in later model layers corresponds to better performance on downstream tasks (Ansuini et al. 2019 Recanatesi et al.
2019 Chung et al. 2018). This paper makes the following novel contributions.
1. We propose IsoScore", a robust method for measuring isotropy that is stable even when the number of samples point cloud is small.
2. We present a novel regularization method, I-STAR: IsoScore* -based, STable Anisotropic Regularization. I-STAR effectively shapes the geometry of network activations stable manner that overcomes the current limitations of other methods that backpropagate through the calculation of principal components during stochastic gradient descent.
3. In contrast to existing theories of NLP, we demonstrate that decreasing isotropy in LLMs tends to improve performance on various fine-tuning tasks and models.
RELATED WORK
Improving Isotropy. Methods to restore isotropy in contextualized embedding models fall into two categories: post-processin methods and regularizers. Mu et al. (2017) propose All-But-The-Top post pestpprocessig algorithm that masks out several of the top principal components of the data. The authors show that their simple algorithm improves performance for Word2 Vec and GloVe embeddings algorithm where the top principal components of the last hidden state LLMs are masked or removed (Rajaee and Pilehvar 202 Bihani and Rayz 2021 Liang et al. 2021 Liao et al.) 2020 Sajjad al. 2022). Across each of these studies, the authors argue that improving isotropy in the embedding space improves model performance. However, studies evaluating the impact of improving isotropy in embedding space by masking principal components tend to be limited to word similarity tasks, which do not provide complete picture of the importance of isotropy in model representations.
on word similarity tasks. Several slight variations have occurred on the All-But- -eeu--h----ee
2

---

Published as a conference paper at ICLR 2024
Zhou et al. (020) propose Isotropic Batch Normalization, modified whitening transform that forces representations to be zero- eero-mean but allows the covariance matrix of model representations to be block diagonal and does not entirely remove all correlations from the data. The authors apply their novel transformation to the final hidden state representations of a BERT model before heing input to the classification head and show that Isotopic Batch Normalization minimally improves the performance of BERT on several datasets in the GLUE benchmark. Several authors argue that isotropy can be restored in contextualized embedding space by applying simple zero-mean transformation the data (Bis et al. 2021 Cai et al. (02T). Given that isotropy is a property of the covariance matrix of the data and is unrelated to the mean, the improvements on the textual similarity tasks shown in various studies are likely unrelated to the property of isotropy. There have been far fewer attempts in the literature to improve isotropy using regularization penalties. [ao et al. (2019) propose CosReg, regularization technique that penalizes the model when the average cosine similarity of model representation approaches 1. The motivation behind CosReg is that by reducing the average cosine similarity between embeddings. models will be penalized when representations occupy "narrow cone" in vector space (Gao et al. 2019; Zhang et al. 2020). Although the authors report modest gains when using CosReg, more current studies have argued that average random cosine similarity does not accurately measure isotropy (Rudman et al. 20222..
Although large number of papers in NLP argue that isotropy is beneficial for representations, the broader machine learning community has found that 1) anisotropy is a natural consequence of stochastic gradient descent; 2) anisotropy allows for networks to generalize better to unseen examples, and; 3) networks that compress data into lower dimensional manifolds show better performance on downstream tasks (Zhu al. 2018 Ansuini et al. 2019 Recanatesi et al. 2019). We argue that the primary reason for the differences between claims on isotropy in NLP literature and machine learning literature stems from the noisy range of often flawed methods of measuring isotropy on which many claims are based (Rudman et al., 2022). In Section 2.1 we discuss the most common methods used to measure isotropy in embedding space and detail why most attempts to measure isotropy in the NLP literature do not accurately reflect properties of isotropy.
2.1 MEASURING ISOTROPY
Average random cosine similarity is the most common method for measuring "isotropy" in embed- ding space. An average random cosine similarity approaching I is thought to represent a minimally isotropic space, while average random cosine similarity of O constitutes maximally isotropic space (Ethayarajh] 2019). Ethayarajh (001)) finds that the average random cosine similarity between activations of BERT and GPT-2 approach 1, which the authors use to argue that model representations form a "nartow cone" in vector space. However, Rudman et al. (2022) show that average random cosine similarity not a measure of isotropy since the average cosine similarity of points artificially approaches 1 when the mean of the data is far from the origin and will be 0 when the data are zero-mean, regardless of the shape of the distribution.
The partition isotropy score is based on a partition function, Z(C) :- = exp(e"x>) x). developed by Arora et al. (2155), where c Rdd represents the set of unit vectors and X C Rd is finite point cloud. Since calculating the entire partition function is intractable, studies typically approximate the full partition function as / ((X) min,.er2 ((c)///cmmm/m/c// /m where is chosen from the eigenspec- trum of XX" (Mu et al. 2017). Mu et al. (211)) prove that there exists a choice of C' such that their method for measuring isotropy reflects the uniformity of principal components of the data. However, approximating from the eigenspectrum of XXXT results in undesirable properties, such as being heavily influenced by the mean vector of X and influenced by orthogonal transformations of the data that are unrelated to isotropy (Rudman et al. 2022).
Intuitively, IsoScore measures how "far away" the covariance matrix of the data is from a Ia, where is a positive scalar and Ia is the d x d identity matrix. Algorithm details details the steps required to calculate IsoScore Rudman et al. (2022) develop rigorous testing suite to demonstrate that IsoScore is the first tool in the literature that accurately measures isotropy in embedding space. To ensure that IsoScore is not biased towards a given basis of the data, the authors "reorient" the data by projecting point cloud of data by its principal components. An IsoScore value of indicates that distribution is fully isotropic, and a score near 0 suggests that a single dimension in vector space dominates representations. A Although IsoScore is an accurate measure of isotropy, we demonstrate in Section
3

---

Published as a conference paper at ICLR 2024
Algorithm 1 IsoScore" Forward Pass
1: Input: X C IRd point cloud, Es E Rddd shrinkage covariance matrix, Ç E (0, 1).
2: Outputs: I-STAR penalty of X.
3: calculate covariance matrix: Σx of X
4: calculate shrinkage matrix: Σg := (1 - -)-Ex + -
5: calculate eigenvalues: A = (AI..., Ad) of Ex
6: normalize eigenvalues: À == vd. A/1 such that ||Å|| Vd
7: calculate the isotropy defect:
5(A) = π 111/ 2 2(d- va)
where 1 (l..... .... 1)T E R2d
8: calculate: ((A)= (d - (a))2/
9: calculate: ((A) = (d ((A) ()-
that IsoScore will systematically underestimate the true isotropy score of a distribution when the number of samples is small relative to the dimensionality of the vector space.
Since many current works in NLP use "flawed" measures of isotropy, such as average random cosine similarity and the partition isotropy score, the connection between isotropy in LLMs and their performance on downstream tasks has not been established. This study devises a novel regularization penalty, I-STAR, to investigate the relationship between model performance and isotropy in model representations. In contrast to several previous works based on average cosine similarity or the parti- tion score, we use IsoScore* to demonstrate that decreasing isotropy tends to improve performance on downstream tasks, while increasing isotropy hampers performance nearly all tasks and models considered this paper.
3 ISOSCORE"
Previous methods for measuring isotropy in embedding space are either 1) not accurate measures of isotropy, 2) not differentiable, or 3) not stable on mini- batch computations. In this section, we propose IsoScore", novel, fully differentiable measure of isotropy that stable, even for small sample sizes. First, we thoroughly describe IsoScore*. We then demonstrate that IsoScore" can accurately measure the isotropy of small data samples, while vanilla IsoScore systematically underestimates the isotropy of distribution when the number of points in a finite point cloud is smaller than the dimensionality of the vector space. For the remainder of this section, let X C Rd and C Rdd be finite point clouds SS].
drawn from a distribution X such that X| < d <
Intuitively, IsoScore* measures the extent to which the principal components of a distribution are uniformly distributed. Mcasuring isotropy
function of the principal components allows to backpropagate through IsoScore* since PCA is a differentiable function (Huang et al./ (2018). An IsoScore* value of 1 implies that for principal components {^1; --; Xa), A; = Vi j. An IsoScore* value of 0 implies that only a single principal component is non-zero. Although both IsoScore* and the partition score measure isotropy via the principal components of a distribution, IsoScore* directly measures the uniformity of the eigenspectrum of principal components and does not have to rely on approximations the eigenspectrum like the partition function defined by Mu al. (2017).
IsoScore* Pseudocode. Step 3) of Algorithm T begins by calculating the covariance matrix of X ddd that we assume sampled from the distribution X. Next, in Step 4) we calculate the RDA- Shrinkage matrix, Σg. as weighted sum between the covariance matrix of X and the covariance matrix of S, Es, to produce more accurate estimate of the covariance matrix of the true distribution X. The shrinkage parameter, Ç. controls how much covariance information is used from X and S when estimating the covariance matrix of X. In Step 5), we calculate the eigenvalues of Σg Step 6) normalizes the eigenvalues so that the L2 norm of the eigenvalues equals the norm of the vector containing all Is (i.e.. 1, 1, .... 1). The remaining normalizing Steps 7-9 are derived in the same manner vanilla IsoScore. For detailed pseudocode analysis of both IsoScore and IsoScore", as well as a proof that IsoScore approximates IsoScore* when the number of samples in our point cloud large, Section A in the appendix.
4

---

Published as a conference paper at ICLR 2024
Mini-batch stability of isotropy estimates. The mini-batch stability of methods measuring isotropy has yet to be investigated. We test the stability of mini- isotropy by sub-sampling small batches of points, X, from point cloud of data, X C R7 consisting of 250,000 points sampled from 768-dimensional Gaussian distribution with a zero mean-vector and a covariance matrix, Ex. such that X has d
{10, 6, 4, 4, 1, 1,... }} and zero off-diagonal elements.
In Section oo the Appendix, we prove that when Ç = 0 (i.e. no shrinkage is performed), IsoScore (X) IsoScore (X, C, Es). digoont2demossttattte that for sub-sample X of X, if [X] is not sufficiently larger than d, IsoScore systematically underestimates the true degree of isotropy (dashed horizontal line) of the distribution, X. from which X sampled. This means IsoScore << IsoScore (X) For isotropy results to be reliable, future work should ensure that the number of points in given sample is significantly larger than the dimensionality of the distribution. IsoScore underestimates the true degree of isotropy of the distribution because IsoScore relies on calculating the covariance matrix of sample. When the number of points in a sample, |X], less than the dimensionality of the space, the covariance matrix of X may be singular (Friedman 1989). Existing methods to improve isotropy and some of the most common metrics to evaluate isotropy, such as the partition isotropy score (Mu et al. 2017), rely investigating the principal components of the data. As consequence. the problem of underestimating the isotropy of sample distributions will affect nearly all previous works. Fortunately, many well-established methods in the statistics and machine learning literature exist to ensure that a covariance matrix will be better conditioned and invertible, leading to more reliable methods to alter and measure isotropy in embedding space (Friedman 1989)
Stabilizing covariance estimation. Shrink- age is simple operation that adds a known, stable covariance matrix to non-invertible. singular sample covariance matrix (Fried man 1989). Performing shrinkage ensures that the resulting covariance matrix is in- vertible. In situations where one does not have access to multiple samples or a sam- ple where S C X such that d << [S), the matrix Ç - Ia + Ex is used as the shrink- age matrix (Friedman 1989). However, if one has access to the larger point poige cloud S or multiple samples from X, then a more faithful estimate of the covariance matrix of X can be obtained using regularized dis- criminant analysis (RDA) (Friedman) 1989).
Impact of Sample Size on IsoScore*
10
0.8
0.2
IsoScore*
0.6
0.6
0.8
0.4
0.2
0.0
200
1000
2000
3000
4000
50000
Sample Size
RDA shrinkage pools covariance matrices together using Σς := Ç- Ex + +1 {) Es to get a more accurate measure of the covari- ance matrix of the distribution from which demonstrates that
Figure 2:
IsoScore (X, values for different
choices of Ç. The dashed line indicates the correct IsoScore* value of X, which is IsoScore' (X) = 0.86. We calculate Eg from a subsample S X such that X n = @ and |S| = 75, 000.
X is sampled. Figure 2 performing this shrinkage operation on Σx drastically improves the stability of IsoScore" even when |X] = 700 and d = 768. Step 4 of Algorithm Juses RDA shrinkage stabilize IsoScore* mini-batches during training. In stochastic gradient descent, mini-batches of data are sampled randomly from the larger training set. We per- form shrinkage on the mini-batch covariance matrix with a covariance matrix calculated from token embeddings obtained from small fraction of the training data to obtain the most accurate estimate of mini-batch isotropy. We initialize the shrinkage matrix, Eg, by computing the covariance matrix of a sample, S, of 250k points obtained from running partial forward pass on the training data before training with I-STAR. We update Es after each epoch during training by running partial forward pass on the training data. We use IsoScore" as the penalty in our loss function to produce stable updates during training that alter the isotropy of model representations. Figure illustraaess how we incorporate IsoScore* to form our I-STAR loss. We take the union of token embeddings from each hidden state in the model and calculate global IsoScore* penalty to incorporate into our loss function. We stress that shrinkage is crucial for the success of I-STAR. In Section D we show that without shrinkage, the performance of models tuned with I-STAR can drop by as much as R 6%.
5

---

Published as a conference paper at ICLR 2024
4 METHODS
CosReg, To compare CosReg to I-STAR, we adapt the cosine similarity regularization term presented by (Gao et al. 2019) and calculate our CosReg penalty on the last-layer hidden states of the model. Let {#1, 22 FM denote the mini-batch representation obtained from the last hidden layer, X,, of contextualized embedding model, let é; || CosReg loss function of our model is defined as:
and let À E R be tuning parameter, then the
M
LeosReg = LCE + A
of
a
(I)
M
We use À = 1 as Gao et al. (2019) find that using cosine similarity and that using À > does not provide any additional training benefits. Since an average cosine similarity of I is thought to reflect an anisotropic space and an average cosine similarity near 0 reflects isotropic space, using À 1 is believed to encourage "isotropy" as measured by cosine similarity (Ethayarajh] 2019). However, we show Figure [ that CosReg impacts the mean of embeddings but has impact isotropy. Namely, CosReg does not properly regularize isotropy.
is sufficient for altering average random
I-STAR. Figure I outlines the calculation of I-STAR loss. I-STAR computes global IsoScore' penalty from token embeddings obtained from all layers in the network. Calculating a global isotropy penalty from the representations at every layer of the network allows the model to determine where changing the isotropy of representations in the network will lead to the largest improvements in performance. In Section Ginn the Appendix, we examine the impact of applying I-STAR individual layers and find that calculating a global isotropy penalty leads to the most consistent performance. Let X = U X₁ denote the union of all hidden states from a network with n layers and Es. be the shrinkage covariance matrix for epoch of training. We define our I-STAR loss follows:
LISTAAR LCE + A IsoScore" Sx
(2)
negative value of À will decrease the levels of isotropy in model representations, while positive choices of À will increase levels of isotropy.
Intrinsic dimensionality estimation. Previous studies have shown that compressing the intrinsic dimension of LLM activations in later layers is correlated with improved performance downstream tasks (Recanatesi et al.) 2019). To determine the relationship between isotropy and intrinsic dimen- sionality, we use TwoNN N to estimate the intrinsic dimension of activations similarly to (Recanatesi) et al. 2019). TwoNN is a widely used intrinsic dimensionality (ID) estimator based on the ratio between each point's first and second nearest neighbors (Facco et al.) 2017). Note that calculating TwoNN is an iterative process based on the pairwise distances of nearest neighbors in space and is non-differentiable operation.
4.1 EXPERIMENTAL DESIGN
Models & datasets. In this paper, we fine-tune BAppendixepbert, ALBERT (Lan et al. 2020), and
DistilBERT (Sanh et al. 2020) for nine NLP common benchmark tasks: SST-2 (Socher et al. 2013), QNLI (Rajuurkar& al. 2016a), RTE (Dagan et al. 2005), MRPC (Dolan and Brockeu 2005). QQP WWang et al. 2018). COLA (Warstadt et al. 2019). ,, STS-B (Cer et al., 2017). SST-5 Socher et al. (2013) and SQUAD Rajpurkar al. (2016b). A detailed description of each dataset available in Section
Hyperparameters & training details. For each model and each task, we hyperparameter tune for batch size (8, 16, 32), training epochs (3,4.5), and learning rate (1e-5, 3c-5, 5c-5). For I- STAR, we tune for the optimal zeta (0.2, 0.4, 0.6, 0.8) and use the tuning parameters values, E (-5, -3, -1, 1, 5J. For CosReg, we use tuning parameter of I in accordance with Gao et al. 2019). All reported performance metrics are calculated as an average over five random seeds to demonstrate the robustness of our results. After perform hyperparameter tuning, fine- tune our models using two 3090-RTX GPUs, use mixed-poo point precision training for all models/tasks, and set gradient accumulation step to 2.

---

Published as a conference paper at ICLR 2024
5 RESULTS
In this section, we demonstrate that 1) there is an inverse relationship between isotropy and per- formance; 2) CosReg implements zero mean transform and does not improve isotropy, and 3) increasing/deerrasingg isotropy using I-STAR increases/decreases the TwoNN intrinsic dimensionality estimation of model representations.
Table 1: Performance of CosReg and I-Star for each model and task. "Base" indicates that no regularization methods were used. For COLA, we report Matthew's 's Correlation; for STS-B, we report Pearson's 's Correlation; for SQUAD, we present FI/EM. For all remaining tasks, we report accuracy. We report the average/standard deviation over 5 random seeds. Each I-STAR value comes from training with a negative tuning parameter. See Table
2 for more details.
Mathel
SST-2
ONLJ
RTE
MRR
QQP
CXILA
STS-B
SST35
SQUAD
ALHERTESTAR
33
=144::043
72.56=1.29
87
55.22=1611
5X 5191
55.02:00.46
ALBERT Base
23.0620.24
90
20.28:0.42
73,3
81
30.96
19.98:0006
54.5620.55
$9.77-0..00
54, $4.2320.30
90.3250 13/82 Mean.. 1.
ALBERT CosReg
01886+042
90 90.1+025
65.09
81
96.1+0..6
89
49 49.59+0..00 90
87 67.98-0.45
54.76+0 11
11/82 64-0. 32 90.88
BERT FSTAR
124510.15
89311006
623331070
56.76.0..57
90336005
59.6010.25
0.144
50.4400.3
87.761611100
0.10
BERT
02.40+0.41
39.4 30
62.24:00.88
86.51=01.21
90-44-031
59.22+00.3
8.15-00.0
50.0040.20 28
N7 11/78 #2=0.25
BERT CosReg
92.0110..2
62:11:1441
85:29:00.7
9..45-0.22
0.30
85
DisnIDERT USTAR
91.210011
550550095
DEsiRRERT Rase
01.36+0.15
87.41-0.34
5682-0067
63.69-0.41
50.16+0.99
64.5500.2
49/48-00035
23-0 I874
DistBBERT CodReg
90.9420.38
86.880019
57.04=1.13
53.58=0.44
39.55=0008
49. 50.30:+0.88
54.41=0.08
48.65+0:25
83.47= 0.3474.40000224
There is an inverse relationship between isotropy and performance. In contrast to previous studies, Figure g demonstrates that increasing isotropy tends to decrease performance, whereas decreasing isotropy tends to increase performance. We observe that the trend's 's strength is somewhat task and model-dcpendent, with the most robust correlation emerging for SQUAD, RTE, and STS-B B. However, for both MRPC and COLA, ALBERT does not exhibit a strong correlation between isotropy and performance. However, for both bases, the optimal tuning parameter negative. The lack of a distinct trend for ALBERT on MRPC and COLA is likely due to the large amounts of noise in fine-tuning performance and isotropy values occurring on these tasks. Further, Table | demonstrates that decreasing isotropy using negative tuning parameters in 1-STAR improves performance over both baseline models and CosReg on most tasks and models considered in this paper.
CosReg implements zero-mean transform. Figure demonstrates that CosReg alters the mean of model activations, particularly in a single dimension. Encouraging the average random cosine similarity of activations be (i.e., when À = 1) forces representation to be zero-mean, whereas encouraging an average random cosine similarity to be (i.e., when À = -1) causes the mean of representations to increase further. Although CosReg impacts the mean of the data, CosReg does increase isotropy in model representations. After fine-tuning BERT, ALBERT, and BiTT on SST-2 using CosReg with tuning-paramest value of À 1, the last layer representations of each model receive IsoScore* values of 0.004, 0.007, and 0.007, respectively.
Isotropy and intrinsic dimensionality estimation. Several studies have found that lower intrinsic dimension of later layer representations is correlated with improved performance on various down- stream tasks (Ansuini et al., 2019 Recanatesi et al. 2019). Figure demonstrates that adjusting isotropy with I-STAR corresponds to changing the intrinsic dimension of model representations. Importantly, encouraging isotropy in model representations does not allow for model representations to compress into lower dimensional manifold in later layers.
6 DISCUSSION
Our study challenges a dominant belief in the NLP literature that encouraging isotropy improves performance on downstream tasks. In contrast to several previous works, we find that encouraging isotropy detrimental model performance and that decreasing isotropy in representations improves performance broad range of tasks and models. Table [ and Figure provide strong empirical evidence, in support of (Zhu et al. 2018), that anisotropy is essential for a model's downstream performance.
The primary reason for the discrepancy between our results and existing studies in the NLP literature on isotropy is that previous studies have made claims using "flawed" measures of isotropy, such as average random cosine similarity. Figure4show showws that using CosReg implements a zero-mean trans- form and does not improve isotropy. Given our findings that isotropy and classification performance
7

---

Published as a conference paper at ICLR 2024
SST2
QNLI
RIE
0.900
0.00
0.955
Accureo
Accurec
0.85
Accuracy
0.000
0.90
0.5
0.825
0.4
0.0
0.2
0.6
03
InsSoooee
LooSaaee QQP
IsoSoore COLA
MIRP
0.00
Marthow's Comelation
0.90
Aucuracy
Accurscs
0.95
0.5
8.5-
0.4
0.50
01:3
03
01.11
ST1B
SST-5
SQUAD
Accuracy
0.50
0.575
0.46
0,500
00
0.1
02
0.3
6.6
0.6
IusSomee
LooScare
BoNcore"
Medel
HERT
ALBERT
DIGIBERT
Figure Relationship between IsoScore* (x-axis) and model performance (y-axis). We fine-tune each model with I-STAR using the tuning parameters À E {-5, -3, -1, 0.50, 1,3,5). We train each model over five random seeds and report the standard deviation of both performance and IsoScore" (X, S. Es) values. We set Ç = 0.2 for all computations of IsoScore", and we compute Es from random sample of 250,000 token embeddings from the training data.
ALBERT
BERT
DisUBERT
Average Value
-2
100
600
600
200
400
000
800
0
200
100
600
800
Embrriding Dimension
Embodding Dimension
Emhheding Dirnension
CosReg
Base
CoeReg
Figure 4: Comparing the activation values the validation data for each dimension of ALBERT, BERT, and DistilBERT fine-tuned on QNLI, with CosReg using a of A 1, 1 and without any regularization. Trends are representative of all tasks.
value
negatively correlated and that CosReg does not adjust isotropy, argue many of the current claims regarding isotropy in NLP need to be reassessed.
Although majority of prior studies in NLP have argued that isotropy is beneficial to LLMs, recent works have helped to explain our finding that isotropy negatively correlates with classification performance. (Mickus et al. 2024) provide robust mathematical framework demonstrating that
8

---

Published as a conference paper at ICLR 2024
ALBERT SST-2
BERT SST-2
DistilBERT SST-2
80
60
90
Intrinsic Dinension
60
30-
10
40
30
20
Layer
Layor
Layer
Taming Parameler
Base
Figure 5: TwoNN Intrinsic Dimensionality estimate of ALBERT, BERT, and DistilBERT sentence embeddings, i.e. [CLS] tokens, obtained from SST-2 validation data for models fine-tuned on SST-2 using I-STAR with muning-parameters À E (-5. -3, 3,J. "Base" represents the case where no regularization is used. Trends are representative of all tasks.
isotropy and clustering are incompatible objectives and that clustering behavior is crucial for an effective classifier. Namely, encouraging isotropy inhibits clustering behavior, which is harmful to downstream performance.
Our findings strongly support arguments in the literature outside of NLP that anisotropy is a natural outcome of stochastic gradient descent and that compressing representations necessary for model performance. Additionally, studies have shown that model representations that occupy lower intrin- sic dimension in the ambient vector space tend outperform those sampled from higher dimensional manifolds (Recanatesi al. 2019] Ansuini al.) 2019). We demonstrate that encouraging isotropy in the embedding space increases the intrinsic dimension of model representations, which is detrimental to performance. Importantly, also show that reducing isotropy in the embedding space leads to the compression of representations into lower dimensional manifold, resulting in improved model performance. This underscores the critical role of isotropy in determining the intrinsic dimension of model representations and the subsequent impact on model performance.
Limitations. Although having isotropic representations theoretically desirable for both the in- terpretability of model decisions and for improved quantization abilities, encouraging isotropy in pre-trained models way that preserves improves downstream task performance challenging. This study is limited to fine-tuning LLMs, which may not provide a definitive answer to whether encouraging isotropy in embedding space inherently detrimental model performance. Our results demonstrate that encouraging isotropy in pre-trained models causes a decline in downstream fine-tuning performance. Fine-tuning requires models to make rapid and drastic adjustments their representations within limited number of training steps. A fruitful direction for future work would consist using I-STAR in LLM pre-training to enforce isotropic representations throughout training.
7 CONCLUSION
Previous works in NLP have argued that anisotropy in contextualized embedding models limits the expressiveness of word representations and forces embeddings to occupy a "narrow cone" in vector space. Several studies have claimed that improving isotropy leads improved performance on downstream tasks. However, most studies use faulty isotropy measures, tend to be limited to word similarity tasks, and only investigate isotropy for last-layer representations. We propose I-STAR, differentiable, mini-batch-stable isotropy- regularization scheme, to study the relationship between fine-tuned model performance and isotropy. Contrary to previous works in NLP, we find that further decreasing isotropy improves downstream model performance. Fundamentally, show that enhancing isotropy in embedding space increases the intrinsic dimensionality of model representations and causes model performance to decrease. Given the connection between isotropy, intrinsic dimensionality, and performance, I-STAR shows great promise for application various areas of deep learning.

---

Published as a conference paper at ICLR 2024
8 REPRODUCIBILITY
We have taken several steps to make our paper as reproducible as possible. Firstly, we have made all code used to produce the project publicly available and attached an anonymous version along with our submission. Further, we have released pip install of IsoScore" to facilitate future works. Section Section[G Gutllines all values of the two critical hyperparameters needed train with I-STAR loss. Lastly, all datasets and models used in this paper are publicly available on Huggingface.
REFERENCES
Alessio Ansuini, Alessandro Laio, Jakob H. Macke, and Davide Zoccolan. Intrinsic Dimension of Data Representations in Deep Neural Networks. Curran Associates Inc., Red Hook, NY, USA, 2019.
Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Random walks on context spaces: Towards an explanation of the mysteries of semantic word embeddings. CoRR, abs/1502.03520,, 2015. URL httt:: org/
03520
Geetanjali Bihani and Julia Rayz. Low anisotropy sense retrofitting (LASeR): Towards isotropic and sense enriched representations. In Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 81-95, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/2022.deell. 1.9.
URL
2021 deelio-1.9
Daniel Biś, Maksim Podkorytov, and Xiuwen Liu. Too much common: Shifting embeddings in transformer language models and its implications. In Proceedings of the 2021 Conference of
the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5117-5130, Online, June 2021. Association for Computational Linguis- tics. doi: 10. 10.18653/v1/200 haacl-ma in. 403
satacclmmii.440 URL https: http:////aaaanttoolcc.ooo///////
Xingyu Cai, Jiaji Huang, Yuchen Bian, and Kenneth Church. Isotropy in the contextual embedding space: Clusters and manifolds. In International Conference on Learning Representations, 2021.
URL httpt:ttp://tttett htttps:://openeeveet net forum? XYNOO
Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. SemEval-2017 task 1:
Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2001)) pages 1-14, Vancouver, Canada, August 2017. Association for Computational Linguistics. doi: 10. 11118/6////11-20000
URL https::////.. /apps://ace http://aaaaatthhc wwwhhhoooyyor....o........o oww..oorg/1112000
Sue Yeon Chung, Daniel D. Lee, and Haim Sompolinsky. Classification and geometry of general perceptual manifolds. Physical Review X, 8(3). jul 2018. doi: 101.1///pppyyee...0.....0. URL
htttps://doi http://tto.og//...11 www.oogg......... 1tt.:1..111042ppyaeemee 03110 031003
Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Proceedings of the First International Conference on Machine Learning Challenges: Evaluating Predictive Uncertainty Visual Object Classification, and Recognizing Textual Entail- ment, MLCW page 177-190, Berlin, Heidelberg, 2005. Springer- Verlag,--Verlag, ISBN 3540334270. doi: 00.100//11336990 9. URL htt.l:tts:///// org. 10.../////. 1007/11736690
Chris Ding, Ding Zhou, Xiaofeng He, and Hongyuan Zha. R1-pea: Rotational invariant 11 Io-m principal component analysis for robust subspace factorization. In Proceedings of the 23rd International Conference on Machine Learning. ICML '06, page 281-288, New York, NY, USA, 2006. Association for Computing Machinery. ISBN 1595933832. doi: 10. 10.1114/111 1431144444443888888 URL Cittps://aclanthee htw.aaantoooooooooooooooooo 1gy.
1ttp::......11111111111111111111 1143888
William B. Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases.
In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005. URL
10

---

Published as a conference paper at ICLR 2024
Kawin Ethayarajh. How contextual contextualized word representations? comparing the geometry of bert, elmo, and GPT-2 embeddings. CoRR, hts//199.00512, 2019. URL http:://axxii
org/abs/ / 1909. 00512
Elena Facco, Maria d' d'Errico, Alex Rodriguez, and Alessandro Laio. Estimating the intrin- sic dimension of datasets by a minimal neighborhood information. Scientific Reports, 7
(1), sep 2017. doi: 1111038/41158-007-11111177 2Fs41598-017-11733
URL httpss:/doi. wtt.d/..ogg/...110011
Jerome H. Friedman. Regularized discriminant analysis. Journal of the American Statistical Association, 84:165-175, 1989.
Jun Gao, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tie- Yan Liu. Representation degeneration problem in training natural language generation models. CoRR, abs/1907.12000 2019. URL
hitt:://axiv: org/ abs/ /tbs/1990..0 12009
Lei Huang, Dawei Yang. Bo Lang, and Jia Deng. Decorrelated batch normalization. CoRR, abs/1804.08450, 2018. http:
org/ abs 1804. (805000400)
Olga Kovaleva, Saurabh Kulshreshtha, Anna Rogers, and Anna Rumshisky, BERT busters: Outlier dimensions that disrupt transformers. In Findings of the Association for Computational Linguistics:
ACL-IJCLLP 202/, pages 3392-3405, Online, August 2021. Association for Computational Linguistics. doi: 10. 10.18653/v1/2021..m findings--cl.300 URL htthttss:: /ttpas://alaatata /aaaa/aaaannntooooooaoeaaeeeooll org/
2021. fi2lifindiigs acl. a00
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self-supervised learning of language representations, 2020.
Yuxin Liang, Rui Cao, Jie Zheng, Jie Ren, and Ling Gao. Learning to remove: Towards isotoopic pre-trained BERT embedding. CoRR, abs/2104.05274, 2021. URL https::/arxiv..org abs/ / 2104. 05274
Siyu Liao, Jie Chen, Yanzhi Wang, Qinru Qiu, and Bo Yuan. Embedding compressioon with isotoopcc iterative quantization. CoRR, abs/2001.05314, 2020. URL.
2001
05314
Timothee Mickus, Denis Paperno, Mathieu Constant, and Kees van Deemter. What do you mean, bert? assessing BERT as distributional semantics model. CoRR, abs/1911.05758, 2019. URL
Timothee Mickus, Atig-Ame Grönroos, URL hbt 03191
Grönoroos. and Joseph Attieh. Isotropy, clusters, and classifiers, 2024.
abs/2402..222
Jiaqi Mu, Suma Bhat, and Pramod Viswanath. All-but-the-top: Simple and p for word representations. CoRR, abs/1702.01417, 2017. URL http::/arxiv.or///a0
1702. 01417
Sara Rajaee and Mohammad Taher Pilehvar. How does fine-tuning affect the geometry of embedding space: A case study on isotropy. CoRR, abs/2109.04740., 2021a. URL
abs/2108 04740
Sara Rajaee and Mohammad Taher Pilehvar. A cluster-based approach for improving isotropy in contextual embedding space. In Proceedings of the 59th Annual Meeting of the Associ- ation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 575-584, Online, August 2021b. As- sociation for Computational Linguistics. doi: 10.18653/v1/200 wwc.//222...sorr..2. URL https
7/aclanthology. org/
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383- 23922 Austin, Texas, November 2016a. Association for Computational Linguistics. doi: 10.18653/v 1/D16- 1264. URL https: http://:: httpt::://alaatthllle Btg//166-133
11

---

Published as a conference paper at ICLR 2024
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100, 000+ questions for machine comprehension of text. CoRR, abs/1606.052550, 2016b. URL http:://axxi..orr
abs/1606. 05250
Stefano Recanatesi, Matthew Farrell, Madhu Advani, Timothy Moore, Guillaume Lajoie, and Eric Bioea-Broww. Dimensionality compression and expansion in deep neural networks. CoRR, abs/1906.00443, 2019. URL http::/axxiv org/ abaa 1906. 00443
William Rudman, Nate Gillman, Taylor Rayne, and Carsten Eickhoff. IsoScore: Measuring the uniformity of embedding space utilization. In Findings of the Association for Computational Lin- guistics: ACL 2022, pages 3325-3339, Dublin, Ireland, may 2022. Association for Computational ac1.262. URL https: htttt::://aclantooooggo.o
Linguistics. doi: 10. 10.18653/ 2022. www.iiiinnnggacccccon act. 262
Hassan Sajjad, Firoj Alam, Fahim Dalvi, and Nadir Durrani. Effect of post-processing on contextual- ized word representations. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3127-3142, Gyeongju. Republic of Korea, October 2022. International Commit- tee on Computational Linguisties. URL https::/tsaatt htttpss//aclanthooooy.cooo
coling-
277
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter, 2020.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng. and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1642, Seattle, Washington, USA, October 2013. Association for Computational Linguistics. URL / org/ Dr9/011111100
William Timkey and Marten van Schijndel. All bark and no bite: Rogue dimensions in transformer language models obscure representational quality. CoRR, (bss/2199044// 2021. URL https
www.gg/bbb///222 04404
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353-355, Brussels, Belgium, November 2018. Association for Computational Linguistics. doi: 1/1/3///1/1111///// URL
5446
Alex Warstadt, Amanpreet Singh, and Samuel Bowman. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625-641 2019. doi: 10.1162/
tacl_a_0029p URL httpp://a// http://lllaantttloooo.ooooooo.//
9-1040
Zhong Zhang, Chongming Gao, Cong Xu, Rui Miao, Qinli Yang, and Jumming Shao. Revisiting representation degeneration problem in language modeling. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 51 518-527, Online, November 2020. Association 1t0.000.nnniing-mmml.....4 URL https://
for Computational Linguistics. doi: 10. 00.18653/v120000.fl
waw.antholooy..rg/g/// http://o///22....niinn.eeeemeeee findings em.....mmpp.......
Wenxuan Zhou, Bill Yuchen Lin, and Xiang Ren. Isobn: Fine-tuning BERT with isotropic batch nor- malization. CORR, abs/2005.02 0278 2020.
http://aaxxivv.orrr///b//00//0./ 021
Zhanxing Zhu, Jingfeng Wu, Bing Yu, Lei Wu, and Jinwen Ma. The anisotropic noise stochastic gradient descent: Its behavior of escaping from sharp minima and regularization effects, 2018. 00195
URL https: :ectps://azee //axxivaxivvvv//aa
12

---

Published as a conference paper at ICLR 2024
Algorithm 2 IsoScore
1: Input: Let X C IRd be a finite collection of points.
2: Let XPCC denote the points in X transformed by the first n principal components.
3: Define Ep E R" as the diagonal of the covariance matrix of X XPGA
4: Normalize diagonal to Σn == vn. Ep/ / where || is the standard Euclidean norm.
5: The isotropy defect is
(X): = 15= 111/ 2(n vn)
where 1 = (1, 1)T E R..
6: X uniformly occupies
(X) = - 8(X) ( /r))2/n²²
percent of ambient dimensions.
7: Transform ((X) so it can take values in [0, 1]. via (XX) == (n o(x 1)/(n 1).
8: return: (X)
A ISOSCORE VS ISOSCORE
IsoScore(X) = IsoScore" (.C) when <= 0. Let X C R" be a finite point cloud that we assume sampled from some larger distribution X such that the number of points in X sufficiently larger than the number of points in X. Let Ç E (0, 1) be a shrinkage parameter, and let Es E RTxx be a shrinkage covariance matrix obtained from a sample of points, S, drawn from X such that
|S| >> We will first demonstrate that without RDA-Shrinkage (i.e. when C = 0), IsoScore IsoScore*(X, Es). The key insight is that En in Step 3 of Algorithm is equivalent to the principal components of X.
Algorithm 2 shows that the first step of IsoScore is to transform X to its principal components to get what the authors denote as XPCA. Let ΣPCA be the covariance matrix of XPCA, and let Σx denote the covariance matrix of X. Projecting X to its principal components removes correlations from the data, meaning that ΣPCA will be diagonal. Since ΣPCA is diagonal matrix, its eigenvalues are equal to diog( EPCA). Therefore, diag(Erex) are the principal components of XPCA since principal components are the eigenvalues of the covariance matrix. Note that principal components are invariant under orthogonal transformations and that "reorienting" the data by PCA applies the orthogonal transformation V XV, where V are the eigenvectors of Σx. Namely, the principal components of X are the principal components of XPCA For simple proof demonstrating that the principal components of X are invariant under any orthogonal transformation applied to X, see (Ding et al. 2006). Therefore, IsoScore* is equivalent to IsoScore when no covariance matrix shrinkage is performed, That is, when ( = O, IsoScore(X) = IsoScore" (X, &, Es) VX E TRA. To see that IsoScore (X) approaches IsoScore* (X, Ç, ss), we use the Law of Large Numbers to show that the larger the sample of X, the more close X approximates the true distribution X. Therefore, IsoScore X) + IsoScore" (X, S, Es) as XX| increases.
Comparing Pseudocode. IsoScore* addresses two fundamental flaws in vanilla IsoScore. Firstly, IsoScore* uses RDAA-sriikage (Friedman 1989) to stabilize the calculation of a sample covariance matrix. Secondly, IsoScore" removed all non-differentiable operations present in IsoScore. The pseudocode for IsoScore and IsoScore* has two primary steps: 1) extract distribution/isotropy information from the point cloud, S and 2) normalize the isotropy information into score in the closed interval [0. 1]. In Algorithm 3 Steps 3-4 calculate the covariance matrix of the point cloud S, and perform RDA- RDA-shrinkage by taking the weighted sum of the covariance matrix of S and a covariance matrix of larger distribution from which S was sampled. Step 5 then calculates the eigenvalues from the resulting sum of covariance matrices. When we set Ç Step 4 to O, the resulting cigenvalues are the principal components of our sample S. All normalizing steps are identical in IsoScore and IsoScore*. Namely, lines 6-9 in Algorithm are equivalent to lines 4-7 in Algorithm
of IsoScore. We want to highlight the exact step in Algorithm 2 that makes IsoScore non difddffeernniable.. Step 3 in Algorithm 2 involves selecting the diagonal of a covariance matrix, which is non-differentiable operation. Note that the non-differentiabiiiyy of IsoScore does not imply that IsoScore* is -d-iffferentiable even though IsoScore and IsoScore* are equivalent when no shrinkage is performed. To further emphasize this point, consider the two functions f(x) x/W
13

---

Published as a conference paper at ICLR 2024
and g(x)) x for all z! = 0 and g(x) |a| when 0. Here, |æ| indicates the absolute value function. For all inputs 2, f(x) = g(x). However, f(z) is differentiable for all values of ø, where g(z) is not differentiable when B
O. Section demonstratess
that IsoScore* is equivalent IsoScore
when no shrinkage is performed. IsoScore* preserves all of the desirable theoretical properties of IsoScore while improving stability small inputs and removing the operation that makes IsoScore non-differentiable (namely, an iterative selection of the diagonal).
B WHY Do MODELS PREFER ANISOTROPY?
C LAYER-WISE ISOTROPY
ALBERT QNLI ISTAR
BERT QNLI I-STAR
DisciBERT QNLI E-SAAR
00
11.2
0.5
IuSoore
(12
00
Model
Figure 6: Liser-wise IsoScore* values for ALBERT, BERT, and DistilBERT fine-tuned with I-STAR using tuning parameters À f-1,1]. IsoScore" values are calculated the QNLI validation data using a shrinkage parameter of Ç = 0.2. "None" indicates that no regularizer is used in fine-tuning.
When fine-tuning models with I-STAR, we compute the IsoScore* penalty from the union of all token embeddings from each model layer. This section analyzes what layers in the model are impacted the most by our I-STAR regularizer.
Figure 6 shows that encouraging isotropy in token embeddings using I-STAR primarily impacts early layers in the network. Even when isotropy is encouraged using positive tuning parameter values in I-STAR, token representations from the later layers of the network remain highly anisotropy. These results provide further evidence that anisotropy in the form of outlier dimensions that emerge in the last layers of the network is crucial to the model decision-making process. An interesting direction for future work could be to explore applying I-STAR to various layers in the network.
D IMPACT OF THE SHRINKAGE PARAMETER ON I-STAR
In this section, evaluate the impact of changing the shrinkage parameter, Ç. downstream performance when fine-tuning with I-STAR regularization. To test the effect of varying Ç. we fix all hyperparameters found in Table and fine-tune with Ç E {0, 0.2, 0.4, 0.6, 0.8. 1). We train ALBERT, BERT, and DistilBERT on RTE. All results are reported as an average over random seeds.
ALBERT RTE
BERT RTE
DistiBBRTT RTE
0.7
0,70
Aerurac
1600
600
Oyeimall
60
022
0.1
0.6
DUB
1.1
80
0.2
0.1
0.6
D/5
1.0
80 0.2
0.4
0.6
DR
10
V-Tho
V/
Figure 7: Performance of ALBERT, BERT, and DistiBERT on RTE with changing values of Ç. The red dashed line denotes the optimal value of Ç. Setting Ç = indicates that no shrinkage is performed (i.e., equivalent to regularizing using IsoScore), and setting Ç = signifies that no mini-batch covariance information is included in the gradient updates during a backward pass.
14

---

Published as a conference paper at ICLR 2024
+ demonstrated that IsoScore systematically underestimates the actual value of point clouds when the number of samples lower than the dimensionality of the vector space. IsoScore" overcomes this fundamental limitation by using shrinkage to improve the stability of the covariance matrix calculation. Figure 7 demonstrates the importance of using IsoScore' when computing isotropy scores of mini-batch. When ( 0 and shrinkage is performed, the performance of our fine-tuned model decreases by 3.85% 6.19%, and 0.76% compared to optimal values of Ç for ALBERT, BERT, and DistilBERT, respectively.
In Section
In addition to testing the utility of using covariance matrix shrinkage, test the impact of excluding mani-bacch covariance information by setting Ç = 1. When Ç 1, IsoScore" calculated from the stabilizing covariance matrix, Σs, obtained by calculating the covariance matrix from a sample of 250,000 points at epoch i. Figure demonstaates the utility of using mini-batch covariance matrix information during fine-tuning as the optimal tuning parameter is always value of Ç E (0, 1).
E APPLYING I-STAR TO DIFFERENT LAYERS
Throughout this paper, we calculate a global isotropy penalty on the vector space of all token embeddings from all layers in the model. Our motivation for selecting global isotropy penalty instead of penalizing individual layers is two-fold. Firstly, IsoScore* is more faithful representation of the true isotropy of the global vector space when the number of samples is large, and the sample covariance is more likely to be full-rank, meaning that isotropy-based gradient updates will be more stable. Secondly, selecting individual layers penalize would drastically increase the hyperparameter search when using I-STAR. Lastly, we ultimately decided on global isotropy over regularizing individual layers as a global isotropy penalty resulted the most consistent results across different tasks.
In this section, we fine-tune ALBERT, BERT, and DistilBERT on COLA using 5 random seeds. We fix all hyperparameters in Table eexcept for the layer we select to apply I-STAR.
ALBERT COLA
BERT COLA
DISTBERT COLA
0.25
Stallhee''s Callbeen'CUrreaattts Carveaatee
061
0.0- wik
Matthen's
800
Mallthee
0:29
0.30
006
ALU
Model
Taverr
Mode'
Taver
Medel Tower
Figure 8: Impact of applying I-STAR different layers in ALBERT, BERT, and DistilBERT fine- tuned on COLA. The dashed red line is the result of applying a global I-STAR penalty on all layers of the model.
Figure X reports the performance of applying I-STAR to individual layers in the network. Although regularizing individual layers in ALBERT and BERT improve performance compared global isotropy penalty, a global isotropy penalty provides much more consistent performance across models and tasks. Namely, no consistent patterns emerge when applying I-STAR to individual layers. However, more work is needed to determine if the occasional improvement in performance is worth the extensive hyperparameter search induced by layer-wise I-STAR.
F DATASET DETAILS
Stanford Sentiment Treebank with 2 classes (SST-2) is binary classification task where models must determine whether a short movie review positive or negative in sentiment (Socher et al.) 2013). SST-5 is five-class version of SST-2 where the models must determine whether a movie review is negative, somewhat negative, neutral, or positive. QNLI is a binary natural language inference task where models must decide whether not given answer is entailed from specified question (Wang et al.) 2018). Stanford Question Answering Dataset V1 (SQUAD)is an extractive question- answering task where model must select the span of text in a passage that answers given question
15

---

Published as a conference paper at ICLR 2024
(Rajpurkar et al. 2016bb. Recognizing Textual Entailment (RTE) is binary classification task where a model must determine if a given sentence logically follows a preceding sentence. STS-B (Semantic Textual Similarity Benchmark) is collection of sentence pairs annotated with similarity score from 1-5. STS-B is commonly evaluated with Pearson's 's correlation coefficient. The Microsoft Research Paraphrase Corpus (MRPC) tasks models with determining if a pair of sentence are paraphrases of each other (i.e. semantically equivalent). Quora Question Pairs (QQP) consist of question pairs from Quora. Models must determine if the sentence pairs are semantically equivalent. Corpus of Linguistic Acceptability (COLA) task models to determine if given string is linguistically acceptable sentence. SST-2, QNLI, RTE, MRPC, STS-B, QQP, and MRPC are all datasets in the GLUE benchmark (Wang et al. 2018).
G I-STAR HYPERPARAMETERS
Table 2: Optimal 1-STAR hyperparameter values of the tuning parameter, À and shrinkage parameter, We searched over {-5, 3, -1, 1, 3,5, and {0.2, 0.4, 0.6, 0.8]. We present results as AlÇ. Note that all values of À are negative, meaning I-STAR will decrease isotropy in embedding space.
Method
SST-2
QNL
RIE
MRPC
QOP
COLA
STS-B
SST-
SQUAD
Avg.
T-STAR
-1
0.2
112
108
0.4
0.4
-1
0.2
-1
0.8
- 0.4
- 1.2
-5 0.2
-1.44
00.36
BERT I-STAR
0.2
0.4
-1
1.8
0.2
-1
-
-.2
-2.11
10.44
DistiBERT I-STAR
0.6
0.2
-06
0.8
0.8
-1 0.2
3 04
3 004
-1 -1.0.2
-502
-2.33 0..40
In this Section, outline the optimal hyperparameters used in I-STAR for each task and each model. I-STAR requires two key hyperparamcters: A, the tuning parameter, and Ç. the shrinkage parameter. Recall that the tuning parameter controls both the strength of the signal of the IsoScore* in the loss function and whether isotropy is encouraged or discouraged in model representations. Namely, when 0, I-STAR will increase isotropy in the embedding space, and when À < 0, I-STAR will decrease isotropy in the embedding space. The shrinkage parameter, Ç, determines how much covariance information comes from sample point cloud, X, and how much covariance information comes from Co. For all tasks and models, we limit our hyperparameter search to À {-5, -3, -1, 1, 3, 5] and E (0.2, 0.4, 0.6, 0.8}. Note that all optimal tuning parameter values occur when X < O, meaning further decreasing isotropy leads to better performance gains than encouraging isotropy.
16


Reviews:
[{"review_id": "YHK5HV24np", "reviewer": "Reviewer_GH3h", "summary": "This study investigates the connection between isotropy and model performance by employing 3 distinct LLMs and 9 fine-tuning tasks. It introduces I-STAR, a method for adjusting model isotropy based on a novel differentiable metric called IsoScore*.\n\nSurprisingly, the study's findings contradict the prevailing notion in NLP literature. That is, it demonstrates experimentally that discouraging isotropy leads to better performance across the different models and downstream tasks.", "questions": "Could you add in the Appendix the same experiments done in Figure 3 but for CosReg and also the same in Figure 4 but for IsoScore*? This will give a better intuition of how these 2 different methods work.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "This paper challenges the dominant belief in NLP literature showing that anisotropy is beneficial. Its findings have the potential to significantly influence future research directions in the field.\n\nIt also introduces new way to compute the Isotropy in models. The authors conducted a set of experiments to show its efficiency comparing it to CosReg.", "weaknesses": "If we see a trend in Figure 3 on how higher IsoScore* leads to lower accuracy, some correlation and significance score should be added to support this claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "clPWHCeRVj", "reviewer": "Reviewer_ZAyV", "summary": "This submission proposes a new method for measuring isotropy in neural models, improving over previous proposals and leading to a new regularization technique I-STAR. They show that LLMs actually seem to benefit from less isotropic internal representations, contrary to previous claims in the NLP literature.", "questions": "* p.4: Presumably all norms in steps 6 and 7 of Algorithm 1 are 2-norms?\n* p.4: Is RDA \"regularized discriminant analysis\"? It is only introduced on p.5.\n* p.4: As \\Sigma_S is computed from a large number of tokens and re-estimated after each epoch, this is presumably quite costly computationally. Could you comment on that? As you are computing these anyway, why not use that directly to estimate isotropy? This would essentially correspond to \\zeta=1, which is not tested here as far as I can tell.\n* p.6: \"Section F\" -- do you mean Section D?\n* p.7: I do not clearly see Fig. 4 as supporting the claim that CosReg only alters the mean of activations (all means seem to be ~zero on the plots, with one outlier). Maybe a plot of the distribution would support this claim better?\n* p.9: Why are -1 and +1 lambdas excluded from Fig. 5? -1 seems the most popular choice in Fig. 2.\n* Regularization to a manifold in parameter space is well studied in Machine Learning, and indeed supports the argument that anisotropy may benefit performance. This is also linked to the idea that there is the level of representativity in the model that must match the intrinsic dimension or complexity in the data. Fig. 5 actually seems to show roughly consistent intrinsic dimensions for the three models used here. Is there a way you could put this in perspective with e.g. model size? Showing for exemple how anisotropy favors a roughly stable internal dimensionality in parameter space as model size increases?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well written and clear. The proposed improvement of IsoScore into IsoScore* is fairly straightforward. The fact that it is a more accurate and more convenient estimate of isotropy in LLMs is argued very well and supported by some empirical results.\n\nThe paper convincingly argues that isotropy and its impact on performance are not properly understood in NLP, which is a very significant contribution.\n\nExperimental results mostly support the arguments in the paper (more comments below).", "weaknesses": "* There is no significance testing on the results (Table 1) but there are error bars (good!) — these seem to indicate that most differences outlined are hardly significant (e.g. RTE, 72.56+/-1.29 vs 71.34+/-0.91). This makes it difficult to get a clear picture of the resulting effect of decreasing isotropy.\n* Similarly Fig. 3 is difficult to interpret — there are clear decreasing trends in some plots, not so much in most of them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l0mFxxxErH", "reviewer": "Reviewer_tWNn", "summary": "The authors improved upon IsoScore, a measure for the isotropy of point clouds (Findings of ACL'22), and proposed the IsoScore* as a measure of **an**isotropy. IsoScore* employs PCA (as a natural method for estimating the covariance matrix) and utilizes shrinkage estimation, making it a differentiable measure that operates robustly even with small data sizes. In experiments, IsoScore* was used as a regularization term when fine-tuning masked language models. As a result, under optimal hyperparameters, performance improvements were observed across multiple downstream tasks.", "questions": "- If anisotropy is a natural consequence of SDG, what is the significance of deliberately adding anisotropy as a regularization term? The computational cost increases slightly, and the disadvantage of increased hyperparameter tuning cost needs to be offset by some substantial benefits. If there are such advantages, the appeal of the proposed method would be enhanced.\n- What is the significance of adjusting the hyperparameter $\\zeta$ for shrinkage estimation when using IsoScore* as a regularization term? Even if the value of IsoScore is estimated to be on the lower side (Figure 2), if the estimated value has a monotonic relationship with the true value, wouldn't there be no issue in regularization? If the goal is to ensure differentiability, couldn't we just fix it at an appropriate value?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- In the NLP field, the isotropy of internal representations was believed to be the key to model success. The message of this paper, suggesting that the **an**isotropy of internal representations might be the key to performance improvement, will likely resonate intriguingly with many readers.\n- The paper comprehensively covers a collection of works related to the isotropy of NLP models, making it a highly self-contained piece for readers.", "weaknesses": "### 1. The reasons for contradictions with prior research are unclear, weakening the persuasiveness of the main claim.\nThe authors' main claim that \"**an**isotropy is the key to model performance improvement\" isn't reconciled with prior research which posits that \"isotropy is the key to model performance improvement\". While the authors suggest that the discrepancy arises from the evaluation metrics used (as stated “previous studies have made claims using “flawed” measures of isotropy,” on page 7), both this work and prior studies differ *not* only in metrics but also in tasks (GLUE tasks vs. word similarity tasks). Therefore, it's not an apple-to-apple comparison. If there's an inverse correlation between anisotropy and performance in word similarity tasks, it becomes challenging to coherently explain the overall results. This issue might possibly be resolved if there were experiments or comprehensive discussions specifically for the word similarity tasks.\n\n### 2. The claim that the proposed method improves performance on downstream tasks seems a bit overstretched.\nEven when choosing the best settings across multiple hyperparameters, the performance improvement is modest (as seen in Table 1). Moreover, performance can deteriorate compared to the baseline depending on hyperparameter choices (as shown in Figure 3). Furthermore, adopting the proposed method incurs an additional cost of hyperparameter selection. Therefore, for practitioners aiming to employ IsoScore* for their problems, it's hard to advocate for the use of the proposed method. Of course, submissions to ICLR shouldn't be evaluated solely on empirical performance. However, in this paper, improving performance on downstream tasks is one of the main contributions; thus the lack of compelling experimental results will inevitably impact the paper's peer-review evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)