On the Analysis and Distillation of Emergent Outlier Properties in Pre-trained Language Models (2025)

Abstract: A small subset of dimensions within language Transformers’ representation spaces emerge as “outliers” during pretraining, encoding critical knowledge sparsely. We extend previous findings on emergent outliers to Encoder-Decoder Transformers and instruction-finetuned models, and tackle the problem of distilling a student Transformer from a larger teacher Transformer. Knowledge distillation reduces model size and cost by transferring knowledge from a larger teacher to a smaller student, necessitating a trade-off among representation dimensions. We show that emergent outlier dimensions contribute significantly more to zero-shot performance than non-outlier dimensions. Based on this, we propose the Emergent Outlier Focused Distillation (EOFD) method, which prioritizes critical outlier dimensions in distillation using a weighted MSE loss. We empirically demonstrate that EOFD outperforms state-of-the-art distillation methods and generalizes well across Encoder-only BERT, Decoder-only GPT-2, and Encoder-Decoder T5 architectures.

## introduction and background emergent properties in large language models (llms) have recently garnered great interest @xcite @xcite . they have been shown to elicit complex capabilities in llms. emergent properties and features arise spontaneously in these models during self-supervised pretraining, without being explicitly optimized for specialized tasks. specifically, it has been shown @xcite @xcite that, in encoder-only bert family @xcite and in decoder-only gpt family @xcite @xcite models, a small subset of dimensions within the high-dimensional representation spaces of language transformers emerge as "outliers" during pretraining: weight or neuron activations with unusually large magnitudes out of several standard deviations from the mean. interestingly, these emergent outlier features seem to encode critical linguistic knowledge in a sparse way: muting only a few outlier dimensions significantly deteriorates language modeling performance. in this work, we begin by systematically extending previous findings about emergent outlier properties on pretrained encoder-only and decoder-only models to encoder-decoder t5 models @xcite and to instruction-finetuned flan-t5 models @xcite at scale, for the first time. we discover that t5-11b exhibits emergent activation outliers with surprising magnitudes exceeding 10 5 , much larger than the bert outliers observed by @xcite ; @xcite . furthermore, unlike the rapid emergence of gpt outliers around a model size of 6.7b as found by @xcite , we notice that the growth in outlier magnitude primarily comes with increasing layer depth rather than model size: from t5-large to t5-3b and to t5-11b, all with a same number of layers, the outlier magnitudes actually decrease as model size increases. we further find that, agreeing with previous work, outlier dimensions are consistent across layers within either the encoder or decoder stack; but, contrary to previous knowledge, the outlier dimensions in the encoder differ from those in the decoder. moreover, consistent with @xcite , we notice that outlier features suddenly become crucial to performance when the model size exceeds 6.7b in t5: zeroing out only 4 outlier dimensions out of its 1024 total dimensions (only 192 of its total 11b parameters) in t5-11b degrades absolute performance by 14.7%. however, despite this and @xcite , for instruction-finetuned flan-t5, we notice that larger models like flan-t5-xxl are relatively less sensitive to interventions on outliers than smaller models. nevertheless, we confirm that disabling outlier dimensions hinders performance significantly more than disabling the same number of non-outlier dimensions, for all our different settings. for instance, muting as much as 512 non-outlier dimensions in flan-t5-xxl only drops its performance by 0.64%. leveraging our findings on emergent outlier properties, we further tackle the challenging problem of distilling a student language model from a larger and stronger teacher model, by focusing on these critical properties. training massive models is computationally demanding and requires a vast treasure trove of varied data, both of which are not easily available. additionally, at inference, hosting such large models also gets progressively expensive. to mitigate this, knowledge distillation (kd) @xcite aims at reducing model size without significantly compromising performance by transferring knowledge from a stronger teacher to a smaller student by minimizing the divergence of their soft responses and intermediate features @xcite . for distilling these intermediate features, conventional methods treat different dimensions equally @xcite @xcite . however, matching student's intermediate features to teacher's is inherently imperfect, as student transformers typically have fewer dimensions of representation than teachers, necessitating a tradeoff among these dimensions. given our discovery that emergent outlier dimensions contribute much more to performance than non-outlier dimensions, when distilling these intermediate representations, we propose emergent outlier focused distillation (eofd). this approach prioritizes these critical outlier dimensions and deprioritizes the less impactful non-outlier dimensions, addressing the dimension trade-off in distillation. specifically, eofd computes a weighted mse loss to weight more on the emergent outlier dimensions, recognized by the standard deviations of neuron activations. on the standard benchmark of distilling bert on the 8 tasks and datasets in the general language understanding evaluation (glue) @xcite ) benchmark, we outperform state-of-the-art distillation methods by a large margin. on the relatively larger datasets in glue, student models distilled with eofd outperform the teacher models. beyond distilling encoder-only bert models, we further demonstrate that eofd generalizes well to other architectures, including decoder-only gpt-2 and encoder-decoder t5 models. we also provide detailed ablation and analysis. we discuss further related works in appendix a. our contributions are two-fold: (1) for the first time, we extend previous findings on emergent outliers to encoder-decoder transformers and to instruction-finetuned models at scale. we further systematically study their zero-shot performance with interventions on muting representation dimensions by factors of different numbers of disabled outlier/non-outlier dimensions, varying model size, and whether or not the pretrained model is further instruction-finetuned. (2) leveraging our findings on emergent outlier properties, we propose the emergent outlier focused distillation (eofd) method prioritizing these critical outlier dimensions to address the dimension trade-off in knowledge distillation. we empirically show that eofd outperforms state-of-the-art distillation methods. we further show that eofd generalizes well across the tasks of distilling encoder-only bert, decoderonly gpt-2, and encoder-decoder t5 models. ## a closer look at emergent outliers in pre-trained language models in this paper, we use plain lower case letters x for scalars, bold lower case letters x for vectors, bold upper case letters x for matrices, and x t for transposes. we index each transformer block/layer in a given transformer by @xmath0, • • • , l}, and each token in token sequence by @xmath1, • • • , n }, as illustrated in fig. 3 . typically, each transformer block/layer contains a multi-head attention module, a feed forward network (ffn), and some layer-norm (ln) transformations. denote the dimension of a transformer as d model as illustrated in fig. 3 ; specifically, denote d t and d s for the dimensions of a teacher and a student model, respectively. we consider for only one data sample (not batched) for notation simplicity unless otherwise specified. ## distilling transformers: background for distilling transformers, l pred , l att , and l hid are commonly applied to transfer knowledge from the teacher t to the student s. the prediction logits distillation loss l pred minimizes the divergence between the soft response from t and s: l @xmath2, z t /τ d ), where τ d denotes the temperature, z s and z t refer to the classification logits (commonly over the tokens in vocabulary) from s and t, and ce denotes cross entropy, i.e. and l @xmath3. the attention map distillation loss l att minimizes the average mean squared error (mse) between the attention matrices of each head of t and s: @xcite , where h denotes the number of attention heads, a @xmath4, and n refers to the sequence length of tokens. the intermediate representation distillation loss l hid minimizes the divergence between the hidden state matrices for each transformer block, as in eq. ( 3 ), and we will investigate this loss by outlier or non-outlier dimensions. ## emergent outlier focused distillation we have just shown that different representation dimensions in pretrained language models do not contribute equally to performance: muting the outlier dimensions harm performance significantly, while disabling as much as 512 non-outlier dimensions do not for flan-t5-xxl. therefore, when distill-ing intermediate representations, we propose to focus on these more important outlier dimensions and pay relatively less attention to the non-outlier dimensions. for notation clarity, we consider for a single data sample (not batched) unless specified otherwise. as in fig. 3 , for the l-th layer, denote h t @xmath5,i ) t } @xmath6,••• ,n , and denote h s @xmath7. conventional intermediate representation distillation loss computes the mean squared error (mse) as below, where w @xmath8: we propose to compute a weighted mse loss, the emergent outlier focused distillation loss l eofd , instead, weighting more on the outlier dimensions and weighting less on other dimensions, recognized by the standard deviations of activations of these dimensions in the teacher: to compute w eofd l,j , we normalize σ l,j by dividing it by the mean of these standard deviations for the given layer l, so that the normalized σ l,j for a given layer l has a mean of 1. we then raise its quotient transformer block 1 how to distill a large language d @xmath9: #308 an outlier dimension: #308 a non-outlier dimension: #600 a non-outlier dimension: #600 residual teacher transformer block 1 how to distill a large language transformer block 2 student linear proj (h s 1 w proj ) t (h t 1 ) t outlier-focused weighted mse loss larger weight w 1,308 on outlier dim #308 smaller weight w 1,600 on non-outlier dim #600 to the power of p to tune the strength of weighting. we use @xmath10.5, 1} for our experiments. note that when @xmath11, with uniform weighting of w eofd l,j ≡ 1, the eofd loss in eq. ( 4 ) will be degraded to the vanilla mse loss in eq. ( 3 ): we provide pytorch (paszke et al., 2019) code for the proposed eofd loss in appendix i. ## experiments on eofd we distill bert @xcite , gpt-2 (radford et al., 2019) and t5 (radford et al., 2019) models and evaluate on the general language understanding evaluation (glue) @xcite benchmark. we report model architectures and dataset details in appendix d. ## conclusion we have analyzed the emergent outlier phenomenon and its effect on performance for pretrained t5 and instruction-finetuned flan-t5 models of size varying from 60m to 11b. based on the analysis, we have proposed, for the first time, to leverage these findings on outliers for more effective knowledge distillation methods, and have empirically shown that our proposed eofd method achieves sota performance.