Spectral Insights into Data-Oblivious Critical Layers in Large Language Models (2025)

Abstract: Understanding how feature representations evolve across layers in large language models (LLMs) is key to improving their interpretability and robustness. While recent studies have identified critical layers linked to specific functions or behaviors, these efforts typically rely on data-dependent analyses of fine-tuned models, limiting their use to post-hoc settings. In contrast, we introduce a data-oblivious approach to identify intrinsic critical layers in pre-fine-tuned LLMs by analyzing representation dynamics via Centered Kernel Alignment (CKA). We show that layers with significant shifts in representation space are also those most affected during fine-tuning—a pattern that holds consistently across tasks for a given model. Our spectral analysis further reveals that these shifts are driven by changes in the top principal components, which encode semantic transitions from rationales to conclusions.We further apply these findings to two practical scenarios: efficient domain adaptation, where fine-tuning critical layers leads to greater loss reduction compared to non-critical layers; and backdoor defense, where freezing them reduces attack success rates by up to 40%.

## introduction large language models (llms) have revolutionized natural language processing, excelling in text generation, reasoning, and comprehension @xcite @xcite . their success has driven the adoption of leading models like llama @xcite and gemma @xcite , which exhibit near-human intelligence. however, a critical need remains to understand how an llm's representations evolve across layers (i.e., representation dynamics). gaining such insights can help elucidate how these models process information at our code is available at github. different depths and identify key layers where semantic shifts occur or are most vulnerable to adversarial attacks @xcite @xcite . some previous studies analyze representation dynamics to provide explanations of llms. for example, @xcite ; @xcite ; @xcite ; @xcite reveal distinct layer functions by connecting the representation space to the vocabulary space, offering insights into linguistic processing. however, their explanations are often data-dependent, at the instance or group level, and are affected by the domain of the data. other prior works aim to identify critical layers that are associated with specific llm behaviors or are more vulnerable to adversarial attacks. for example, recent studies @xcite show uneven layer training during fine-tuning, while @xcite find certain layers better distinguish backdoor from normal data. however, these studies often require full access to the fine-tuning data and processes to identify critical layers. in practice, identifying critical layers in pre-fine-tuned models-whether pre-trained or initially fine-tuned-can guide downstream fine-tuning strategies, such as adjusting layer-wise learning rates @xcite or applying targeted defenses @xcite . furthermore, we require these layers to be data-oblivious-i.e., unaffected by the finetuning data-to ensure their applicability across various fine-tuning tasks. unlike previous works, we identify dataoblivious critical layers-those most susceptible to modification during fine-tuning-by analyzing the representation dynamics of pre-fine-tuned models. furthermore, we take a spectral perspective to analyze the implications of these data-oblivious critical layers and provide explanations for their role in the model. in more detail, we use centered kernel alignment (cka) @xcite to track representation changes across layers and find that change-point layers remain unaffected by different fine-tuning data. additionally, these layers undergo the most significant modifications during fine-tuning, reflecting an intrinsic property of the model. to understand why these layers are the most critical and what factors drive the intrinsic representation shifts (measured by the change in cka), we conduct spectral analysis of these layers. our results show that the changes in cka are attributed to variations in the principal components of the representations. in particular, the top three principal components change significantly at these layers and play a key role in summarizing the rationales into conclusions. this spectral analysis of data-oblivious critical layers yields two key implications. first, when fine-tuning is limited to a subset of layers, updating the critical layers enables more efficient domain adaptation, as evidenced by faster and greater loss reduction. second, freezing these layers during fine-tuning improves robustness to backdoor attacks, as confirmed by our experiments. we summarize our contributions as follows: ## preliminaries we begin by introducing the notation and definitions used in this paper, followed by background information on llms, measures of representation similarity, and supervised fine-tuning. notation. the i-th input sequence, consisting of t tokens, is represented as z @xmath0,1 , z i,2 , . . . , z i,t ], where each token z i,s belongs to a predefined vocabulary. an llm with parameters θ processes this sequence, where θ (ℓ) represents the parameters of layer ℓ. at layer ℓ, each token z i,s is mapped to a hidden representation h during supervised fine-tuning of an llm, each input sequence has a corresponding ground-truth completion, denoted as: ## analysis on data-oblivious critical layers & representation dynamics in this section, we first explain how we find dataoblivious critical layers during the sft process. next, we analyze representation shifts in the prefine-tuned models to identify change-point layers. finally, we show that these change-point layers align with the critical layers identified during the sft stage. fig. 1 illustrates this process. ## spectral analysis on representation space as highlighted in the previous section, changepoint layers are inherent properties of a pre-finetuned model. in this subsection, we aim to explore the underlying factors driving these inherent representation shifts. motivated by the relationship between cka and spectral properties @xcite , we conduct a spectral analysis of the representations. specifically, we are interested in the in this subsection, we analyze how representation spaces evolve across layers by examining changes in their principal components. given a representation matrix x (ℓ) ∈ r n ×d ℓ from the ℓ-th layer (sec. 2), we perform singular value decomposition (svd) on the centered matrix x (ℓ) (eq.1) as follows: here, u (ℓ) is an n × r orthogonal matrix whose columns correspond to the left singular vectors of x (ℓ) . similarly, v (ℓ) is a d ℓ × r orthogonal matrix whose columns correspond to the right singular vectors of x (ℓ) , and σ (ℓ) is a diagonal matrix containing the singular values. we obtain the k-th principal features f it represents the k-th direction in the transformed representation space at layer ℓ. additionally, we observe that the eigenvalue distribution is highly skewed, with the largest eigenvalues exhibiting near-identical values. this characteristic poses challenges in tracing specific eigenvectors across layers, as they may not be consistently aligned, nor are the principal features. to investigate the correlation of these principal features, we adopt canonical correlation analysis (cca) @xcite to measure the similarity between subspaces spanned by these principal features of different layers. we then compute the average cca correlations between layer ℓ and its local neighbors l ℓ local to capture how the principal components evolve across the model's layers. fig. 3 shows the average cca correlation between top principal components from layer ℓ and those of the neighboring layers in l ℓ local . notably, the top1 principal component remains largely stable across layers, with values close to 1. in contrast, the top3 principal components closely follow the average cka pattern, showing sharp drops at layers 4 and 12 while staying high elsewhere. this trend disappears when considering the top10 principal components, suggesting that the cka pattern is more closely tied to spatial distortions induced by the top3 principal components rather than the others. to further assess the broader applicability of our conclusions across different datasets, we quantify the relationship between the cca values of the top3 principal components and the cka similarity δ ℓ using spearman's rank correlation, as presented in tab. 3. as shown in the table, both the llama2-7b-chat and llama2-13b-chat models exhibit high correlation across most datasets. the only exception is the llama2-7b-chat model on the alpaca dataset, where the cka metric is primarily influenced by the top4 principal components, yielding a correlation of 0.788. a similar trend is observed across other models, where cka is largely determined by the top3 principal components. these findings support our hypothesis that shifts in the second and third principal components account for most of the observed representation shift, as reflected in the cka results. average cka similarity( ) ## application on critical layers in this section, we present two key implications of identifying data-oblivious critical layers. first, in resource-constrained domain adaptation scenarios where only a subset of layers can be fine-tuned, selecting the critical layers enables more effective adaptation, as indicated by lower fine-tuning loss. second, freezing these layers during fine-tuning enhances the model's robustness to backdoor attacks by limiting its ability to incorporate harmful information. ## conclusion in this work, we uncover a fundamental link between change-point layers-where pre-fine-tuned llms exhibit sharp shifts in representation-and data-oblivious critical layers that undergo the most significant changes during fine-tuning. through spectral analysis, we show that these layers are intrinsic to the model's architecture, with their top principal components driving semantic transitions from input rationales to conclusions. notably, these critical layers remain consistent across downstream tasks for a given model, offering task-agnostic insights into llm behavior. leveraging this discovery, we develop an efficient domain adaptation method that fine-tunes only the critical layers, as well as a lightweight defense strategy that freezes them during fine-tuning-reducing backdoor attack success rates by 40% with minimal overhead. our findings advance the understanding of llm layer dynamics and inform the design of data-oblivious training and defense strategies.