Title: Order-Level Attention Similarity Across Language Models: A Latent Commonality
Venue: NeurIPS 2025
Accepted: yes
Keywords: Exploration of language model commonalities

Language model adapter transfer
Abstract: In this paper, we explore an important yet previously neglected question: Do context aggregation patterns across Language Models (LMs) share commonalities?
While some works have investigated context aggregation or attention weights in LMs, they typically focus on individual models or attention heads, lacking a systematic analysis across multiple LMs to explore their commonalities.
In contrast, we focus on the commonalities among LMs, which can deepen our understanding of LMs and even facilitate cross-model knowledge transfer.
In this work, we introduce the Order-Level Attention (OLA) derived from the order-wise decomposition of Attention Rollout and reveal that the OLA at the same order across LMs exhibits significant similarities.
Furthermore, we discover an implicit mapping between OLA and syntactic knowledge.
Based on these two findings, we propose the Transferable OLA Adapter (TOA), a training-free cross-LM adapter transfer method.
Specifically, we treat the OLA as a unified syntactic feature representation and train an adapter that takes OLA as input.
Due to the similarities in OLA across LMs, the adapter generalizes to unseen LMs without requiring any parameter updates.
Extensive experiments demonstrate that TOA's cross-LM generalization effectively enhances the performance of unseen LMs. 
Code is available at \url{https://github.com/jinglin-liang/OLAS}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel study exploring the commonalities in context aggregation patterns among LMs. First, it unifies the attention weights of the models into a comparable representation and introduces the concept of Order-Level Attention (OLA) derived from the decomposition of Attention Rollout. Both quantitative and qualitative methods reveal significant similarities in the same-order OLA across different LMs, and it is found that OLA encodes rich syntactic information. Based on this finding, the researchers propose the Transferable OLA Adapter (TOA), a training-free cross-model adapter transfer method. The experimental results demonstrate that TOA can effectively enhance the performance of LMs on syntactic-related tasks and facilitate cross-model knowledge transfer.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The research questions explored in this paper are far-reaching; exploring commonalities between different LMs can support cross-model knowledge transfer.\n2. The research methods are comprehensive and insightful. From discovering common structural aspects to applications, the authors clearly explain the motivations and thought processes involved in the whole process, which will benefit similar studies.\n3. This paper conducts extensive experimental verification and theoretical proof, which fully proves the rationality of the OLAS phenomenon.\n\nWeaknesses:\n\n1. The motivation for linking OLA with syntactic information is vague.\n\n\n2. The applicability of the TOA method is limited.\n\nOn the one hand, TOA seems to focus mainly on syntax-related tasks, and on the other hand, it is only suitable for models that originally lack the ability in syntactic structure-related tasks. From the results of Gemma2-9B, TOA does not seem to achieve effective results when the model performs well. This may be because the improvement in TOA performance mainly comes from OLA, which contains rich general syntactic structure information. When the model already has strong capabilities in syntactic structure-related tasks, OLA's capabilities may not be enough to support its original capabilities, thus losing its advantages.\n\nQUESTIONS:\n1. What is the principle of linking OLA with syntactic knowledge? \n2. Intuitively, the authors' findings stem from the observation that the OLA mappings of the same sentence in different models are highly similar. However, does this phenomenon also occur in sentences with similar semantic/syntactic structures? If semantically similar sentences do not show similar OLA mappings, but syntactically similar sentences do, then the motivation for this association may hold.\n3. The TOA method shows ineffective results on G-9b, and I noticed that the OLA's ability to predict syntactic dependencies for G-9b is not reported in Table 3. Could there be different conclusions for G-9b? For example, does G-9b's OLA fail to accurately predict syntactic dependencies, or does it achieve better performance in predicting syntactic dependencies compared to other models? This could further explain the applicability of the TOA method.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates context aggregation patterns across LMs by using Order-Level Attention (OLA). The authors find that OLA patterns are consistent across models and align especially well with syntactic patterns. Leveraging this, they introduce a Transferable OLA Adapter (TOA) that shows some transfer abilities to unseen models without further training.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The authors address an interesting and relevant problem of (i) analyzing higher-order patterns in LMs and (ii) investigate good agreement of resulting patterns across different models.\n- The methods are clearly defined and appear to be reproducible with reasonable efforts. Ablations, i.e. perturbation of the model parameters, further help to  support the results\n- Extensive experiments on a variety of language models (CLMs, MLMs) and various tasks.\n\n**Weaknesses**\n- The main theoretical contribution appears to be a reformulation of attention rollout from a multi-order perspective.\n- “We employed an image classification model as a proxy for human evaluation to objectively assess the similarity of OLA maps“  (l.182-183). I do not see how this is related to human evaluation. I was surprised to not see a more direct measure of relatedness, i.e. using correlation or distance measures between attention matrices (also in relation to the methods of representational alignment/representational similarity analysis). The image-based retrieval (Section 3.3.2 is a good step in this direction).\n- This evaluation makes it difficult to infer the main result that “LMs exhibit significant similarity”, as it is accessed by a classification into the same group. It does not directly measure that the patterns across LMs are comparable or similar (as suggested in Figure 3). It could further be that different source models exhibit different patterns and the classification step merely matches it to one of these.\n- The practical utility of the transfer setting (TOA) is not clearly demonstrated. While the authors show that training on the source model OLAs allows transfer to other LMs with improved performance over zero-shot, overall, performance is not competitive, e.g. for the NER task on CoNLL 2012 current state-of-the-art is at 80-84% with best scores on TOA is at around 69%. \n- Missing related works on higher-order explanations that encode structural information such as messages between sequence elements (see relevant related works below).\n- Lack of clear discussion of methodological limitations (see also questions below)\n\n\n**Relevant related works:**\n\nHigher-order explanations for DNNs:\n\n- [Tsa18] M. Tsang, D. Cheng and Y. Liu, \"Detecting statistical interactions from neural network weights\", Proc. Int. Conf. Learn. Representations, 2018.\n- [Ebe20] Eberle, O., Büttner, J., Kräutli, F., Müller, K. R., Valleriani, M., & Montavon, G. (2020). “Building and interpreting deep similarity models”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(3), 1149-1161.\n- [Jan20] J. D. Janizek, P. Sturmfels and S. Lee. \"Explaining explanations: Axiomatic feature interactions for deep networks\", CoRR, 2020.\n- [Sch21] Schnake, T., Eberle, O., Lederer, J., Nakajima, S., Schütt, K. T., Müller, K. R., & Montavon, G. (2021). Higher-order explanations of graph neural networks via relevant walks. IEEE transactions on pattern analysis and machine intelligence, 44(11), 7581-7596.\n- [Vas24] A Vasileiou and O Eberle. “Explaining Text Similarity in Transformer Models”, NAACL 2024.\n- [Fum24] Fumagalli, F., Muschalik, M., Kolpaczki, P., Hüllermeier, E., & Hammer, B. (2024). KernelSHAP-IQ: Weighted Least-Square Optimization for Shapley Interactions. ICML 24.\n\nQUESTIONS:\n- Do tokenizers need to be aligned for this to work? It appears that subtokens were aggregated in Figure 3. How was the aggregation performed?\n- In Section 3.3, four LMs were using during training as the source LMs. Why did you decide to take several and how do the results depend on training on less source models, i.e. only one? \n- “This suggests that OLA primarily encodes syntactic structures rather than semantic knowledge.” (l. 326) If I understand correctly, OLA maps do not directly encode representations but relationships of representations. This should be discussed more clearly. What are the limitations of this focus on relatedness over representations?\n\nLIMITATIONS:\nSome specific limitations are discussed in Section 4.2 regarding the experimental results of the TOA experiments, but no general methodological limitations are discussed as part of the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-27"}

comments: SUMMARY:
From the perspective of information flow, this paper decomposes the information flow formed by multi‑head attention (MHA) and residual connections, proposing Order‑Level Attention (OLA). Through a series of qualitative and quantitative experiments, it demonstrates the validity of OLA, reveals its potential as an invariant shared across different large language models (LLMs), and then introduces Transferable OLA Adapter (TOA). Experiments on four basic NLP tasks further verify TOA’s effectiveness and, by extension, the soundness of OLA.

STRENGTHS AND WEAKNESSES:
> Strengths

- The paper offers a novel viewpoint by decomposing the information flow generated by MHA and residual connections, uncovering a common contextual aggregation pattern among LLMs and proposing OLA.

- Experiments are thorough and well designed. Qualitative studies first reveal the similarity of same‑order OLAs (OLAS) across different LLMs for the same text; this is then substantiated through carefully crafted image‑classification and image‑similarity‑retrieval experiments, as well as quantitative analyses of “Relation between OLA and Syntactic Knowledge.”

- The proposed methods are simple yet effective; on quantitative tasks based on image classification and image‑similarity retrieval they substantially outperform a range of baselines.

> Weaknesses

- The TOA method is restricted to syntax‑related tasks and cannot be directly applied to text‑generation or other semantics‑heavy tasks.

QUESTIONS:
1. Have you tried applying TOA to transfer for text‑generation tasks?

2. Why are there no experiments for fourth‑order OLA and above? Is it because higher‑order features are heavily affected by the attention‑sink phenomenon?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates whether different large language models (LLMs) exhibit common patterns in how they aggregate contextual information through attention mechanisms. The authors propose a novel attention decomposition method called Order-Level Attention (OLA), which breaks down cumulative attention rollout into multiple “orders” (e.g., 0th-order: residual path only, 1st-order: single-layer aggregation, and so on). They observe that OLA representations of the same order across different models show a high degree of similarity, a phenomenon they term Order-Level Attention Similarity (OLAS). Building on this insight, the authors introduce the Transferable OLA Adapter (TOA), a training-free adapter mechanism that enables cross-model transfer. Experiments conducted on 12 different models and multiple downstream tasks—including relation extraction, part-of-speech tagging, and named entity recognition—demonstrate the effectiveness of their approach.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- OLA introduces a novel perspective for unified attention representation by decomposing attention rollout into order levels from a path-based view, facilitating interpretability and structural alignment across different language models.\nThe authors measure attention structure similarity using image-based methods and support their OLAS observation with thorough theoretical analysis and empirical validation.\n- TOA leverages attention representation for transfer and offers a simple, training-free mechanism that outperforms baseline models in multiple tasks.\n- The paper covers both theoretical formulation and practical application, showing strong breadth and depth.\n\n**Weaknesses**\n- OLA bears substantial similarity to existing methods like Attention Rollout or path tracing (e.g., Abnar & Zuidema, 2020). Attention Rollout is already a widely used method for quantifying input importance at any layer of encoder-based models like BERT. Its applicability to causal language models (CLMs) remains under-discussed. Furthermore, the proposed order-wise decomposition may amplify the overestimation of distant token importance in long sequences—an issue not addressed in the paper.\n- In the experiments, the authors did not adequately compare against other interpretability methods (e.g., LogitLens, ACDC) or transfer approaches (e.g., LoRA, P-Tuning). The main baseline used is zero-shot, which is a relatively weak comparison. No comparisons are made to representation transfer methods such as hidden-state alignment or DSKD, leaving the true advantage of TOA unclear.\n- The claim of being the “first” to achieve training-free adapter transfer is overstated, as similar ideas have been explored in prior works.\nTheoretical properties, generalization boundaries, and computational efficiency of high-order OLA are not clearly discussed.\n\nQUESTIONS:\n- The current experiments do not clearly establish the advantage boundaries of TOA. Could you add comparisons with other representation transfer methods, such as Representation Editing, Middle-layer Alignment, or Hidden-State Transfer? Also, how does TOA perform relative to trained adapters like LoRA , or Prefix Tuning? Is there a trade-off between accuracy and training cost?\n- Have you evaluated OLA or TOA on models with significantly different architectures, such as encoder-only versus decoder-only Transformers? How well does the method generalize across these divergent settings?\n- How does OLA behave in long-sequence settings? In such cases, what changes (if any) occur in TOA's training complexity, inference latency, and accuracy? Does high-order OLA risk overestimating distant token contributions in longer contexts?\n- The current experiments focus exclusively on tasks suitable for encoder-only models (RE, NER, DP, POS). Can you expand the evaluation to include tasks like text continuation or summarization, which are more natural for causal language models (CLMs), to validate OLA and TOA in that setting?\n- The paper reports the performance differences across different OLA orders (first-order, second-order, and third-order) in Table 2 (retrieval task) and Table 3 (dependency parsing task), with notable variations observed across both CLM and MLM model families. However, the authors do not provide sufficient analysis or discussion to explain these differences.  For example:\n  - In Table 3(a), the Qwen2-1.5B model achieves a UAS/LAS of 63.58/48.24 using first-order OLA on the dependency parsing task, which is significantly higher than the 55.19/36.82 performance obtained with third-order OLA.\n  - Moreover, in the ablation study on data augmentation in Appendix Table 9, the cross-model transfer results vary across different OLA orders, showing inconsistent trends across different models.\n  - This raises the question of whether the choice of OLA order has a complex, model- and task-dependent impact on performance.\n\nLIMITATIONS:\nOLA performs an order-wise decomposition of attention rollout. However, attention rollout itself has been primarily applied in encoder-based models such as BERT, where it serves to quantify input token importance across layers. Its applicability to causal language models (CLMs) remains unclear and is not addressed in the paper. In addition, although the paper presents a wide range of experiments, many of them remain relatively superficial. The baselines used are fairly basic, and several important issues in the experimental data are not thoroughly examined or discussed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the similarity of context aggregation patterns across different language models. The authors introduce Order-Level Attention (OLA) as a method to characterize these patterns. Specifically, the i-th order OLA is defined as the sum of the i-th order terms in the expansion of the attention rollout. The authors demonstrate that OLA exhibits notable similarities across models and is associated with syntactic knowledge. Building on these observations, they propose the Transferable OLA Adapter (TOA), claimed to be the first approach enabling cross-LM adapter transfer without additional tuning. By leveraging OLA as a unified syntactic representation across LMs, the adapters are trained with OLA for downstream tasks. Since OLA appears to be similar across models, the trained adapters can be directly applied to other LMs without further adaptation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1.\tThe proposed TOA enables transfer across different language models without requiring additional training.\n2.\tThe authors employ a range of qualitative and quantitative analyses to examine OLA similarity across LMs.\n3.\tThe experiments are conducted on models of varying sizes and architectures.\n\nWeaknesses\n1.\tThe evaluation is limited to NLP tasks involving syntactic and simple semantic understanding; more complex tasks are not considered.\n2.\tThere is no comparison against related methods, such as Cross-model Control: Improving Multiple Large Language Models in One-time Training.\n3.\tThe approach requires modifying the adapter structure and training separate adapters for different tasks, limiting its universality.\n4.\tThe cross-model generalization capability appears limited; transferring to other models often results in performance degradation exceeding 10 percentage points.\n5.\tSome descriptions lack clarity. For example, Sec 3.2 refers to “zeroing maxima for first-/second-order OLA,” while Appendix B mentions “setting outlier values in the OLA to zero,” creating inconsistencies. Additionally, Section G.1 contains an incomplete phrase: “The diagonal features of …”\n\nQUESTIONS:\n1.\tCan TOA generalize to more challenging tasks that require deeper semantic understanding, such as generative tasks like question answering?\n2.\tI am puzzled as to why 7B-scale instruct models (e.g., Qwen2-7B, LLaMA 3.1-8B) perform poorly on these tasks, given that models of this size have achieved strong results on more difficult mathematical benchmarks. Could the authors provide further analysis or case studies to explain this behavior? How does TOA performance compare to few-shot prompting?\n3.\tWhat is the rationale for employing visual models or image similarity metrics to assess similarity, rather than simpler measures such as vector similarity?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-05"}