Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings
Authors: Stephen Zhang
Mustafa Khan
Vardan Papyan
Venue: NeurIPS 2025 poster ()
Keywords: Attention Sinks
Mechanistic Interpretability
Language Models
Primary area: deep_learning

Abstract: Large language models (LLMs) often concentrate their attention on a few specific tokens referred to as *attention sinks*. Common examples include the first token, a prompt-independent sink, and punctuation tokens, which are prompt-dependent. While the tokens causing the sinks often lack direct semantic meaning, the presence of the sinks is critical for model performance, particularly under model compression and KV-caching. Despite their ubiquity, the function, semantic role, and origin of attention sinks—especially those beyond the first token—remain poorly understood. In this work, we conduct a comprehensive investigation demonstrating that attention sinks: *catch* a sequence of tokens, *tag* them using a common direction in embedding space, and *release* them back into the residual stream, where tokens are later retrieved based on the tags they have acquired. Probing experiments reveal these tags carry semantically meaningful information, such as the truth of a statement. These findings extend to reasoning models, where the mechanism spans more heads and explains greater variance in embeddings, or recent models with query-key normalization, where sinks remain just as prevalent. To encourage future theoretical analysis, we introduce a minimal problem which can be solved through the 'catch, tag, release' mechanism, and where it emerges through training.

Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings
Stephen Zhang, Mustafa Khan, Vardan Papyan
University Toronto, Vector Institute
{stephenn zhang, kwann www..l.ttooooooooo ca, vardan. vardan.papyaanee papyanâttoronto.ca
Abstract
Large language models (LLMs) often concentrate their attention a few specific tokens referred to as attention sinks. Common examples include the first token, a prompt-independeer
sink, and punctuation tokens, which prompt-dependent While the tokens causing the sinks often lack direct semantic meaning, the presence of the sinks is critical for model performance, particularly under model compression and K K--caching. Despite their ubiquity, the function, semantic role, and origin attention einks-lspecially those beyond the first token-remain poorly understood. In this work, we conduct a comprehensive investigation demonstrating that attention sinks: catch sequence of tokens, fag them using common direction in embedding space, and release them back into the residual stream, where tokens are later retrieved based on the tags they have acquired. Probing experiments reveal these tags carry semantically meaningful information, such as the truth of statement. These findings extend reasoning models, where the mechanism spans more heads and explains greater variance in embeddings, or recent models with query- key normalization, where sinks remain just prevalent. To encourage future theoretical analysis, we introduce minimal problem which can be solved through the 'catch, tag, release' mechanism, and where it emerges through training.
Introduction
1.1 'Catch, Tag, Release' in Aquatic Conservation
In marine biology, the 'catch, tag, release' mechanism is a vital tool for tracking fish populations. A fish is caught, fitted with tracking tag encoding critical information, and then released back into the water stream, where can monitored over time. This process enables researchers to understand migration patterns and ecosystem interactions. Remarkably, LLMs exhibit strikingly similar mechanism when processing tokens. To understand why, we first examine a recent discovery in LLM research.
1.2 The Prevalence of Attention Sinks
In many models, tokens often focus disproportionately on a select few positions in the sequence. attention
Xiao al. [024]] identified the first token as a common focal point, referring
sink. Follow-up studies have shown that, beyond this fixed sink, additional ones can emerge based on
the input often appearing on punctuation Yuu al. 2024 Sun al. 2024 Canceddal 2024).
Preserving these sinks has proven critical for retaining model performance in several key areas, including K KV-caching [Xiao al. 2024 Liu al.
2024 Guoo et al.) 2024b Willette al. 2024
quantization (Lin et al. 2024 Son et al. [024], and pruning [Zhang and Papyan 2025a]. Motivated by these findings, researchers have begun investigate the following question.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Pash
C--3
Pank
Ceo3
C1-2
Cr
(a) Catch:
Each box corresponds to different token at the input of the attention layer, whose activation is denoted by e,, and the arrows represent attention interactions. The attention sinks Caink and exinkg catch the attention tokens e2; c...... et-1 1 and e:41, 2.422.... er, respectively. This causes vertical bands to emerge in the attention weights A, as shown in Figure 2a dtn.
sinnk
Allcation
(b) Tag: The left grid shows the attention value matrix V [el:-... ET where activation vectors stacked vertically. The right grid shows the output of the attention layer Z = [Zanky 22; zr] AV, with output vectors &, also stacked vertically. The value vectors of the sinks, 1ask1 and Caing are copied to all tokens that attend to them, thereby tagging them. These tags cause the token representation to cluster based on the sink they attended to, revealed in the PCA plot Figure 2c show such clustering, as shown in Figure 2h
The inputs to the attention layer, prior to the tagging,
(c) Release: Each box corresponds to a different token at the output of the attention layer. The attention outputs Zink. creating common directions in representation space, in the form of the tags Zuinky and Zuinky shared across multiple tokens. These tags cause the token representations cluster in deeper layers, revealed in PCA plot in Figure 2d]
are added to the residual stream
Figure 1: An illustration of the 'catch, tag, release' mechanism.
1.3 Why Do Attention Sinks Emerge?
Existing answers generally fall into three main categories:
To Create Implicit Attention Biases: Attention layers lack explicit bias parameters. Attention
sinks emerge as compensatory mechanisms that artificially introduce such biases, and they be mitigated by incorporating explicit key value bias parameters [Sun al. 2024
Darcet et al. 2024 Gu et al. 2024].
To Turn Off Attention Heads: Attention heads are not needed for certain sequences. Attention sinks emerge as a learned, data-dependent mechanism that effectively disables them by
capturing nearly all the attention [Bondarenko et al. 2023 Guo et al. 2024a].
To Prevent Over-Mixing of Tokens: Transformers are prone excessive token mixing and rank collapse [Geshkovski et al.) 20244 Barbero et al. 2024 20255). First-token attention sinks help mitigate these issues by anchoring the sequence and limiting uncontrolled interaction across positions.
1.4 Open Questions
While each of the three perspectives shine a light on the role of first-token attention sink, they still leave the following questions unanswered:
Q1: Why do attention sinks emerge in later tokens?
None of the perspectives addresses this question. These sinks are sequence-dependent (Yu et al. 2024, Cancedda 2024), contradicting the bias perspective, and the presence of multiple sinks seems unnecessary from both the active-dormant and over-mixing perspective.
Q2: Do the representations of attention sinks despite corresponding to semantically meaning- less tokens like nonteaheien-nnneeeeesssse encode meaningful information?

---

laughed
with
my
cm
acc
hurt
PP
awway
Iears
nesss
nothing
enger
table
(b) PCA imput to the attention layer. Tokens exhibit clustering.
laughed
with
friends
unti
ach
hurt
D
leara
sed
and
said nothing
ammeed
anger on
the
table
(c) PCA on the output of the attention layer. Tokens cluster according to their attended sink: those attending to the first sink are shaded red, while those attending the second sink are shaded green.
laughed with friands unti
ach hurt w
away
tears
and sald nothing
ammed
my
int
anger on the table
(a) Attention Weights. Two atten- tion sinks catch the attention of sub- sequent tokens the sequence.
(d) PCA on the residual stream in a deeper layer. Tagged tokens propagate through the residual stream, clustering in deeper layer based on their previously attended sink. Tokens that attended to the first sink are green, while those attending to the second sink are brown and yellow.
Figure 2: Qualitative analysis of the 'catch, tag, release' mechanism. The second and third subplots use PCA-based coloring of embeddings, described in Section 2 Appendix [A] presents additional measurements across wide range of models, layers, attention heads, and prompts, including chain- of-thought [Wei et al. 2022]. and zero-shot chain-of-thought [Brown et al. 2020
This question remains largely unexplored (Yu et al. 2024], leading to conflicting lines of research,
with some studies exploring how sinks can be preserved Xiao et al. 2024 [Snn et al., 2024], while
others investigating how they be dispersed removed [Sun al. 2024 Yu et al. 2024 Zuhri
et al. 2025
Fu et al. 2025 Kang et al. 2025].
Q3: Are there differences in the number of attention sinks beiween pretrained LLMs and those fine-tuned for reasoning tasks (DeepSeek-Al 2025
If attention sinks play functional role in reasoning, one would expect them to be more prevalent in models optimized for such purposes. Furthermore, the locations of these sinks may align with semantically meaningful boundaries, segmenting content in ways that support structured reasoning.
Q4: How does query-key normalization affect attention sinks?
is conceivable that query-key (QK) normalization (Henry al. 2020) alters the effective tempera- ture of the softmax distribution in attention layers. This, in turn, may smoothen or sharpen attention weights, potentially affecting the formation of attention sinks.
1.5 Contributions
Through an empirical study, we answer the questions posed above and establish the following claims:
A1: Attention sinks implement a 'catch, tag, release' mechanism, the steps of which are detailed in Figure (Sections 2
A2: Probing experiments reveal that the resulting tags are not arbitrary they encode semantically meaningful information, such as the truth value of a statement (Section 4.
A3: Compared pretrained models, DeepSeek-distilled models exhibit the mechanism across more attention heads and explain a greater proportion of variance in the embeddings, indicating stronger and more pronounced instantiation of the mechanism (Section 5
A4: Attention sinks remain prevalent models with QK normalization, despite the normalization explicitly imposed tokens prior to computing attention scores. (Section 66)
To support future theoretical analysis, we introduce minimal problem that is solvable via the explicit use of the 'catch, tag, release" mechanism and demonstrate empirically that the mechanism naturally emerges through standard training.
3

---

2 Visualizing the 'Catch, Tag, Release" Mechanism
This section presents a visual exploration the 'catch, tag, release' mechanism. Figure 2 illustrates representative example selected for clarity, but similar behavior consistently emerges across wide range of models, prompts, layers, and attention heads. To support this generality, we include additional visualizations in Appendix A
Evidence for Catch As illustrated Figure La the evidence of the catch mechanism amounts to showing the existence of an attention sink. We therefore feed prompt to the PHI-3 MEDIUM model [Abdin et al. 2024a], and save the attention weights of layer 3, head and plot them Figure [a The visualization shows that there are two sinks that are catching the attention of subsequent tokens.
Evidence for Tag As shown in Figure lb demonstrating the tagging mechanism requires verifying token clustering based on the attended sink. Following Oquab et al. [2024). we compute the top two three (depending on the setting) principal components of the attention head outputs, where d is the embedding dimension, and project the activations onto this basis. The projected values are then normalized to [0, 255] and mapped to RG or RGB channels. Figure 2c illustrates the results, showing clear grouping of tokens by their attention sinks at the output of the attention head. In contrast, Figure depicts the absence of such grouping at the input the attention head.
covariance matrix of the
Evidence for Release As depicted in Figure Ic] evidence for release amounts to showing that tokens in the residual stream in deeper layers have the same clustering behavior exhibited in an carlier attention head output. We therefore hook the inputs into the feedforward network in layer 17, and apply the same PCA projection and normalization step described earlier. The results, presented in Figure 2d] reveal a similar grouping, providing evidence that the model has utilized the tags generated in the earlier layers cluster the embeddings.
3 Measuring the 'Catch, Tag, Release' Mechanism
We provide quantitative analysis to substantiate the presence of the 'catch, tag, release' mechanism across a wide range of model families, including QWEN 2.5 (Yang al. [024]. Pın13 [Abdin et al.) 2024a], LLAMA-3 [Grattafioni et al.] 2024). and MISTRAL 7B (Jiang et al.) [023]] The analysis is aggregated over 170 prompts collected by Gu et al. [2024), truncated to 150 tokens.
3.1 Identifying Attention Sinks and Their Associated Tags
We leverage the metric proposed by Gu et al. (0024] to identify which tokens are attention sinks. Letting A denote the attention weights (i.e. softmax probabilities) of attention head, token is identified as an attention sink for that head if and only if:
= T
A >6
k=1
where 7' is the length of the prompt and e is a predetermined threshold that is set to 0.2 for all the following experiments
For any token designated as an attention sink, its tag is defined to its value vector extracted from the attention head's 's value matrix: U2 V,, E dhee where dhead is the head dimension.
3.2 Quantifying Variance Explained by Tags
Suppose n tokens are identified as attention sinks. We concatenate their tags into matrix V Vun x and apply Principal Component Analysis (PCA):
V V UDUT.
lug
'A sensitivity analysis of the threshold choice is provided in Appendix

---

where U is the matrix of eigenvectors and D is the diagonal matrix of eigenvalues. The matrix UUT, which has rank n, defines projection onto the subspace spanned by the tags. We take the output of the attention head, given by AV (where A and V denote the attention weights and values, respectively), and project onto this subspace, yielding the:
AAUU LF
Variance Explained
|VV|
This quantity is upper bounded by one with equality only if the attention head's output lies entirely within the tag subspace, providing soft measure of how much of the output explained by the tags.
Figure 3 below shows that 1-2 tags typically explain 20% - 40% of the variance in the outputs of the attention heads, but in many cases up 70%. This provides evidence that the tokens being tagged and that these tags contribute significantly to the tokens' representations.
Number of Sinks
Number of Sinks
Number of Sinks
Number of
2.5
2.5
PH6 Mini
Modium
Phs
LLAM LLIMA
QWEN Family
PHI Family
LLAMA Family
MISTRAL
(a) Attention Sink Counts. Counts computed for each head using Equation and their distribution is
visualized for each model using violin plots.
0.6
0.7
Varience Explained
0.6
Varience Explained
2.5
Variance Explained
0.6
Verience Expleined
0.6
0.5
2.4
0.3
0.4
0.3
0.3
0.2
0.2
0.2
Owen 2.5
Owen 2.5 78 Owen 2.5 148
Ph:3 Mni
Phi Medium
LLaKA LLWAA.8BLLOM... 85LLLMA-3..
Mistral 75 V0.3
QWEN Family
PHI Family
LLAMA Family
MISTRAL
(b) Variance Explained by Tags. The metric is computed for each head using Equation and summary statistics are shown for each model using box plots, Boxes indicate the 25th, 50th, and 75th percentiles, while whiskers represent the 5th and 95th percentiles.
Figure 3: Quantitative analysis of the 'Catch, Tag, Release' mechanism.
4 Investigating the Semantic Role of Tags
The previous section demonstrated that attention sinks tag tokens. This observation prompts deeper question: do the distributed tags carry semantically meaningful information that is being released back into the residual stream as part of the release mechanism? To answer this question, we turn to dataset designed to test semantic encoding.
4.1 Cities Dataset Marks and Tegmark 2024
The dataset consists of prompts involving ([CITY], [COUNTRY]) pairs that render a statement either true or false (see Figure The goal assess whether the activations of individual tokens can be used to classify the True/False label of the statement. focus specifically on the final period token - located at position i, indicated by the arrow in Figure as punctuation tokens frequently cause attention sinks.
The city Tokyo in Japan. This statement is: TRUE. The city Hanoi is in Poland. This statement is: FALSE. The city of [CITY] in [COUNTRY]] This statement is:
Figure 4: cities prompt example.
5

---

4.2
Decomposing Activations into Tag and Non-Tag Components
The decomposition of the activation given by:
A Vk.
Zug
Zno tag
where:
T
T [ause]]
Zag
[ouse]
Ar. VA,
and
20 tag
V,,
and ag was defined in Equation [ This decomposition explicitly expresses the token't 's activations as the linear combination tag and non-tag components.
Constructing Mass-Mean Probes Using Tags
First, we input N+ true prompts and N_ false prompts from the cities dataset to compute the class means:
(i)
(i)
Htag
N+
mgg
Mrag
N_
2,
sETrue
iEFalse
(4)
(6)
Poo lag
2t tag
Muo
t,,0 tag
TET
&alle
Next, we compute similarly the within-class covariances, Eug: two sets of mass-mean probes:
Eno Lag.
These are used to generate
(zg(z) = (2 - ( Ptag))
Ono (2))2)==)))))))))))
(z) =
x tag (o Moo tag
where (-) is the logistic function. This closely follows the experimental setup of [Marks and Tegmark 2024). with the key distinction that we decompose activations into tag and non-tag components, rather than using the full activation directly.
We employ total of 600 prompts from the dataset: 400 of which are utilized to generate the probes and 200 for validation, ensuring each contains an equal number of True and False statements. For each model, specific attention heads are identified where the Bug convey True/False information. Further details, including a taxonomy of which tokens were identified as tags for the probes are provided in Appendix[]
4.4 Comparing Probe Performance
Table summarizes the performance of probes derived from the tag component of the activations aag. the non-tag component 0mo lag- and the full activation Oattivation- The superior performance of Ong confirms that the tags contain semantically meaningful information, while the disparity between &nn and 6no tag demonstrates that the tags distribute information not present in the tokens. Notably, Oug can outperform the full-activation probes, suggesting that the tags can provide denoised representation of the True/False direction.
QWEN 2.5
LLAMA-3
LLAMA-3.1
Probe
3B
7B
14B
8B
8B
Owg 98%
94.5%
99.5%
99.0%
92.5%
0m tag
50.0%
50.0%
50.5%
56.5%
50.0%
0 8activation
50.0%
83.0%
60%
97.0%
86.0%
Table 1: Classification Accuracy of Probes. The probe Ong is computed from the tag component of the activation, Ona tag from the non-tag component, and Dxsivatioo from the full activation.

---

A recent architectural intervention that appears intended to counteract this is QK normalization [Henry et al. 2020], which normalizes query and key activations prior to the computation of attention scores. This normalization aims to reduce token magnitudes, thereby dampening the influence of as Comparing Pretrained and Reasoning Models
DeepSeek-A] [2025] distilled the DEEPSEEK-RI model into two widely used pretrained architectures: LLAMA 3.1 8B and QWEN 2.5 14B. In this section, we compare these reasoning-distilled variants to their original pretrained counterparts to examine how distillation for reasoning affects the emergence of
Pretrained Model
Geasoning-Distilles Variant
LLAMA 3.1 8B
DEEPSEEK-RL LLAMA 8B
QWEN 2.5 14B
DEEPSEEK-RI QWEN 14B
Table 2: Summary of models investigated.
the catch, tag, release mechanism.
Figure pressenss the average number of attention sinks, alongside heat maps of the variance explained by tags across attention heads and layers. Reasoning-distilles models exhibit more sinks, particularly in the case of QWEN, and also feature more attention heads with high variance explained by the tags. This suggests that the 'catch, tag, release" mechanism is more prominent reasoning-distee models than in their pretrained counterparts.
Head Index
Head Index
Head Index
Head Index
1.0
1.0
1.0
1.0
Layer Index
Layer Index
Layer Index
0.8
Danaaza
0.8
0.8
Duttam
0.8
Layer
0.6
0.6
0.6
0.6
Avg. Sinks: 1.993 Pretrained
Avg. Sinks: 2.025 Reasoning
Avg. Sinks: 1.125 Pretrained
Avg. Sinks: 2.196 Reasoning
LLAMA 3.1 8B
QWEN 2.5 14B
Figure 5: *Catch, Tag, Release' in Pretrained vs. Reasoning Models. The figure compares the number attention sinks and the variance explained by tags in sessoning-distille models and their pretrained counterparts, respectively.
6 Analyzing the Impact of Query-Key Normalization on Attention Sinks
Massive outlier activations refer to phe- nomenon in which certain tokens LLMS ex- hibit unusually large activation entries Koval-
Deye t4B dlea
Gentraa
Gemora
OK Norm
eva et al. 2022]] Dettmers et al. 2022 Puccetti
ABl Deeriee Leabte
al. 2022 Hämmerl et al. 2023 Rudman et al.
Drwer 2. Messuum
78
2023 Crabbé et al. 2024]. These tokens tend to
Laase Chear
Drean
dominate attention computations, their large values lead to high inner products with other tokens. This, in turn, draws disproportion- ate share of attention to them, causing them to become allention sinks [Sun al. 2024 Kaul et al. 2024 Guo et al. 2024a].
No OK Norm
Por Mes
1.4
1.8
Average Number of Sinks
Figure 6: Sink Count in Models with QK Norm. For each model, the average number of sinks is computed across all attention heads and layers.
activation outliers. One might therefore expect it to also reduce the number of attention a result, suppress the 'catch, tag, release' mechanism we introduce in this paper.
Figure 6 reports the average number of sinks across all attention heads in models with and without query-key normalization. Surprisingly, the total number of sinks remains similar between the two settings, suggesting that QK normalization does not eliminate sink formation.

---

7 Establishing a Theoretical Foundation for 'Catch, Tag, Release'
In this section, introduce minimal problem that be solved by explicitly leveraging the 'catch, tag. release' mechanism. We further show that this mechanism naturally emerges through optimization.
7.1 Setup
Task:
Given a sequence of T tokens, consisting of numbers, 2, E, separated by special
token:
(1) ,..,, 1,, [SEP] F++1, TT);
the objective is to compute the average of the numbers appearing after the [SEP] To increase the complexity of the task, the position of the [oke], denoted by t, varies across different sequences.
Embeddings:
The number tokens and [SEP] are embedded into:
(x;)ed(x;)
E ?R?
i t
e = Embed [SEP])
8nun
E R²²
Stag
respectively, where Spun: Stag IR are learnable parameters. The first coordinate the embed- dings represents the numbers, while the second coordinate represents the tag. The embeddings are concatenated into a matrix to form:
E= e1
C2
erl
Tx2
Model: The embeddings are passed as input to a two-layer transformer Vaswwnn et al. 2017
H Attentionn E, E, EW +
fe( = Attention h w H HW?),
(4)
parameterized by:
0 = (W.) W W K W 8om: Seag)
where the attention is causal and computes:
Attention Q, K, V) softmax (ootmax(QQ]])) IQ
7.2 Main Result
Theorem 7.1 below captures how the 'eatch, tag, release' mechanism explicily solves the sequence averaging task, with the complete proof provided in Appendix
Theorem 7.1 ('Catch, Tag, Release' Theory)
Assume the learnable parameters, 8, satisfy:
0
0
Snua = O,
W
-1
w
W
=
d
where d> 0,
W =
Then:
T
Stog +o0
fo(x)
Σ
x;
i=t+1
for any I1 T1.... VT E R. The model will have the features for any sequence x:
Catch: The [SEP] forms an attention sink.
Tag: The tokens after the [SEP] are tagged by the [SEP] token's 's values.
Release: The tag is used to identify the tokens that should be averaged.

---

7.3 Emergence of *Catch, Tag, Release' Through Optimization
We train the model on 8,192 sequences for 50 epochs, using AdamW [oshchilov and Hutter, 2019) with learning rate of 5e-2 and weight decay of 1e-3. Depicted in Figure 7 are the components of the 'catch, tag, release mecha- nism that emerge with optimization.
Catch: (SEP) (token
(c) Release: Tagged tokens are avoraged.
forms an atlention sink.
Catch: Figure /a depicts the attention weights of the first layer where the [SEP] forms an at- tention sink.
Tag: Figure 7b visualizes the attention head output, showing that the [SEP] token tagged all the subsequent tokens through their second coordinate, which exhibit significantly larger magnitude.
Tokens
(b) Tag: Tokens after the [SEP] are tagged.
Figure Emergence Through Optimization.
The emergence of the 'catch, tag, release" mecha- nism the theoretical model through optimization. The toke]token occurs at index 5 for the prompt used generate the plots.
depicts the attention weights the second layer, demonstrating how the model only averages the tokens that were tagged, i.e., tokens following [SEP].
Release: Figure 7c
Additional Related Works
Understanding the Origins of Attention Sinks A recent line of investigation considers the role of positional encodings. Guo et al. (2024b) showed that first-token sinks emerge even in the absence of positional encodings, suggesting that their formation is not solely positional artifact. In contrast, other recent work has demonstrated that the varying frequencies RoPE (Su et al.] [021] are exploited by models to produce distinct attention patterns, which may be either positional or semantic in nature [Barbero et al. 2025aa..
Another hypothesis is that these phenomena may be optimizer-induced. Both Kaul et al. 2024 and Guo et al. [0244a show that the Adam optimizer [Kingma and Ba, 2015] leads attention sinks and
outhier features. The former, along with many other works [Hu et al. 2024 Nrusimha et al. 2024 He al. 2024). develop techniques to prevent outlier dimensions from forming, which has been linked to the formation of attention sinks.
Connection to Rank Collapse A closely related phenomenon rank collapse, where token representations progressively lose dimensionality with depth Anagnostidis et al. ) Geshkovski al. 2024),, Baabbero al. (224), Naderi et al. [2025), Kirsanov et al.) [2025] This collapse may itself stem from the same 'catch, tag, release" dynamics attention sinks catch tokens, imprint shared
tags, and release them into the residual stream, where the representations eventually collapse into the
subspace defined by these tags.
9 Conclusion
This work uncovers and formalizes 'catch, tag, release', ubiquitous mechanism in LLMs mediated by attention sinks, demonstrating that sinks are not mere quirks of attention maps, but instead implement functional tagging system that propagates semantically meaningful information across tokens. The mechanism persists across diverse model families, intensifies in models fine-tuned for reasoning, and remains robust even under architectural modifications such as QK normalization. Beyond describing this behavior, we introduce a theoretical construction where the mechanism arises naturally and proves sufficient for solving a well-defined task. This offers foundation for deeper mechanistic investigations into token interactions and the role of implicit memory in transformers.

---

Acknowledgments and Disclosure of Funding
We would like to thank David Glukhov for his helpful feedback and discussion. We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC). This research was supported in part by the Province of Ontario, the Government of Canada through CI- FAR. and industry sponsors of the Vector Institute (www. www.eeetrrinie insti tute. at/partnerships/ current- parttess- This research was also enabled in part by support provided by Com- pute Ontario (https: /wwp://wwww..oo www.ttenntario.cccooooooo ca) and the Digital Research Alliance of Canada (attps:/aallaancecaa) ca).
References
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffiman, Mahoud Khademi, Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan Liu, Mengchen Liu, Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Teodoro Mendes, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez- Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy, Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Yelong Shen, Swadheen Shukla, Xia Song, Masahiro Tanaka, Andrea Tupini, Praneetha Vaddamanu, Chunyu Wang, Guanhua Wang, Lijuan Wang, Shuohang Wang, Xin Wang. Yu Wang, Rachel Ward, Wen Wen, Philipp Witte, Haiping Wu, Xiaoxia Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian Xu, Jilong Xue, Sonali Yadav, Fan Yang, Jianwei Yang, Yifan Yang, Ziyi Yang, Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, and Xiren Zhou. Phi-3 technical report: A highly capable language model locally on your phone, 2024a. URL https://arxiv. org abs/ 2404. 14219
Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, and Yi Zhang. Phi-4 technical report, 2024b. URL http:tt/aa.i.i...oto org/ abs/ 22122 08905
Sotiris Anagnostidis, Luca Biggio, Lorenzo Noci, Antonio Orvicto, Sidak Pal Singh, and Aurelien Lucchi. Signal propagation transformers: Theoretical perspectives and the role of rank collapse. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL FxVH7iToXS
fet/oommmii
Federico Barbero, Andrea Banino, Steven Kapturowski. Dharshan Kumaran, João Guilherme Madeira Araújo, Alex Vitvitskyi, Razvan Pascanu, and Petar Veličković. Transformers need glasses! infor- mation over-squashing in language tasks. In The Thihty- Annual Conference Neural Infor- mation Processing Systems, 2024. URL https:://ppenerr///
Federico Barbero, Alex Vitvitskyi, Christos Perivolaropoulos, Razvan Pascanu, and Petar Veličković. Round and round we go! what makes rotary positional encodings useful? The Thirteenth International Conference on Learning Representations, 2025a. URL https://openrevies: nett/forrum?ie-Gttwllkksss.
Federico Barbero, Álvaro Arroyo, Xiangming Gu, Christos Perivolaropoulos, Michael Bronstein, Petar Veličković, and Razvan Pascanu. Why do lmms attend the first token?, 2025b. URL.
https://arxiv org/ abs/ 02732
10

---

Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. Quantizable transformers: Removing outliers by helping attention heads do nothing. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL htt.itttss//opereviieet wwt.://oouondddddsussusttroeeee
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Ben- jamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, llya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc., 2020. URL https: h/ppo//ee////ooeeengeeeee neurips cc/ pdf
Nicola Cancedda. Spectral filters, dark signals, and attention sinks, 2024. URL. https: //axxiv org//bs/2442.099
Jonathan Crabbé, Pau Rodríguez, Vaishaal Shankar, Luca Zappella, and Arno Blaas, Interpreting clip: Insights on the robustness to imagenet distribution shifts, 2024. httpss/////r org/ abs0 13040.
Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers, In The Twelfth International Conference on Learning Representations, 2024. URL
htttps://opmtt
DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025. Utt.pttps://arxi org/ 2201. 12948
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. GPT3.int8(): 8-bit matrix multiplication for transformers at scale. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors. Advances in Neural Information Processing Systems, 2022. URL httt/forrmmidddXx. GWgBoxaD
Zichuan Fu, Wentao Song, Yejing Wang, Xian Wu, Yefeng Zheng, Yingying Zhang, Derong Xu, Xuetao Wei, Tong Xu, and Xiangyu Zhao. Sliding window attention training for efficient large language models, 2025. URL htllsttts::. //arxiv http://awxw.ooorab//////// org/abs/25022 18845
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanczyk, Sertan Girgin, Nikola Momehev, Matt Hoffman, Shantanu Thakoor, Jean-Bastien Grill, Behnam Neyshabur, Olivier Bachem, Alanna Walton, Aliaksei Severyn, Alicia Parrish, Aliya Ahmad, Allen Hutchison, Alvin Abdagic, Amanda Carl, Amy Shen, Andy Brock, Andy Coenen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Bo Wu, Brandon Royal, Charlie Chen, Chintu Kumar, Chris Perry, Chris Welty, Christopher A. Choquette- Choo, Danila Sinopalnikov, David Weinberger, Dimple Vijaykumar, Dominika Rogozińska, Dustin Herbison, Elisa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan Senter, Evgenii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna Klimezak-Plucine Harsh Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson, Jeff Stanway, Jetha Chan, Jin Peng Zhou, Joana Carrasqueira, Joana Iljazi, Jocelyn Becker, Joe Fernande7, Joost van Amersfoort, Josh Gordon, Josh Lipschultz, Josh Newlan, Ju yeong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katic Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sodhia, Kish Greene, Lars Lowe Sjoesund, Lauren Usui, Laurent Sifre. Lena Heuermann, Leticia Lago, Lilly McNealus, Livio Baldini Soares, Logan Kilpatrick, Lucas Dixon, Luciano Martins, Machel Reid, Manvinder Singh, Mark Iverson, Martin Görner, Mat Velloso, Mateo Wirth, Matt Davidow, Matt Miller, Matthew Rahtz, Matthew Watson, Meg Risdal, Mehran Kazemi, Michael Moynihan, Ming Zhang, Minsuk Kahng, Minwoo Park, Mofi Rahman, Mohit Khatwani, Natalie Dao, Nenshad Bardoliwalla, Nesh Devanathan, Neta Dumai, Nilay Chauhan, Oscar Wahltinez, Pankil Botarda, Parker Barnes, Paul Barham, Paul Michel, Pengchong Jin, Petko Georgiev, Phil Culliton, Pradeep Kuppala, Ramona Comanescu, Ramona Merhej, Reena Jana, Reza Ardeshir Rokni, Rishabh Agarwal, Ryan Mullins, Samanch Saadat, Sara Mc Carthy, Sarah Cogan, Sarah Perrin, Sébastien
Harleen Batra,
11

---

M. R. Arnold, Sebastian Krause, Shengyang Dai, Shruti Garg, Shruti Sheth, Sue Ronstrom, Susan Chan, Timothy Jordan, Ting Yu, Tom Eccles, Tom Hennigan, Tomas Kocisky, Tulsee Doshi, Vihan Jain, Vikas Yadav, Vilobh Meshram, Vishal Dharmadhikari, Warren Barkley, Wei Wei, Wenming Ye, Woohyun Han, Woosuk Kwon, Xiang Xu, The Shen, Zhitao Gong, Zichuan Wei, Victor Cotruta, Phoebe Kirk, Anand Rao, Minh Giang, Ludovic Peran, Tris Warkentin, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, D. Sculley, Jeanine Banks, Anca Dragan, Slav Petrov, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Sebastian Borgeaud, Noah Fiedel, Armand Joulin, Kathleen Kenealy, Robert Dadashi, and Alek Andreev. Gemma 2: Improving open language models at a practical size, 2024. URL https://arxiv. http:/aaxxvv...ggaaa/ abs/ ab/p//08.244.....// 2408 00118
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot. Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, Iain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan- Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky. Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng. Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang. Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlic Chen, Charline Le Lan, Christopher A. Choquette- Choo, CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivakumar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eugene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimezak- Pluciíska, Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadie, Jetha Chan, Joe Stanton, John Wieting, Jonathan Lai, Jordi Orbay, Joseph Fernandez, Josh Newlan, Ju yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Nilay Chauhan, Noveen Sachdeva, Oskar Bunyan. Pankil Botarda, Paul Caron. Paul Kishan Rubenstein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, pingmei Xu, Piotr Stanczyk, Pouya Tafti, Rakesh Shivanna. Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq Iqbal, Shashir Reddy, Shruti Sheth, Simm Pöder, Sijal Bhatnagar, Sindhu Raghuram Panyam, Sivan Eiger, Susan Zhang, Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evei, Vedant Misra, Vincent Roseberry, Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff' Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Jean-Baptiste Alayrac, Rohan Anil, Dmitry, Lepikhin, Sebastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and Léonard Hussenot. Gemma 3 technical report, 2025. URL /ttps::// org/ abs habb//550... 19786
GeoNames. GeoNames.org: A geographical database.
org/ 2025.
Accessed: 2025-05-15.
Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet. The emergence of clusters in self-attention dynamics, 2024. URL httttttt////rtti org/ 2005be03330055555..5.
Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet. A mathematical perspec- tive on transformers, 2025. URL httLsttss///arxioorrg wwtp://..ww.......r...... 10794
Aaron Grautafiori, Abhinav Jauhri Abhimanyu Dubey,, et al. The llama 3 herd of models, 2024. URI. https://arxiv. wrw./oog//////.....o..oooo. abs/ 21783
12

---

Xiangming Gu, Tianyu Pang, Chao Du, Qian Liu, Fengzhuo Zhang, Cunxiao Du, Ye Wang, and Min Lin. When attention sink emerges in language models: An empirical view. arXiv preprint arXiv:2410. 10781, 2024.
Tanyu Guo, Druv Pai, Yu Bai, Jiantao Jiao, Michael Jordan, and Song Mei. Active-dormant attention heads: Mechanistically demystifying extreme-token phenomena in Ilms, 2024a. URL
https://arxiv.. org/ hogg/abs/2410 13835
Zhiyu Guo, Hidetaka Kamigaito, and Taro Watanabe. Attention score is not all you need for token importance indicator in KV cache reduction: Value also matters. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21 158-21 166,, Miami, Florida, USA, November 2024b, Association for Computational Linguisties. doi: 1...1865////2222mmmlm.pp 1178. URL
www/2024........o. emn smmlp-main 1178/
Bobby He, Lorenzo Noci, Daniele Paliotta, Imanol Schlag, and Thomas Hofmann. Understanding and minimising outlier features in transformer training. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL htt.:tttt://ppereeeet mpJQ6qgSbbg
net/ ftt//forrm?dd
Alex Henry, Prudhvi Raj Dachapally, Shubham Shantaram Pawar, and Yuxuan Chen. Query-key normalization for transformers. CoRR, abs/2010.04245, 2020. |RLttts:://axxi orgg abs/
2010.04245
Jerry Yao-Chieh Hu, Pei-Hsuan Chang, Haozheng Luo, Hong- Yu Chen, Weijian Li, Wei-Po Wang, and Han Liu. Outlier-effiiient hopfield layers for large transformer-based models. In Forty-first International Conference on Machine Learning, 2024. URLLhttps://ppenrviiee wwt.wiwwwwwwee.uuniiiea.m
id=kL
Katharina Hämmerl, Alina Fastowski, Jindfich Libovický, and Alexander Fraser. Exploring anisotropy and outliers in multilingual language models for cross-lingual semantic sentence similarity, 2023. URL btttps:/arxiv.orgg wtw.g/////////oo////////// 2000/00558000
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Ana Lachaux. Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang. Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URLhttpps: //attis///rxiv /arxiv orgg 06825
Seil Kang, Jinyeong Kim, Junhyeok Kim, and Seong Jae Hwang. See what you are told: Visual attention sink in large multimodal models. In The Thirteenth International Conference on Learning Representations, 2025. URL httph: ntttss//opperevieee htt.
Prannay Kaul, Chengcheng Ma, Ismail Flezi, and Jiankang Deng. From attention to activation: Unravelling the enigmas of large language models, 2024. URL htl.itttts://axxi.ooggottt 17174
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL /ttt://arxiv.. www.ooogg/////oo.eeooeeoe waws//41.... 1988
Artem Kirsanov, Chi-Ning Chou, Kyunghyun Cho, and Yeon Chung. The geometry of prompting: Unveiling distinct mechanisms of task adaptation in language models, 2025. URL https://
arxiv. wwwo/o///aao.e abs/ 08009
Olga Kovaleva, Saurabh Kulshreshtha, Anna Rogers, and Anna Rumshisky. BERT busters: Outlier dimensions that disrupt transformers. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3392-3405, Online, August 2021. A Association for Computational Linguistics. doi: 18653vv//v1/ 1021..indings ain.iings-aal.30.. URL https: http:///aaaaaahhhooooooo org/ 2021. findingsn-aan acl ac1.300/
13

---

Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-awar weight quantization for Ilm compression and acceleration. In MrSys, 2024.
Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. Kivi: A tuning-free asymmetric 2bit quantization for kv cache. arXiv preprint arXiv:2402.02750, 2024.
Iyya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL
Bkg6RicqY7
Mehdi Makni, Kayhan Behdin, Zheng Xu, Natalia Ponomareva, and Rahul Mazumder. A unified framework for sparse plus low-rank matrix decomposition for LLMs. In The Second Conference on Parsimony and Learning (Proceedings Track), 2025. URL htttp:///ppeneviiet wtwpeeeiiiiieeeeoounnn
id=hyN755AJTI
Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In First Conference on Language Modeling, 2024. URL htttps::/openrreiieet http:/ppaararaeeeeeee/oooooaaaaa
Alireza AlirezaNaddri. Thiziri Nait Saada, and Jared Tanner. Mind the gap: a spectral analysis of rank collapse and signal propagation in attention layers, 2025. URL https://arxiv, /wwpvv..ivvv.. org/ abs/ 2abb//210... 2410. 07799
Aniruddha Nrusimha, Mayank Mishra, Naigang Wang, Dan Alistarh, Rameswar Panda, and Yoon Kim. Mitigating the impact of outlier channels for language model quantization with activation regularization, 2024. URL https:/::://// /ttp://aaivvvv... org/ abs /404. 03605
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Mare Szafraniec. Vasil Khali- dov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po- Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features with- out supervision. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL Featured Certification.
httpps://penrrevie mett/forum?iddaasSuteaa
Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, and Samuel Bowman. QuALITY: Question answering with long input texts, yes! In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5336-5358, Seattle, United States, July 2022. Association for Computational Linguistics. URL
http://aaaaanttoogoyyy org/2022. 2022. aaaanclmmai.... 391
Giovanni Puccetti, Anna Rogers, Aleksandr Drozd, and Felice Dell' Orletta. Outlier dimensions that disrupt transformers are driven by frequency. In Findings of the Association for Computational Linguistics: EMNLP 2022, page 1286-1304. Association for Computational Linguistics, 2022. doi: 10.18653/w1/202.. http://w//222...nnnggeeemme findings-emnll.93. URL http: //dx doi org 10. 18653/v1/2022 findings- - emmlp. 93
William Rudman, Catherine Chen, and Carsten Eickhoff. Outlier dimensions encode task specific knowledge. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 1.596-14605, Singapore, December 2023. Association for Computational Linguistics. doi: 118865//1/222..mm// main. 901. URL /ttLLtttts//aaaaattollloo hwwpetthooogoooooogr/eoo org/ ewnlp-main.nneeemeeemmmmmmaaaaem 901/
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng. and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1642, Seattle, Washington, USA, October 2013. Association for Computational Linguistics. URL ponttps://wwrrr www.ww/wwwwww....... wtt.:///wb.rrr/.atttoogg////---/
Seungwoo Son, Wonpyo Park, Woohyun Han, Kyuyeun Kim, and Jaeho Lee. Prefixing attention sinks can mitigate activation outliers for large language model quantization, 2024. URL https
// //arxiv.. org/ /bbs/246.... 12016
14

---

Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. CoRR, abs/2104.09864, 2021. URLjatpss//axxivorrg
Nishant Subramani, Nivedita Suresh, and Matthew Peters, Extracting latent steering vectors from pretrained language models. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Findings of the Association for Computational Linguistics: ACL 2022, pages 566-581, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.1866.60 18653/1112022.. acl.ng.-acll8.. URL
wwwpgggg..rrrggeeeee org/2022.cc. findings- acl. ac1.48/
Mingjie Sun, Xinlei Chen, J Zico Kolter, and Zhuang Liu. Massive activations in large language models. In First Conference on Language Modeling, 2024. URL https://openrevier net/ forum?id=F/ahbfitt
Alexander Matt Turnrr, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmi.. Steering language models with activation engineering, 2025. URL
htttps://openrevieu www.:/eee/eeeeeee htt///oormmmmdXPaeaee
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, edi- tors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https::/proccedd hewpeeeeeddinnnnnnnn..s.....pep neurips. cc/ oc/paper__. fopperffllst files/ wailes/papele pppp:///01///i////// pdf
Haoran Wang and Kai Shu. Trojan activation attack: Red-teaming large language models using activation steering for safety-alinmment 2024. URL /Rttss/::///ar/t wttpeiii.rrg/.oooooo... org/ abss/2311. 09433
Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang. Xionghui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song. Bowen Yu, Gao Huang, and Junyang Lin. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for IIm reasoning. 2025. URL URL. htttps://aaxi:i wwwrxiv.org//ee orgg 01939.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https:: net/ wttp://wneeeoooomomiia forum?id ViQQMeSB_]]
Jeffrey Willette. Heejun Lee, Youngwan Lee, Myeongjae Jeon, and Sung Ju Hwang. Training- free exponential extension of sliding window context with cascading kv cache. arXiv preprint arXiv:2406. 17808, 2024.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the-ar natural language processing, In Qun Liu and David Schlangen, editors, Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online, October 2020. Association for Computational Linguistics. doi: 11853//1//0200 http://000.mmmmddmmmm....e URL https: http://aclantthhn www.aaahllnnooooaaaa org/ 2020. emnlp-denme
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. In The Twelfth International Conference on Learning Representations, 2024. URL /ttppptts///ppeereeii wtt..eeeeooorrradnNNTesSSS..
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412. 15115, 2024.
15

---

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang. Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang. Jianxin Yang. Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin,, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng. Mei Li, Mingfeng Xue, Mingze Li,, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL sttps://arxie org/ ab0///////././//////////////// 093888
Itay Yona, Ilia Shumailov, Jamie Hayes, and Yossi Gandelsman. Interpreting the repeated token phenomenon in large language models. In Forty-second International Conference on Machine Learning, 2025, URL https://ppeerreiee htw.:ppaaeeeeieeeeeeeeooromemmme http//forumm..daWWcchheeeee
Zhongzhi Yu, Zheng Wang. Yonggan Fu, Huihong Shi, Khalid Shaikh, and Yingyan Celine Lin. Unveiling and harnessing hidden attention sinks: Enhancing large language models without training through attention calibration. Forty-first International Conference on Machine Learning, 2024.
URL httptttst//:oorrviaiiii http:/ewwwweeeeeeeeeeeeeeeeeeeee wwt.neet/oooooe
Stephen Zhang and Vardan Papyan. Low-rank is required for pruning L.MMs. In Sparsity in LLMSs (SLLM); Deep Dive into Mixture of Experts, Quantization, Hardware, and Inference, 2025a. URL
https://openreevixx wwt.:t/orrunnddddooooo......
Stephen Zhang and Vardan Papyan. OATS: Outlier-aware pruning through sparse and low rank decomposition. In The Thirteenth International Conference on Learning Representations, 2025b. URL httttp::///peereeieet
Zayd M. Zuhri, Erland Hilman Fuadi, and Alham Fikri Aji. Softpick: No attention sink, no massive activations with rectified softmax. 2025. URL https: //arxiv...iia/a/iaa org/ h/////b///// aa////bb//254.... 2 20966
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Section 1.3 highlights all claims made in the abstract along with references to sections where the claims are validated.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of work are detailed in Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Justification: Theorem 7.1 is stated with the full set of assumptions and the proof is provided in full in Appendix Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide all code needed reproduce the results and visualizations pre- sented in the paper in the supplementary material. Furthermore, we list the parameters of our experiments in Appendices) G
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code is all provided in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips.co. ccippi.cc guwdes/ddeeeee CooeeSumeeeeeeeeeeeeee ssi onyylicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: All training and test details are provided for our probing experiments in Section 4.4 and the training of our theoretical model in Section 7.3 and Appendix G
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Figure 3bb demonstrating the variance explained by tags includes error bars that are explained in the figure's 's caption. Additionally, Figure Ga plots the entire distribution of attention sinks across various models' attention heads.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
19

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Details on the GPUs utilized, compute time including for preliminary experiments presented in the paper reported in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttp::/neeuippit http:///ww.p..pp.ao.....ccaaaaaa
Answer: [Yes]
Justification: The authors have reviewed and verified that this paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA
that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
Justification: The research conducted in this paper is foundational in nature and explains a mechanism already present in many LLMs. It not directly related to any particular application.
20

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., dep.oymmly of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: All data and models utilized are already publicized and do not contain any information that is at high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: A comprehensive list of all the models utilized, along with appropriate citation and how they were accessed, is provided in Appendix M as well throughout the main paper. The cities dataset is created utilizing data provided by GeoNames under a CC-BY 4.0 license.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
21

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The code provided in the supplementary material includes documentation and contains README. detailing how to run the experiments presented in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor human subjects.
Guidelines:
22

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question:
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: The experiments that involved LLMs are fully described in Sections We also provide code that reproduces the results.
and O
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy hbtppp///earrrpsscccooooeeeeeeee for what should or should not be described.
23

---

A Additional Visualizations
In this section we provide further visualizations the one presented in Section across variety of prompts and models.
Layer 3, Head 30
+SP
John
met
Mary
the
park
She
smileed
pweed
basc
Theyy
ta kaik
about
(b) Input embedding visualization at layer 3.
APP
John mel
Mary
the park
She
pmiled
Ho
pved
back
They
talked about
(c) Attention head output at layer 3, head 30.
JOR
med
Mary
park
She
ambledd
baack
Theyy
balked
(d) PCA on residual stream at layer 18.
ypeg paka
(a) Attention prohabilitics at layer 3,
head 30.
Figure 8: Visualization of the 'catch, tag, release" mechanism on a sample referential prompt on the PHI1- MEDIUM model.
Layer Head
The derivative of What the derivative of
(b) Input embedding visualization at layer 2. The derivative of
What is the derivative of (c) Attention head output at layer 2, head 21. What derivative (d) PCA residual stream layer 38.
derivative
The derivator
10
What the derivative
(a) Attention probabilities at layer 2,
head 21.
Figure 9: Visualization of the 'catch, tag, release' mechanism sample one-shot learning prompt the QWEN -2B--NSTRUCT model.
24

---

Leyer
derivalive
(b) Input embedding visualization at layer
Might____stttetss sentenci
The dentoalxa
(Whrd
detivativo
(c) Attention head output at layer 3, head 24.
denvatto
Wher
ceetvvtttve
(d) PCA on residual stream layer 14.
<[jbegin _ot senten The
derivali
Whin th Corrooann Becrooo
(a) Attention probabilities at layer 3, head 24.
Figure 10: Visualization of the 'catch, tag, release' mechanism on a sample one-shot learning prompt on the
Ine
Iallownng
and Determann
LW
Pran
Ch
Ch
ysuh
ard
yOu
SOLE
CTT
road
you
Shut
my
Uh
And VCU
CISLY
migh
pours
Righa
WHY
0:0h
roatico
TTS
TuuDbond
MIDE
CUICN
CAUDROY
LnTi
brought
ond
home
ona
Hyp
thesis
Tuscandd
WAS
SUOT
sudker
Tor animals
Answel
(b) Input embedding visualization at layer
Read
the
Iollwing
parograph
and Cenernano The
hypothesee
Irue
From
pewe
every
Ire STA
you
my
And you 0o creny
miniing
might be yours
Rggal
3am
mocken
my
hausbard
wan
ruch
Lm
brought
one
botra
one night
husbann
wis
sushh
arimats
Attention head output layer 2, head 16.
Reed
losinn
Oetarrno
Impottwos
Prem
yush
and
every
500
tw
road you
say
(a) Attention probabilities layer head
001
Un
And
crizy
miininn
6 mani
yours
Right
WH1
015
rosioon
husband
ruth
pucker
uni tor
onimalo
brougni
one
home
cm
hunbard
mucher
for intimata
Anewet
(d) PCA on residual stream at layer
Figure 11: Visualization of the 'catch, tag, release' mechanism longer prompt the QWEN
111--IIIIBUUEE model.
25

---

Inpuu
L
Lace
Lnbo
(b) Input embedding visualization at layer 1.
be
2012
200
Late
Lab
Autoriio head outpun at layer L. hoad
(c)
12.
FEB
LAB
T
28
Labo
(d) PCA residual streamt at layer 7.
(a) Attention probabilitee at layer
1. head 12.
Tigure S 12: Visualization of the 'catch, tag. release' picchaniin on sequence averaging prompu on
the PBERSSERR R i 88 model.
VOL
Label
Input
Label
Inpui
(b) Input Sombodddiiggg Stuaaiizaiioo at layer 2.
Impul
Impul
(c) Aicezitoo head oarpurt at layer 2. head 39.
TD
2
80
Labe
Lacol Impar
Tb
0
(a) Attentionn potbabblliii at layer
(d) PCA on residual stream al layer 37.
2. head 39.
Figure 13: Seamaiizzttio of the 'eatch, tag. release mecchaniin on a sqguance averaging brrggingg promp on the www.aaa..... Saraaaaaaaiii model!
Sooo 5. Food
AD
more
Hoow
morr
Loi
brink
eibbeddae embedding Eieualizzztii at layer's 3. pink
bb) Input
Roso Gol
nave
(c) Alcantio head output at Layer 3, head 6. Eain
BOG
Now
cel
(d) PCA residual stream at layer 3.
layer (a) Mitention 3. head 6.
cobbabbliie at layer 
Figure 14: Visualizatioon of the "catch, tag. relense geccaaiiinn on a Zeno-Shan Chain of Though
(CoT) prompe on the P1l--3 MEbiUN model.
26

---

Now
hoxe
(b) Input embedding visualization at layer 3.
and bought
more
Now
have
app
lss
and
miDe
How
do
have
(c) Attention head output at layer 3, head 30. have
ongee
and
bough
1-00-S --
Now
and
HOw many
do
have
(d) PCA on residual stream layer 16.
(a) Attention probabilities layer head 30.
Figure 15: Visualization of the 'catch, tag, release' mechanism one-shot math prompt the
PHI-3 MEDIUM model.
Layer 5, Head 16
<S>
derivative
What the derivative
(b) Input embedding visualization at layer 5.
The derivative
What derivative
(c) Attention head output at layer 5, head 16.
derivative
What derivative
(d) PCA on residual stream layer 21.
The derivative
00
What is
derivative
(a) Attention probabilities at layer
5. head 16.
Figure 16: Visualization of the 'catch, tag, release' mechanism Chain of Thought (CoT) prompt on the PII133 MEDIUM model.
27

---

Layer 5, Head 14
What
derivative
think
step
by
step
(b) Input embedding visualization at layer
What
derivative
think step step
(c) Attention head output at layer 5, head 14.
What the derivative
think step
Whet
is the derivatio
of
La shin
dags by dare
(d) PCA on residual stream layer
(a) Attention prohahilities at layer 5,
head 14.
Layer 39, Head
What
derivative
step
(1) PCA on residual stream layer
What
derivative
of
Les
Ma
daps ssie
(e) Attention probabilities at layer
39, head 24.
Figure 17: Visualization of the 'catch, tag. release' mechanism on Chain of Thought (CoT) prompt on the PHI-3 MEDIUM model. Later layers reveal an attention sink forming on "Let's think step by step". with the output embedding retaining features from tagged value embeddings.
28

---

B Taxonomy of Sinks
To explore which tokens commonly exhibit sinks, count the frequency of tokens that appear as sinks and measure the average explained variance of each token when it appears as sink. To this end, we analyzed 200 prompls consisting of 1024 tokens from the QuALITY dataset [Pang al. 2022]. using both the base and reasoning-tune versions of the QWEN 14B model. The results are presented in Table below.
Model
Token
Avg. Variance Explained
Frequency
[FIRST]
0.2798
328 487
0,2503
5323
0.1998
3807
Gthe
0.2334
3333
GI
0.2279
3283
QWEN 2.5 14B
G
0,2440
3203
Gto
0.1902
2603
Gbegan
0.2032
2592
GStark
0.1804
2512
0.2156
2253
[FIRST]
0.2994
307 600
THE
0.3087
50 248
The
0.3005
47 535
Doctor
0.3032
26 928
SP
0.2967
24 718
DEEPSEEK QWEN 14B
MON
0.3144
23 680
CAP
0.3020
22 832
IT
0,2880
22 416
GR
0.3019
915
IMAGE
0.2961
20 272
Table 3: The ten most frequent attention sink tokens for the QWEN 2.5 14B and DEEPSEEK QWEN 14B models. For each token, report both its frequency of occurrence average variance it explains. The token [FIRST) denotes the first token of a sequence (which may vary across sequences).
attention sink and the
B.1 Discussion: Impact of Reasoning Distillation
In base models such as QWEN 2.5 14B, sink formation predominantly occurs around function words (e.g., Gthe, Gto, GI) and punctuation (such as or .), likely due to their high frequency and syntactic roles. In contrast, the DEEPSEEK QWEN 14B reasoning-tune model forms attention sinks around semantically significant or task strucstrut tokens, such as:
1. IMAGE suggests segmentation for multimodal inputs, grouping the token with subsequent image-related descriptions.
2. Doctor likely marks named entities relevant for reasoning over domain-specifir content.
3. Tokens like MON, CAP, IT, and GR are plausible abbreviations or categorical labels (e.g. weekdays, captions, grades) that suggest segmentation for structured data.
4. SP may represent speaker or section markers, important in multi-step or dialog-based reasoning.
These sink tokens not only appear with high frequency also explain substantial portion of variance, indicating their importance in structuring the model's internal representation. The presence of such tokens, absent in the base model's sink list, suggests that reasoning fine-tuning reorients the attention sink mechanism from syntactic attractors to semantically meaningful or task-specific units.
29

---

1.0
10
Quary Index
4.0
6.4
0.4
20-
0.0
(a) Layer 1, Head
(b) Layer 1, Head
0.
08
Query rdlex
Qucry
0.4
0.2
02
Koy Indixx
Key
(c) Layer 2, Head
(d) Layer 2, Head
Figure 18: Attention probability maps for each head and layer. Each attention head is dedicated to solving specific portion of the task, which the first layer, requires the different attention heads to form distinct sinks and tags that the model uses constructively to create the desired probability distributions the second layer.
C Extending the Theoretical Model
The theoretical model presented in Section shows why single tag and single attention head suffice in the basic case. A natural extension, however, is to consider model where:
1. multiple attention sinks required, and
2. multiple attention heads are employed.
We generalize theoretical framework to capture these aspects and provide empirical evidence that this extension indeed gives rise to the aforementioned features the solution.
Task: We modified the averaging task by introducing second special token, [SEP2], which is inserted at a random position following the [SEP] token. The task to compute the sum of two quantities:
the average of the numbers that appear after [SEP], and
2. the average of the numbers that appear between [SEP] and [SEP2].
Embeddings: We extended the embedding space by one dimension, such that number tokens are embedded as [z, -1, -1. The embeddings for [SEP] and [SEP2] optimized during training.
Model: used a two-layer attention-only transformer with:
Two attention heads per layer with a head dimension of 3,
A learned output projection Wo in each layer mapping the concatenated heads back to the embedding space.
Depicted in Figure T8 above the visualizations of the attention weights for the different attention heads for given sequence where [SEP] lands on token 3 and [SEP2] lands token 12.
30

---

D Separability of Tags
central question in our analysis is whether the model organizes tags within a well-structured and separable subspace of its representation space. If such subspace exists, it would suggest that tags interact in systematic and potentially predictable ways, rather than interfering with other embedding dimensions. To investigate this, address two questions:
1. How do different tags interact with one another and do their effects combine additively or interfere with one another?
2. How cleanly can the tag-related components disentangled from the underlying embed- dings?
D.1 Interaction Between Tags
We measure the cosine similarity between tags across all layers and heads by mapping the values of the tag tokens back into the residual stream.
For given attention head h (1...., H), denote the value of a tag token to be Ulag E TR
The output projection matrix Wo R x (Ir acts on the concatenation of all heads. To place Vng in the correct block of this concatenated space, we embed via the Kronecker product with standard basis vector:
Utng E E H.d
Ung
eb
where E Rd is the one-hot vector with in the h-th position. This construction is equivalent to inserting Ving the block corresponding to head h and padding with zeros elsewhere.
We then map the padded vector back into the residual stream as:
Oug
Wo Bug
Finally, given two tags a, (possibly from different layers heads), their similarity in the residual stream is defined as the cosine similarity:
(0a, 0b)
cos( 06) =
|0 |6
The results are shown in Figure 19 below.
D.2 Interaction Between Tags and Embeddings
We measure the average cosine similarity between the tag component of each token's representation and the embeddings to which it added in the residual stream.
For token position and each head h = I,..., H, define the per-head tag component:
(h)
Σ 1 (a
A (h) (h)
Ztagt
Concatenate the per-head tag components into the head-concatenated vector:
(1)
Zeag.t
Zeag,t
E Hd.
(I)
Ztag.1
Map this concatenated vector back into the residual stream using the output projection Wo:
Stag.
Wo Ætag,t Rddddd
31

---

-30
JaPA-185
LIMb-1185
Codias Stenlatty
Denzuwak Owan
030 Sirdarity
Figure 19: Cosine Similarity Between Tags. Histograms of the cosine similarity between distinct tags across all layers and heads. The tags typically exhibit very small cosine similarity implying that they are close to orthogonal and do interfere with one another.
32

---

The cosine similarity between the mapped tag component and the attention layer input æ (i.e. the embedding to which the attention output added)
(Ztag,t, Wt4
cos (2rg,,, (i)
Smgg.t
Table 4 below depicts the average of the cosine similarities over 200 prompts consisting of 1024 tokens for various models.
Model
Cosine Similarity
DEEPSEEK LLAMA 8B
-0.0674
DEEPSEEK QWEN 7B
00.1651
DEEPSEEK QWEN 14B
-0.1449
LLAMA 3 LLAMA 3.1 8B
0.0238
0.0099
QWEN 3 8B
-0.0868
QWEN 3 14B
-0. -0.1290
QWEN 2.5 8B
-0. 17.1738
QWEN 2.5
-0. 1581
QWEN 2.5 14B
-0. 1444
Table 4: Cosine similarity between tags and embeddings. The average cosine similarity is low across all models, especially in the LLaMA models, which suggests that the tag subspace remains largely orthogonal to the native token representations.
E
Proof of Theorem 7.1
We will use the auxiliary notation:
A softmax(ERt (EET)
A = softmax ( W
E.1 Proof of Theorem, Part 'Catch, Tag, Release'
The first part of the proof focuses on the first attention layer.
Catch Mechanism The attention weight when token x,, for t attending [SEP] is:
exp(e. et)
A = Σl exp(e. ex)
exp(e, ex)
exp(e: ex) + Σi Ek=1, exp(e. ex)
exp(I;9nun + Stag)
exp(x;5mun + Stag) Ei=Lkee exp(x,0x + 1) exp(Stag)
Rewr 100
1.
exp(Stag) Ek= explxx04 +1 +1)
Furthermore, for > and j # t:
exp(1 + x,JT,)
E E -
(5)
j exp($tag)
E exp(1 + 2731)) kfj
33

---

and therefore:
Scae 000
A
0 for j
Thus, [SEP] acts as an attention sink, where all tokens u; for i > 6, as well as the [SEP] token itself, attend exclusively to the token. As result, the attention of all tokens for i >> / and the [SEP] token have been caught.
Tag Mechanism
The attention output of the i-th token for i 2 is:
Attention(e E, EW,)
Σ f=1
e
- WV +
Σ A w7
(6)
j=1,i
000
- Stag] +
Σ e
W
10
00].
/=1,e11
Recall that the second coordinate of the embeddings represents the tag. The limit above implies that a tag has been created for all tokens F,, for
After adding the tag to the residual stream, we obtain for i > t:
h, = C + Attention(e) E, )W Blog +00
The above implies that all tokens, I₁, for i have now been tagged.
Release Mechanism The tagged tokens have now been released into the residual stream. As we will show in the next subsection, the tags will be leveraged by the second attention layer generate the desired averaging mechanism.
Proof of Theorem, Part 2: Leveraging the Tags E.2
The second part of the proof focuses on the second attention layer.
The attention weight when token Ir is attending to token zy is given by:
exp(h, W W hy)
A
Ek=1 ( W W wki
Dividing the numerator and denominator by exp(h., W W hT), the expression becomes:
exp(h W W h W w
A E cxp(h, W W2 h2 h W h2r
exp(h W W (h; hT))
(8)
Σ exp(h, h W W (he hT))
Consider the term inside the exponent:
h W (h, - h
There are three cases for depending on whether less than, equal to, or greater than In the next subsection, we prove that
Case 1: For j < (non-tagged tokens),
Beag
exp(hi w3 w (h, hr))
0.
34

---

T-tt j > t Case 2: For j = & (sink token),
+00
exp(h 1p - -hr))
0.
Case 3: For j > (tagged tokens),
exp w W (h, hr))
1.
Combining all three cases together with Equation
8 leads
0,
j <
lim Btag 700
A
0,
j =
The above shows how the tags have been leveraged to arrive at uniform distribution over the desired
tokens. Then using this, Equation (7) and that W3
we can conclude that:
fo( Stag
1
Σ x4.
T
E.3
Proof of Theorem, Part 3: Proving the Three Cases
Combining Equation 6 and Equation 77, we get for all SiST:
h = + e W Σ A x Wv
and specifically:
ht = et + A,, W + Σ
A A eg W
Ole (ea))
FT -1
0
hr
+
A
+ OlE ((w)
Brag
Using the two equations above, we get for all 1 ST
hj - h = =] ef + (A), Ahwee w
N wt
e W/.
O(1) if jet, otherwise O(e Plas)
O vLu))
where the order of magnitude of the last two terms is given by Equation Additionally, we will use the fact that for all j 2 t:
lim d A AA At,,) +00
which we show in Appendix E.4
35

---

Case 1: For j < t (non-tagged tokens),
h wF (h, - -rr)
=h Wa,Waw W
1j 2T
A7,
0 Stag
+ +0(1)
= -h
A
b Stag
+ +0(1)
d Stag
T
FT
+ Arr
+ O(E +(((-our)) (le=Boe)
|tag
AT d Seag
Stag
+ +0(1)
A TT))) ): d. Stag + (Strag)
Dc, since > O.
Case 2: For j = t (sink token),
n7 W (hi-
hr)
([
[0-- DT
Stag
+ (Al, Ah.e)
+ O(e )ss))
Stap
bb (1 (1-8ta))
hT dd (1 Stag)
b. Seag
+ (Al, A):,)
+ +0(e osss)
d StaB
- d stag
+ d A (A), 11, 18ag + +0
Poak 10
Frag -00.
Case 3: For j > t (tagged tokens),
h (h; - -hr)
([2)) - IT
= =n
+
0
+ +A4), A7,,)
+ O(e mcss)
Stag]
bb Stag
hT (A) -A,,))
+ +Ole (see)
d SL08
-AA), Al. (xT
b. Staa
d. (.saag)
+ d. Al, (A),,
Ab,
(Srage )1re)
Btoe
O,
where lim o0 (A), 4 is proved in Appendix B.4 below.
36

---

E.4 Limit Proof
We first show that:
lim d d-Ar,(AAe Ap,,)stog
Sig
By definition:
exp( Stag)
A exp(stag) + > 1,ygt exp(xTTxx +
1+ E 1,ott 1 exp(2T<x + Stag)
and that:
exp(stag)
A exp(stag) + (t (5rag) 1 (t 1) exp((sra stag)
Thus,
Ek 1, exp(xTTx + stag) (1 - 1) exp((tre -6208)
A A
(1+ + (t 1) exp((see - stas)) (1+ 1 2R=11 eXp(xTxx + 1 scar)
Then:
lim
L (A A
91g 100
Σt Ek=1 slag exp(xrTk + 1 Stag) - (t - 1) exp($tag - sLag)
lim
Fig +00
(1+ (1 - exp((ttag - slag)) (1 Dx Ek= exp(xraR + 8tag)
0
The rest follows from the fact that lim +00 Al.: = 1 and applying basic limit laws. We now show similarly that for j > t:
lim (A), A 0,
lim (A), )
Beag
Observe that for j > t:
exp (tagg
j.t
exp(srag)
El Ek=1 kege expl2,0k 1)
1 Σ Ek=l exp(x,0x + Stag)
Then
lim (A), A
Stng 100
Σ Ek=I Stag exp(xrxT) + 1 Stag) Ek 1 Stag exp(x,jx + - 8tag)
1+ Ek ( + -swag)
(1+ E+ exp(xrIx + 1 Stag)
1
and similarly,
lim (A). A
Bene +00
Σ km liyt 2 exp(ETxx + 1 Stag)
- -E L exp(I,0x + - Stag)
1 + Hi- 1, exp + 1 -sae)) (1+ (I+E exp(xTI4 + 1 - 1-stag))
0
1
37

---

DEEPSEEK LLAMA 8B
DEEPSEEK QWEN 14B
DEEPSEEK QWEN 7B
LLAMA 3.1 8B
LLAMA 3 8B
PHI-3 MEDIUM
PIII3 MINI
PIII4
QWEN 2.5 14B
QWEN 2.5 3B
QWEN 2.5 7B
QWEN 3 14B
QWEN 8B
0.5
Avg. Explained Variance
0.4
0.3
0.2
0.1
2
5
8
10
Avg. Number of Sinks
Figure 20: Average explained variance plotted against the average number of sinks for different values of E. Each marker indicates a specific E in the set {0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.2.0.1.,..220.333 0.3
F Sensitivity Analysis of the Sink Threshold
We performed a comprehensive sensitivity analysis by varying the threshold across twelve values from 0.03 to 0.4 and evaluating its effect two key metrics:
1. the average number of identified sinks, and
2. the average variance explained by the tags.
The results are shown Figure 20
Across all models tested, the resulting curves are typically logarithmic-like: initially, decreasing the threshold adds new sinks and increases explained variance, but this gain saturates beyond a certain point. This suggests that smaller thresholds tend to capture less meaningful or redundant tokens. The table below provides representative values from this analysis support this observation.
Modcl Prodiction
G Optimization Details for Section 7.3
We generate total of 16,384 sequences of length 16 where half are allocated for training while the other half allocated for evaluation. For each sequence, random index is generated to place the [SEP]. We minimize the mean squared error loss between the predicted and ac- tual average and employ cosine annealing learning rate scheduler. We employ an initialization of Stag = to aid with convergence. Depicted in Figure 21 are the predicted averages the trained model versus actual average on the evaluation set.
Figure 21: The predicted averages the trained model versus the actual average on the evaluation set.
38

---

H
Qwen 3 Results
We present quantitative evidence that the QWEN models (Yagg et al. 2025| which incorporate QK normalization, exhibit the 'catch, tag, release' mechanism.
H.1
Sink Count and Variance Explained by Sinks
Variance Explained
Number of Sinks
0.5
3-
0.4
2-
1-
0.3
Owen
Qwen Qwen
(a) Attention Sink Counts for Qwen 3. Counts computed for each head and their distribution visualized each model using violin plots.
(b) Variance Explained by Tags for Qwen 3. The metric is computed for each head and summary statistics are shown for each model using box plots.
Figure 22: Quantitative Measurements for Qwen 3. Analogous measurements presented in Section for models the Qwen 3 family. Notably, these models exhibit the 'catch, tag, release' mechanism while having QK normalization.
H.2 Probe Performance
QWEN 3
Probe
8B
14B
fug
100%
87.0%
Oo tag
50.5%
64.5%
activattin
56.0%
82.0%
Table 5: Classification Accuracy of Probes for Qwen 3. The probe Oun is computed from the tag component the activation, 0no from the non-tag component, and Oactivation from the full activation.
I Additional Discussion
I.1 Practical Applications to L.LMs
The presence and utility of the 'catch, tag, release" mechanism have important implications for practical use large language models. Section 4 shows the potential for tags be used to steer activations (Subramani et al. 2022 Turner et al. 2025] which can be applied for model alignment and safety (Wang and Shu 2024]. prior work Yona al. (2025]. attention sinks were linked to repeated token phenomena and jailbreaking, suggesting the potential for the mechanism to be used for both exploit detection and mitigation. In model compression, preserving this mechanism may be critical, as we hypothesize it can be captured within low-rank subspace of the model's parameters, which must retained during pruning or quantization to maintain performance [Zhang and Papyan] 2025b Makni et al. 2025 Furthermore, recent work, WWan et al. (2025) has identified low-entropy tokens as central LTLM reasoning abilities, likely overlapping with the tag tokens we highlight, indicating this mechanism may play foundational role in LLM reasoning.
39

---

1.2 Role of the Fully Connected Layers
Empirically, prior work has shown that groups of token representations tend collapse low- dimensional subspaces, or even single points, across layers [Geshkkvski et al. 2025]. This raises the question: which tokens collapse together, and what determines the target of this collapse? We hypothesize that the "catch, tag, release" mechanism plays central role in both the grouping and the collapse target. Specifically, tokens attending the same attention sink receive common tag, which defines the direction and destination of their collapse. We further propose that the MLP layers, via low-rank subspace, progressively drive this collapse across layers by reinforcing the shared tag structure.
J Mass-Mean Probe Details
The ([CITY), [COUNTRY]) pairs are sourced from GeoNames
GeoNames
2025
J.1 Layer and Head Information
For Table I the layer and attention head for each model that we extracted the probes from are depicted Table([/eeoow:
Qwen 2.5
Llama-3 8B
Llama-3.1 8B
3B
7B
14B
Layer Head
16
18
28
40
21
21
Table 6: Layer/Attention Head information for results presented in Table |
J.2 Taxonomy of Tag Tokens
For each tag-only probe presented Table [ during their generation in Figure 23 below.
we present the histogram of the tag tokens that emerged
Healss
iled
400
200
QWEN 2.5 3B
QWEN 2.5 7B
QWEN 2.5 14B
LLAMA-3 8B
LLAMA-3.1 8B
Figure 23: A taxonomy of the tokens that were identified as attention sinks during the generation of the Omg and Ono tag probes. [PREAMBLE] is being used represent all tokens that correspond the following portion of the prompt: The city of Tokyo is in Japan. This statement TRUE, The city of Hanoi is Poland. This statement is: FALSE.
40

---

J.3 Definition of activaation
Following Marks and Tegmark 2024 the activation- based probes, activation are computed from the full activations:
A K...
From these representations, we obtain the class means:
Σ 2t
(
Σ
N+
N
TETrue
VEFabe
The within-class covariance matrix, E, then estimated, and the resulting mass-mean probes defined as:
Onctivation (2) (z)= 0== - E - p -)
J.4 Sentiment Analysis
To explore how the tagging mechanism can poten- tially capture broader range semantic and syntac- tic information beyond just binary True/False classifi- cations, we extend our experiments to assess whether tags encode sentiment. Specifically, construct prompts in the format shown in Figure 24 on the right.
it's charming and often affecting journey, This statement is: POSITIVE. unflinchingly bleak and desperate. This statement is: NEGATIVE. [SENTENCE].
This statement is:
Figure 24: Sentiment prompt example.
[SENTENCE] is replaced with various examples from the SST-2 dataset (Socher al. 2013). Using
the same experimental setup described in Section 4 we evaluated whether tags could capture sentiment polarity by decomposing the activations of the final period token (indicated by the arrow).
The results, presented in Table Table7 show that tags indeed encode positive and negative sentiment information effectively, supporting the broader applicability of the method to variety of different semantic tasks.
QWEN 2.5
LLAMA-3
LLAMA-3.1
Probe
3B
7B
14B
8B
8B
0ug 0 tag
88.5%
90.0%
94.0%
87.5%
86.5%
84.0%
53.0%
53.5%
81.5%
64.0%
activaaion
86.5%
80.0%
90.5%
88.0%
83.0%
Table 7: Classification Accuracy of Probes. Comparison of the three probe sets on positive/negative sentiment analysis.
K
Limitations
K.1
Theoretical Model
While our theoretical result provides a constructive proof that the 'catch, tag, release' mechanism can emerge in transformer architectures, it comes with limitations detailed below.
Theorem 7.1
does not prove that the 'catch, tag release' mechanism is necessary. There may exist alternative solutions that perform the task without attention sinks or outlier features. Consequently, the theoretical result should be viewed illustrative. Although we provide empirical support suggesting that similar dynamies arise trained models, we do not prove convergence to our specific construction under training.
41

---

The theoretical model operates under highly constrained conditions: a two-layer transformer solving a sequence averaging problem. While this setting valuable for analytical tractability, is removed from the complexity of real-world language modeling tasks.
Furthermore, depite the theoretical model being simple, the optimization process does always reach low-loss solution. We found that convergence depends on how the Stag parameter is initialized, with larger initial values generally leading more consistent success. Table 8 below shows how often the model achieved a high fit (R²² > 0.95) across 10 runs for different starting values of stagi
Stng initialization
Success Rate
10
4/10
6
3/10
1
0/10
Table 8: Success rate, measured by n² >> 0.95, across different Stag initializations.
K.2
Connection with Reasoning
While our findings in Section strongly suggest a structural role for sinks in reasoning, they do not yet establish a causal link between the two. This remains an important direction for future work.
K.3 Threshold-Based Metric
The threshold-based metric used to identify attention sinks does not guarantee full coverage of all such tokens. Our choice of 0.2 threshold, while consistent with prior studies, is not necessarily optimal. As discussed in Gu et al. (2024), there currently no principled method for determining this threshold. A sensitivity analysis for this threshold is provided in Appendix
L Compute Resources
All experiments involving LLMs were executed utilizing a single NVIDIA A40 with 48GB of GPU memory. This includes the experiments used to generate Figures 2
Table | and visualizations provided in Appendix [A] Each of the experiments in Sections 3 6 took approximately 40 minutes per model while the visualizations in Section P took roughly 60 minutes per model totalling for roughly 40 hours. Roughly 30 additional hours were utilized for preliminary experiments that did not reach the final paper.
M Models and Implementation
A comprehensive list of all the models used in our empirical study is found below:
Phi-3 Mini and Phi-3 Medium Abdin et al.
2024a
Phi-4 Abdin et al. 2024b
Qwen 2.5 3B, 7B, 14B, 32B (Yng et 2024
LLaMA-3 8B, LLaMM 3.1 8B Grattafiori al. 2024
Mistral 7B v0.3 [Tiang et al. 2023
Deepseek-Ri R1 LLaMA 8B, Deepseck-RR R1 Qwen 14B, Deepseek Math 7B DDeppSekkAA 2025]
Gemma 2 9B [Gemma Team et al.
2024
Gemma 3 4B, 12B [Gemma Team et al.
2025
Qwen 4B, 8B, 14B (Yang et al.
2025
We utilize the Transformers library by Huggingface Wolf et
2020
run all LLM experiments.
42


Reviews:
[{"review_id": "zuSJpvSvrB", "reviewer": "Reviewer_Z8vb", "summary": "This paper investigates the phenomenon of \"attention sinks\" in large language models, proposing that these sinks implement a \"catch, tag, release\" mechanism. In this process, specific tokens act as sinks to \"catch\" the attention of other tokens, \"tag\" them by adding their value vectors to the embeddings, and \"release\" this new information into the model's residual stream for use in subsequent layers. \nThrough probing experiments, they demonstrate that the tags carry semantically meaningful information, such as the truth-value of a statement. The study finds that this mechanism is more prominent in models fine-tuned for reasoning than in their pretrained counterparts. It also shows that attention sinks persist even in models with query-key (QK) normalization, an architecture that normalizes the query and key activations and may potentially suppress attention sinks. To ground these empirical findings, the paper introduces a minimal theoretical problem and provides a constructive proof showing that a two-layer transformer can solve it by explicitly implementing the \"catch, tag, release\" mechanism.", "questions": "- The paper states in Section 7.1 that the theoretical model uses causal attention. However, the proof in Appendix C.1 appears to contradict this. Specifically, in line 580, the equation for the attention weight $A_{i,t}^1$. (for query $i$ attending to key $t$) defines the denominator as a sum over all keys from $k = 1$ to $T$. For causal attention, this sum should only be over keys from $k = 1$ to $i$.\n\n- I notice there is a discrepancy between theoretical constructions and empirical training results. The theorem and its proof rely on the assumption that the tag value $s_{tag} \\to \\infty$. However, in my attempt to reproduce the results using the provided code, I observed that the learnable parameter $s_{tag}$, initialized at $10$ as per Appendix D, does not grow large and instead decreases slightly to around $9.3$ during training. Furthermore, I was unable to reproduce the clear averaging behavior shown in Figure 7(c).\n\n- Minor: there is a typo in line 622 as the initialization value should be $10$ instead of $-10$ according to the codes.", "limitations": "Yes, the authors address some limitations in Appendix B regarding the theoretical results in Section 7, but the discussion could be more comprehensive by also including the limitations of the empirical analyses in Sections 5 and 6.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strenghts\n- The paper introduces a novel \"catch, tag, release\" mechanism to explain the function of attention sinks in LLMs. This work contributes to a deeper understanding of LLMs' behavior, enhancing the interpretability.\n\n- The proposed mechanism is supported by a comprehensive empirical investigation. The authors validate the \"catch, tag, release\" mechanism across various LLM families and sizes (including Qwen, Phi, and Llama), demonstrating the generality of their findings. The claims are complemented by probing experiments that convincingly show that the tags generated by attention sinks encode semantically meaningful information.\n\nWeaknesses\n- While the core \"catch, tag, release\" mechanism is well-supported, the analysis for two of the claims (A3 and A4) seems preliminary and less developed. \n   - Claim A3 (Section 5): The paper shows that attention sinks and the proposed mechanism are more prevalent in reasoning models than in their pretrained counterparts. However, the authors do not go further to explain how this increased prevalence contributes to the models' reasoning abilities. Without this deeper analysis, the claim remains an observation.\n\n  - Claim A4 (Section 6): The analysis of models with QK normalization is limited to showing that the number of attention sinks does not decrease without further discussion. Moreover, the paper does not verify whether the \"catch, tag, release\" mechanism is still present in these models. This omission makes the conclusion in this section less impactful.\n\nOverall, while the core contributions of the paper are strong, the analysis in Section 5 and 6 is insufficient. These sections would need more evidence and discussion to be considered on par with the paper's primary findings.", "was_revised": true, "final_justification": "Most of my concerns have been resolved so I'm happy to raise the score to 4.", "rebuttal": "We would first like to thank the reviewer for their thorough and thoughtful analysis of the manuscript, its appendices, and the supplementary material, which we have leveraged to improve the overall quality of our work.\n \n---\n\n## The analysis of models with QK normalization is limited, ... the paper does not verify whether the \"catch, tag, release\" mechanism is still present in these models. \n\nWe agree with the reviewer that the omission of measurements relating to models with QK norm was an oversight in our original manuscript. To provide evidence that the 'catch, tag, release' mechanism is still present in models with QK norm, **we have now added measurements for Qwen 3 8B, and 14B showing their sink count** (using violin plots), **variance explained by tags** (using box and whiskers plots), as presented in Section 3.2 Figure 3. We summarize these results in the table below, which averages across all attention heads.\n\n| **Probe**                |  **QWEN 3**       | **QWEN 3**  |\n|--------------------|--------------|-------|\n|                    | **8B**| **14B**|\n| Avg. Number of Sinks        |  1.03 | 1.04 |\n| Avg. Variance Explained     | 0.26 | 0.25 |\n\nAdditionally, we **also executed the probing experiments provided in Section 4 on the Qwen 3 models** which are presented in the table below:\n\n| **Probe**                |  **QWEN 3**       | **QWEN 3**  |\n|--------------------|--------------|-------|\n|                    | **8B**| **14B**|\n| $\\theta_{\\text{tag}}$        | 100% | 87.0% |\n| $\\theta_{\\text{no tag}}$     | 50.5% | 64.5% |\n| $\\theta_{\\text{activation}}$ | 56.0% | 82.0%   |\n\nTo summarize, the above results follow the same trends as the models without QK norm, suggesting that the 'catch, tag, release’ mechanism is still present in these models. \n\n---\n\n## The authors do not go further to explain how this increased prevalence contributes to the models' reasoning abilities.\n\nWe thank the reviewer for highlighting the need for a deeper analysis supporting Claim A3. In response, we have **added a new experiment to Section 5 that includes the most frequent attention sink tokens in both reasoning-tuned and base (non-reasoning) versions of the Qwen 2.5 14B model**:\n\n| Model               | Sink String | Avg. Variance Explained | Frequency |\n|---------------------|-------------|--------------------------|-----------|\n| Qwen 2.5 14B        | BOS         | 0.28              | 328487    |\n|                     | .           | 0.25              | 5323      |\n|                     | ,           | 0.20             | 3807      |\n|                     | Ġthe        | 0.23             | 3333      |\n|                     | ĠI          | 0.23              | 3283      |\n|                     | Ġ”          | 0.24              | 3203      |\n|                     | Ġto         | 0.19              | 2603      |\n|                     | Ġbegan      | 0.20              | 2592      |\n|                     | ĠStark      | 0.18              | 2512      |\n|                     | ,”          | 0.22              | 2253      |\n| Deepseek Qwen 14B   | BOS         | 0.30             | 307600    |\n|                     | THE         | 0.31              | 50248     |\n|                     | The         | 0.30              | 47535     |\n|                     | Doctor      | 0.30              | 26928     |\n|                     | SP          | 0.30              | 24718     |\n|                     | MON         | 0.31              | 23680     |\n|                     | CAP         | 0.30              | 22832     |\n|                     | IT          | 0.29              | 22416     |\n|                     | GR          | 0.30              | 21915     |\n|                     | IMAGE       | 0.30              | 20272     |\n\nThis table reveals not only a quantitative increase in sink prevalence, but also a qualitative shift in which tokens form sinks, indicating a meaningful structural adaptation in how the 'catch, tag, release' mechanism operates under reasoning fine-tuning as **we describe in the additional paragraph that will be added to the manuscript**:\n\n>In base models such as Qwen 2.5 14B, sink formation predominantly occurs around function words (e.g., Ġthe, Ġto, ĠI) and punctuation (such as . or ,), likely due to their high frequency and syntactic roles. In contrast, the DeepSeek Qwen 14B reasoning-tuned model forms attention sinks around semantically significant or task-structuring tokens, such as: \n> - IMAGE suggests segmentation for multimodal inputs, grouping the token with subsequent image-related descriptions.\n> - Doctor likely marks named entities relevant for reasoning over domain-specific content. \n> - Tokens like MON, CAP, IT, and GR are plausible abbreviations or categorical labels (e.g. weekdays, captions, grades) that suggest segmentation for structured data. \n> - SP may represent speaker or section markers, important in multi-step or dialog-based reasoning.\n>\n> These sink tokens not only appear with high frequency but also explain a substantial portion of the variance, indicating their importance in structuring the model’s internal representation. **The presence of such tokens, absent in the base model’s sink list, suggests that reasoning fine-tuning reorients the attention sink mechanism from syntactic attractors to semantically meaningful or task-specific units.**\n\nThis new evidence strengthens Claim A3 by showing that attention sinks in reasoning-tuned models are not just more frequent, they are repurposed to segment, tag, and organize input in ways that directly support reasoning behaviors such as modality transitions, entity tracking, and logical clause separation. The 'catch, tag, release' mechanism thus adapts in both frequency and function to facilitate reasoning.\n\nWe also now **explicitly acknowledge in the Limitations section that this connection remains observational**: \n\n> While our findings in Section 5 strongly suggest a structural role for sinks in reasoning, they do not yet establish a causal link between the two. This remains an important direction for future work.\n\n---\n\n## In line 580, the equation for the attention weight... defines the denominator as a sum over all keys from $k=1$ to $T$ .\n\nThank you for catching this error. **The summation should indeed be from $k=1$ to $ k=i$ instead of from $k=1$ to $k=T$** (which would change the first two lines of that derivation). However, **this does not impact the later two equations in that derivation so the ultimate limit and result is still correct**. We have also carefully gone over all other equations in the proof to ensure that this typo does not occur elsewhere. \n\n---\n\n## Unable to reproduce the clear averaging behaviour shown in Figure 7(c)\n\nIf no clear averaging behavior emerges, **this indicates that optimization failed to converge to a low-loss solution**. We have observed this same issue in our own experiments. However, in all successful runs that do converge, we consistently observe the emergence of the 'catch, tag, and release’ mechanism and the clear averaging shown in Figure 7(c). **To clarify these issues, we have added the following paragraph to the limitations section in Appendix B**:\n\n> Although the theoretical model is simple, the optimization process does not always reach a low-loss solution. We found that convergence depends on how the $s_{\\text{tag}}$ parameter is initialized, with larger initial values generally leading to more consistent success. The table below shows how often the model achieved a high fit ($R^2 > 0.95$) across 10 runs for different starting values of $s_{\\text{tag}}$:\n\n| $s_{\\text{tag}}$ initialization | Success Rate ($R^2>0.95$) |\n|----------------------|----------------------|\n| 10                    | 4/10                |\n| 6                   |     3/10           |\n| 1                    |     0/10            |\n\n---\n\n## The learnable parameter, initialized at 10 as per Appendix D, does not grow large and instead decreases\n\nThe theoretical result provides sufficient conditions for the emergence of the mechanism, but these are not necessary. Empirically, **we observe that the mechanism can still emerge under different dynamics**. For instance, instead of $s_{\\text{tag}}$ diverging while the weights remain fixed, it is also possible for $s_{\\text{tag}}$ to remain fixed while the weights shrink. Fully characterizing all such cases is non-trivial. However, **the key point is that the 'catch, tag, release' mechanism consistently emerges in this toy setting**. Moreover, **$s_{\\text{tag}}$ is initialized to a value already $10\\times$ larger than the second coordinate of the number tokens, which may reduce the need for it to grow further**.\n\n---\n\n## There is a typo in line 622\n\nThank you for bringing this to our attention, **we have since revised the appendix to correct this typo from -10 to 10**. \n\n---\n\nWe extend our gratitude to the reviewer for their careful analysis of our paper and for highlighting issues that we have overlooked. Should the reviewer think that we have taken sufficient action to address their concerns, we would be most appreciative of any reconsideration of their score. \n>"}, {"review_id": "u0bgJEd7XF", "reviewer": "Reviewer_RjRh", "summary": "This paper studies attention sinks in large language models, which are tokens that receive a lot of attention, like the first token or punctuation marks. The authors argue that these sinks catch attention, add shared information to other tokens (tag), and release that information into the model’s residual stream. They provide visual and quantitative evidence for this mechanism in several popular models, show that the tags carry semantic meaning (like true/false labels), and observe that the effect is stronger in models trained for reasoning. They also introduce a small theoretical model where this behavior emerges naturally.", "questions": "Main Questions:\n- **(Q1)** Could you examine which tokens become sinks (e.g., punctuation, logical connectors), whether they equally contribute to explained variance, and whether this changes after reasoning fine-tuning? Is it possible to create a taxonomy, possibly using bigger and different dataset? This would clarify whether the additional sinks in reasoning models are semantically meaningful.\n- **(Q2)** The threshold-based sink definition is arbitrary, based on previous studies. Could you provide some intuition why this metric is indeed select all attention sinks (20% of explained variance in some cases is still low)?  Maybe you could provide some sort of sensitivity analysis over threshold and compare to alternatives (e.g., entropy or KL divergence) to ensure robustness? \n\nMinor Questions and Comments:\n- I have some doubts about value for activation probe for QWEN 2.5 3B in Table 1. How could it be that accuracy is still 50% after adding tags? \n- Though one can understand it intuitively, $\\theta_{activation}$ is not defined. As I believe you perform computations similar to tag/no tag, but it would be better to state it in the text", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- **(S1)** Novel unified concept: the “catch, tag, release” mechanism offers a clear and intuitive interpretation of attention sink behavior. This concept unifies several previously scattered observations in the literature.\n- **(S2)** Clarity and accessibility: the paper is well-written, with clear diagrams and good structure. The visualizations are intuitive and support the narrative effectively.\n- **(S3)** Broad empirical scope: the authors examine the phenomenon across multiple model families (QWEN, LLAMA, Mistral, PHI), architectures (with and without QK norm), and tasks (reasoning vs. non-reasoning). This improves the generality of the claims. The experiments showing that sink-derived tags encode the truth value of statements are also a strong contribution.\n\nWeaknesses:\n- **(W1)** Lack of analysis on which tokens become attention sinks: while the paper compares the number of sinks between models (e.g., reasoning vs. pretrained), it does not analyze the types of tokens that serve as sinks. The discussion and analysis of attention sinks types could be very beneficial and provide more intuitive insights\n- **(W2)** Arbitrary sink detection metric: the definition of an attention sink (Eq. 1) relies on a fixed attention threshold, but this choice is not validated. There’s no sensitivity analysis and comparison to alternative metrics (e.g., attention entropy?). This undermines the robustness of the conclusions.", "was_revised": true, "final_justification": "This is a solid, well-written study of attention sinks, and their mechanistic role. The authors provided thorough additional experiments, clear methodological justification. My concerns were addressed properly.", "rebuttal": "We thank the reviewer for their thoughtful review and for offering several valuable suggestions to enhance the comprehensiveness of our analysis.\n\n---\n\n##  Could you examine which tokens become sinks, whether they equally contribute to explained variance, and whether this changes after reasoning fine-tuning? Possible to create a taxonomy, using bigger and different dataset? \n\nWe think this is a great suggestion and have **executed additional experiments that count the frequency of tokens that appear as sinks and measure the average explained variance of each token when it appears as a sink**.\nTo this end, we analyzed **200 prompts of length 1024 from the QuALITY dataset**, using **both the base and reasoning-tuned versions of the Qwen 14B model**. The results, which are representative of broader trends we have observed, are now included in Section 5 and summarized in the table along with a paragraph for discussion below:\n\n| Model               | Sink String | Avg. Variance Explained | Frequency |\n|---------------------|-------------|--------------------------|-----------|\n| Qwen 2.5 14B        | BOS         | 0.28              | 328487    |\n|                     | .           | 0.25              | 5323      |\n|                     | ,           | 0.20             | 3807      |\n|                     | Ġthe        | 0.23             | 3333      |\n|                     | ĠI          | 0.23              | 3283      |\n|                     | Ġ”          | 0.24              | 3203      |\n|                     | Ġto         | 0.19              | 2603      |\n|                     | Ġbegan      | 0.20              | 2592      |\n|                     | ĠStark      | 0.18              | 2512      |\n|                     | ,”          | 0.22              | 2253      |\n| Deepseek Qwen 14B   | BOS         | 0.30             | 307600    |\n|                     | THE         | 0.31              | 50248     |\n|                     | The         | 0.30              | 47535     |\n|                     | Doctor      | 0.30              | 26928     |\n|                     | SP          | 0.30              | 24718     |\n|                     | MON         | 0.31              | 23680     |\n|                     | CAP         | 0.30              | 22832     |\n|                     | IT          | 0.29              | 22416     |\n|                     | GR          | 0.30              | 21915     |\n|                     | IMAGE       | 0.30              | 20272     |\n\n>In base models such as Qwen2.5-14B, sink formation predominantly occurs around function words (e.g., Ġthe, Ġto, ĠI) and punctuation (e.g., . or ,), likely due to their high frequency and syntactic roles. In contrast, the DeepSeek-Qwen-14B reasoning-tuned model forms attention sinks around semantically significant or task-structuring tokens, such as: \n>\n> - IMAGE suggests segmentation for multimodal inputs, grouping the token with subsequent image-related descriptions.\n> - Doctor likely marks named entities relevant for reasoning over domain-specific content. \n> - Tokens like MON, CAP, IT, and GR are plausible abbreviations or categorical labels (e.g., weekdays, captions, grades) that suggest segmentation for structured data. \n> - SP may represent speaker or section markers, important in multi-step or dialog-based reasoning.\n>\n>These sink tokens not only appear with high frequency but also explain a substantial portion of the variance, indicating their importance in structuring the model’s internal representation. The presence of such tokens—absent in the base model’s sink list—suggests that reasoning fine-tuning reorients the attention sink mechanism from syntactic attractors to semantically meaningful or task-specific units.\n\n---\n\n##  The threshold-based sink definition is arbitrary, based on previous studies. Could you provide some intuition why this metric is indeed select all attention sinks\n\nWe agree that this is a limitation of our current methodology and that the threshold we adopt, based on prior work, does not guarantee the identification of all tokens functioning as attention sinks. To acknowledge this, **we have added the following paragraph to the Limitations section (Appendix B)**:\n\n>The threshold-based metric used to identify attention sinks does not guarantee full coverage of all such tokens. Our choice of a 0.2 threshold, while consistent with prior studies, is not necessarily optimal. As discussed in [1], there is currently no principled method for determining this threshold.\n\nTo assess how sensitive our results are to this design choice, we conducted a comprehensive sensitivity analysis. Specifically, **we varied the threshold across twelve values ranging from 0.03 to 0.4 and evaluated the impact on two key metrics: (1) the average number of identified sinks and (2) the average variance explained by the tags.** We will include a plot in the Appendix where each point corresponds to a threshold, the x-axis represents the average number of sinks, and the y-axis reflects the variance explained. \n\nAcross all models tested, **we find that the resulting curves are typically logarithmic-like: initially, decreasing the threshold adds new sinks and increases explained variance, but this gain saturates beyond a certain point**. This suggests that smaller thresholds tend to capture less meaningful or redundant tokens. The table below provides representative values from this analysis to support this observation.\n\n\n| $\\epsilon \\qquad \\qquad$  | Deepseek Llama 8B (Avg. Explained Variance) | Deepseek Llama 8B (Avg. Number of Sinks) | Llama 3.1 8B (Avg. Explained Variance) | Llama 3.1 8B (Avg. Number of Sinks) |\n| :--- | :------------------------------------------ | :--------------------------------------- | :------------------------------------ | :--------------------------------- |\n| 0.03 | 0.429 | 3.230 | 0.449 | 5.325 |\n| 0.04 | 0.406 | 2.343 | 0.411 | 3.744 |\n| 0.05 | 0.392 | 1.886 | 0.388 | 2.897 |\n| 0.06 | 0.382 | 1.615 | 0.372 | 2.382 |\n| 0.07 | 0.376 | 1.453                                    | 0.360                                 | 2.037                              |\n| 0.08 | 0.371                                       | 1.338                                    | 0.350                                 | 1.798                              |\n| 0.09 | 0.368                                       | 1.263                                    | 0.344                                 | 1.636                              |\n| 0.1  | 0.365                                       | 1.207                                    | 0.339                                 | 1.507                              |\n| 0.15 | 0.358                                       | 1.072                                    | 0.324                                 | 1.190                              |\n| 0.2  | 0.355                                       | 1.029                                    | 0.315                                 | 1.068                              |\n| 0.3  | 0.353                                       | 0.992                                    | 0.300                                 | 0.936                              |\n| 0.4  | 0.345                                       | 0.944                                    | 0.274                                 | 0.807                              |\n\n---\n\n## … compare to alternatives (e.g., entropy or KL divergence) \n\nWe initially considered using alternative metrics such as entropy or KL divergence to identify attention sinks. However, attention sinks are defined over columns of the attention probability matrix, that is, a sink corresponds to a token that consistently receives high attention across multiple rows. **Since the columns are not normalized, the sink column does not represent a valid probability distribution. As such, entropy or KL divergence cannot be meaningfully computed over columns.**\n\nLikewise, using row-wise entropy to detect focused attention would be unable to capture attention sinks. **Low entropy across rows merely indicates that each token is attending strongly to a few others, but it says nothing about which tokens are being attended to.** For example, if the attention matrix is close to diagonal, each row has low entropy, yet no token consistently receives attention—i.e., no sink exists.  For this reason, we opted for a threshold-based metric that directly evaluates whether certain tokens receive consistently high aggregate attention, better aligning with the conceptual definition of attention sinks.\n\n---\n\n## How could it be that accuracy is still 50% after adding tags?\n\nWe have double-checked the data, and the result is indeed accurate for that specific attention head. One possible explanation is that the tag portion of the activation has a significantly smaller norm compared to the non-tag portion. As a result, the classifier may effectively ignore the tag component due to its negligible contribution, leading to no improvement in accuracy despite the presence of tags.\n\n---\n\n## $\\theta_{activation}$  is not defined, would be better to state it in the text\n\nThank you for pointing out a potential point of unclarity. We have included in the Appendix the full equations and descriptions of the probe.\n\n--- \n\nWe thank the reviewer for suggesting several valuable directions for additional measurements which have enhanced the comprehensiveness of our analysis. If the reviewer finds that these additions strengthen the paper, we would be grateful for any reconsideration of the score.\n\n>\n\n[1] Gu et al. (2024). When Attention Sink Emerges in Language Models: An Empirical View."}, {"review_id": "TOIRfEjDgn", "reviewer": "Reviewer_ZXyE", "summary": "This paper investigates the phenomenon of \"attention sinks\" in Large Language Models (LLMs), where attention is disproportionately focused on a few specific tokens (like the first token or punctuation). The authors propose and validate a novel functional mechanism they term \"catch, tag, release.\" This framework posits that attention sinks \"catch\" the attention of other tokens, \"tag\" them by adding a common directional vector to their embeddings, and then \"release\" these tagged embeddings back into the residual stream. This tag, as the authors demonstrate, carries semantically meaningful information that can be used by deeper layers of the model.\nThe paper provides extensive evidence for this mechanism through the following contributions:\n1.\tVisual (PCA) and quantitative (\"variance explained\") analyses across a wide range of model families (LLAMA, PHI, QWEN, etc.) confirm the existence and significance of this tagging behavior.\n2.\tClever probing experiments on a truth-falsity dataset show that these tags can encode high-level semantic information, such as the truth value of a statement, with very high accuracy (up to 99.5%).\n3.\tThe authors show that the mechanism is more pronounced in models fine-tuned for reasoning and is robust to architectural changes like Query-Key Normalization.\n4.\tA key contribution is a minimal, two-layer Transformer model designed for a sequence-averaging task. The authors prove theoretically (Theorem 7.1) that the 'catch, tag, release' mechanism is a valid solution and demonstrate that this exact mechanism emerges spontaneously through standard training.\nThe paper concludes that attention sinks are not an artifact or a bug, but a fundamental and functional mechanism for information propagation and labeling within LLMs.", "questions": "⦁\tThe probing experiments in Section 4 are very convincing. Are there other kinds of semantic or syntactic information encoded by these tags? For instance, in narrative text, could tags encode sentiment, character viewpoints, or tense, thereby \"tagging\" clauses for downstream processing?\n⦁\tThe \"Release\" step is described as the tagged tokens propagating through the residual stream. Are there specific types of attention heads or MLP neurons in subsequent layers that specialize in processing tokens based on the direction of these tags?\n⦁\tIn the theoretical model (Section 7), the \"tag\" and \"number\" components of the embedding are cleanly separated into two dimensions. In real-world models, these are obviously entangled within a high-dimensional space. The probing results suggest they are still highly separable. How \"clean\" is this separation in practice? Does the tag component have minimal projection onto the token's original semantic direction?\n⦁\tIn Section 3.1, the threshold ϵ for identifying an attention sink is set to 0.2. How sensitive are the quantitative results, such as the average number of sinks (Figure 3a) and the variance explained (Figure 3b), to the choice of this hyperparameter?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths\n1.\tThe \"catch, tag, release\" analogy is intuitive, powerful, and provides a functional explanation for the widely observed attention sink phenomenon. It shifts the perspective from sinks being a \"bug\" to a \"feature.\"\n2.\tThe paper's claims are supported by comprehensive experiments that are both qualitatively illustrative and quantitatively robust.\n⦁\tThe PCA visualizations in Figure 2 provide a clear and convincing visual proof of the \"tagging\" process, showing token embeddings clustering after the attention layer output.\n⦁\tThe introduction of the \"variance explained by tags\" metric allows for a quantitative comparison across many different models, layers, and heads, firmly establishing the ubiquity of the mechanism.\n⦁\tThe probing experiment in Section 4 is strong. By showing that the \"tag\" component of an activation can classify a statement's truth value far better than the non-tag or even the full activation, it provides powerful evidence that these tags carry specific, high-level semantic content.\n3.\tA major strength is the theoretical analysis in Section 7. By constructing a minimal problem and proving that the 'catch, tag, release' mechanism is a valid solution (Theorem 7.1), the authors provide a solid theoretical footing for their empirical observations. The demonstration that this mechanism emerges naturally through optimization (Figure 7) is a crucial result, suggesting it is a plausible and efficient strategy learned by neural networks.\n4.\tThe paper is exceptionally well-written and organized. The introduction clearly lays out the existing theories and their shortcomings through four specific open questions (Q1-Q4). The contributions (A1-A4) directly address these questions, creating a tight and compelling narrative. The figures are clear, well-captioned, and effectively support the paper's arguments.\nWeaknesses\n1.\tLimited Scope of Semantic Investigation: The paper brilliantly demonstrates that tags can encode truth-values. However, it leaves open the question of the full range of information these tags can carry. It would be interesting to see if this mechanism is also used for lower-level syntactic roles, coreference resolution, or other types of semantic relations.\n2.\tSimplicity of the Theoretical Model: As the authors commendably acknowledge in their limitations (Appendix B), the theoretical model is a highly simplified two-layer Transformer operating on 2D embeddings. While perfect for an illustrative proof-of-concept, it is a significant simplification of modern, deep, high-dimensional LLMs. The connection between the clean, emergent mechanism in this toy model and the potentially messier version in practice could be explored further.", "was_revised": true, "final_justification": "The paper provides a functional explanation for the widely observed attention sink phenomenon. The paper is both theoretically and empirically sound. The writing is also good. I have some minor concerns but the authors have addressed them. I think the paper is ready for acceptance.", "rebuttal": "Thank you for your review and for providing additional ideas and suggestions as to how we can extend our experiments. \n\n---\n\n## Are there other kinds of semantic or syntactic information encoded by these tags? For instance, in narrative text, could tags encode sentiment…\n\nIndeed, the tagging mechanism we explore can potentially capture a broader range of semantic and syntactic information beyond just binary True/False classifications. To show this explicitly, **we extended our experiments to assess whether tags can encode sentiment**. Specifically, we construct prompts in the following format:\n\n```\nit's a charming and often affecting journey. This statement is: POSITIVE. \nunflinchingly bleak and desperate. This statement is: NEGATIVE. \n[SENTENCE]. This statement is:\"\n```\n\nwhere [SENTENCE] is replaced with various examples from the SST-2 dataset. Using the same experimental setup described in Section 4, we evaluated whether tags could capture sentiment polarity. The results, presented in the table below, show that **tags indeed encode positive and negative sentiment information effectively, supporting the broader applicability of the method to a variety of different semantic tasks**.\n\n| **Probe**             | **QWEN 2.5** |    **QWEN 2.5**    |    **QWEN 2.5**    | **LLAMA-3** | **LLAMA-3.1** |\n|-----------------------|--------------|--------|--------|-------------|---------------|\n|                       | 3B           | 7B     | 14B    | 8B          | 8B            |\n| $ \\theta_{\\text{tag}} $       | 88.5%          | 90.0%  | 94.0%  | 87.5%       | 86.5%         |\n| $ \\theta_{\\text{no tag}} $     | 84.0%        | 53.0%  | 53.5%  | 81.5%       | 64.0%         |\n| $ \\theta_{\\text{activation}} $ | 86.5%        | 80.0%  | 90.5%    | 88.0%       | 83.0%         |\n\n---\n\n## Are there specific types of attention heads or MLP neurons in subsequent layers that specialize in processing tokens based on the direction of these tags? \n\nThis is an excellent and non-trivial question. In principle, one could examine the cosine similarity between tag embeddings and individual neurons (i.e., rows in the MLP weight matrices of subsequent layers) to identify whether certain neurons respond selectively to tags. However, as discussed in [1], **it is unlikely that features, such as tags, are cleanly localized to individual neurons**. Instead, model features tend to be superimposed across many directions in activation space due to the model’s limited dimensionality relative to the complexity of the data it learns to encode.\n\nIn this view, the directions corresponding to tags may not be easily detectable when analyzing individual neuron activations. **Instead, we hypothesize that the representation of a tag is distributed and likely spans multiple directions in the activation space, potentially spanned by the top singular vectors of the relevant weight matrices**. This suggests that tag-related information is embedded across a subspace rather than isolated to specific neurons. \n\nProbing this hypothesis would require more advanced interpretability techniques—for example, analyzing representations in the singular vector basis, using sparse autoencoders, or applying subspace probing tools from mechanistic interpretability to recover the directions associated with specific functions. We believe this is a promising direction for future research.\n\n[1] Toy Models of Superposition https://transformer-circuits.pub/2022/toy_model/index.html  \n\n---\n\n## How \"clean\" is this separation in practice? Does the tag component have minimal projection onto the token's original semantic direction?\n\nWe conducted additional experiments to better understand how distinct the tag component is from the token's original semantic representation. Specifically, **we measured the cosine similarity between the tag (after being mapped back into the residual stream) and the token representations of non-sink tokens to which the tag is applied, quantifiying the degree of overlap between the tag direction and the token’s inherent semantics**.\nWe averaged the results over 200 prompts of 1024 tokens each, and across all layers of the model. As shown in the table below, **the cosine similarity is consistently low, especially in the LLaMA models, which suggests that the tag subspace remains largely orthogonal to the native token representations**. This indicates a relatively \"clean\" separation between the semantics introduced by the tag and the token's original meaning.\n\n| | Deepseek Llama 8B | Deepseek Qwen 7B | Deepseek Qwen 14B | Llama3 8B | Llama 3.1 8B | Qwen 3 8B | Qwen 3 14B | Qwen 2.5 8B | Qwen 2.5 7B | Qwen 2.5 14B |\n| :---------------- | :---------------- | :--------------- | :---------------- | :-------- | :---------- | :------- | :-------- | :-------- | :-------- | :--------- |\n| Cosine Similarity                   | -0.0674           | -0.1651          | -0.1449           | 0.0238    | 0.0099      | -0.0868  | -0.1290   | -0.1738   | -0.1581   | -0.1444    |\n\n---\n\n## How sensitive are the quantitative results, such as the average number of sinks (Figure 3a) and the variance explained (Figure 3b), to the choice of this hyperparameter?\n\nTo assess how sensitive our results are to this design choice, **we conducted a comprehensive sensitivity analysis. Specifically, we varied the threshold across twelve values ranging from 0.03 to 0.4 and evaluated the impact on two key metrics: (1) the average number of identified sinks and (2) the average variance explained by the tags**. We will include a plot in the appendix where each point corresponds to a threshold, the x-axis represents the average number of sinks, and the y-axis reflects the variance explained. \n\nAcross all models tested, we find that the **resulting curves are typically logarithmic-like: initially, decreasing the threshold adds new sinks and increases explained variance, but this gain saturates beyond a certain point**. This suggests that smaller thresholds tend to capture less meaningful or redundant tokens. The table below provides representative values from this analysis to support this observation.\n\n| $\\epsilon \\qquad \\qquad$  | Deepseek Llama 8B (Avg. Explained Variance) | Deepseek Llama 8B (Avg. Number of Sinks) | Llama 3.1 8B (Avg. Explained Variance) | Llama 3.1 8B (Avg. Number of Sinks) |\n| :--- | :------------------------------------------ | :--------------------------------------- | :------------------------------------ | :--------------------------------- |\n| 0.03 | 0.429                                       | 3.230                                    | 0.449                                 | 5.325                              |\n| 0.04 | 0.406                                       | 2.343                                    | 0.411                                 | 3.744                              |\n| 0.05 | 0.392                                       | 1.886                                    | 0.388                                 | 2.897                              |\n| 0.06 | 0.382                                       | 1.615                                    | 0.372                                 | 2.382                              |\n| 0.07 | 0.376                                       | 1.453                                    | 0.360                                 | 2.037                              |\n| 0.08 | 0.371                                       | 1.338                                    | 0.350                                 | 1.798                              |\n| 0.09 | 0.368                                       | 1.263                                    | 0.344                                 | 1.636                              |\n| 0.1  | 0.365                                       | 1.207                                    | 0.339                                 | 1.507                              |\n| 0.15 | 0.358                                       | 1.072                                    | 0.324                                 | 1.190                              |\n| 0.2  | 0.355                                       | 1.029                                    | 0.315                                 | 1.068                              |\n| 0.3  | 0.353                                       | 0.992                                    | 0.300                                 | 0.936                              |\n| 0.4  | 0.345                                       | 0.944                                    | 0.274                                 | 0.807                              |\n\n--- \n\nWe are very grateful to the reviewer for their thoughtful comments, constructive suggestions for future work, and positive evaluation of our submission. We hope that our responses and additional results further clarify the contributions and are viewed as a meaningful step forward.\n>"}, {"review_id": "K9wyfV6gdA", "reviewer": "Reviewer_KjwZ", "summary": "The paper discusses a potential reason why \"attention sinks\" (i.e., attention patterns where all attention concentrates on the first token) are ubiquitous in large language models (LLMs). It introduces a linear-algebraic mechanism which attention sinks execute in LLMs, which is labeled as \"catch, tag, release\" --- each attention head experiencing an attention sink will endow the input sequence with a component in a distinctive direction in embedding space given by the span of the value states corresponding to the sink tokens. Empirical evidence on both completely synthetic data and more realistic data show that attention sinks introduce a tag, that these tags explain much of the empirical variance of the outputs of the heads, that they have interpretable semantic meaning in controlled synthetic setups, that they occur more in reasoning-distilled models than only-pretrained models, and that attention sinks occur even when QK normalization is applied.", "questions": "See above, but to summarize:\n- What is the role of multiple attention heads in practice and in the synthetic setups considered in the work?\n- What is the role of other non-attention components in the network (MLP, layer norms, etc) in this mechanism?\n- Are there any potential applications of this mechanism to the practice of LLMs (capabilities, interpretability, etc)?", "limitations": "Yes (Appendix B and the checklist)", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The overall mechanism of each attention head to influence the output is clear and succinct.\n- The experimental setup using large models is reasonable for many of the tasks, and may be sufficient to demonstrate the claims (modulo some ambiguity, see below).\n- The synthetic experimental setup is unambiguous, the claim is cleanly proved, and the setting may be valuable for future work (potentially in a more complex form).\n\nWeaknesses:\n- The writing is not very clear; there is a large ambiguity in the text that seems to be glossed over. Namely: what is the role of multiple attention heads? In particular, for a given input sequence, many attention heads are sinks, not just one. How do the tags interact with each other? Is there constructive or destructive interference (or something more complicated), or are all the \"tags\" orthogonal? It is especially confusing since in Figure 2 the tokens _later in the network_ seem to adhere strongly to the tags produced by the single discussed head earlier in the network, whereas multiple heads surely were attention sinks. It would be great to clarify these as it would strengthen the core contribution of the work.\n- There is little-to-no discussion of applications to the practice of LLMs. The main contributions are in clarifying the different mechanisms proposed in prior work. Even though this work is obviously not applied in nature, it would be good to discuss potential applications of the discussion, e.g. for large-scale interpretability or model surgery. \n- The theoretical analysis of the completely synthetic setup is for a 2 layer iterated attention network with one attention head in each layer, not necessarily a transformer. This is understandable because analyzing transformer dynamics is very challenging. Still it would be better to discuss the role of the MLP for instance.", "was_revised": true, "final_justification": "I initially recommended borderline rejection, but now recommend acceptance. The main weakness for me in the work was the case of multiple attention heads, which was previously ignored, but the authors did lots of leg work in the rebuttal to address this point, in my opinion making it a better paper. The other unresolved issues (in my opinion this is a lack of theory/synthetic experiments for the multi-head case) are relatively minor. The paper does seem to have some minor technical/presentation issues, from looking at other reviews, but I think the overall focus, direction, and results are quite solid.", "rebuttal": "We are grateful for the reviewer's thoughtful feedback highlighting areas that warrant further discussion and clarification. We address these concerns below.\n\n---\n\n## Namely: what is the role of multiple attention heads? In particular, for a given input sequence, many attention heads are sinks, not just one.\n\nThank you for this insightful question. **Our findings indicate that each attention head implements an independent instance of the catch, tag, release mechanism**. Concretely, each head selects its own attention sink, captures a distinct subsequence of tokens, and injects a unique tag direction into those tokens’ representations via its value vector (potentially repeating this process for multiple sinks). In this sense, the sink location, the span of caught tokens, and the resulting tag are all specific to the head.\nWhile some heads may form sinks over overlapping regions, for instance, multiple heads attending to the BOS token, it is also common for sinks to appear at different positions, capturing different subsequences. This enables the model to segment and annotate the input in parallel, with each head providing a distinct structural or semantic signal. To illustrate this more concretely, **the revised manuscript now includes a new figure showing that for a single prompt, two different heads induce distinct sinks corresponding to two different subsequences of the input**.\n\n---\n\n## How do the tags interact with each other? Is there constructive or destructive interference (or something more complicated), or are all the \"tags\" orthogonal?\n\nThank you for this excellent question. In Figure 2(c), each of the R, G, and B channels corresponds to a different top principal component of the attention head output. The emergence of clearly separated red and green coloring indicates that tokens tagged by different sinks project strongly onto different principal components. Since principal components are orthogonal by definition, this **visual separation provides concrete evidence that the corresponding tags occupy distinct, non-overlapping directions in representation space.**\n\nWhile this is only a single example, it is representative of a broader trend we observe across models and prompts: **tags created by different sinks tend to align with orthogonal directions**, suggesting that the model avoids destructive interference by disentangling the subspaces in which tags reside. This supports the interpretation that tags from different sinks are not simply superimposed but are instead composed in a structured, non-interfering manner. To show this, **we have measured the cosine similarity between distinct tags across all layers and heads showing that the tags typically exhibit very small cosine similarity implying that they are close to orthogonal.** This will be displayed in a histogram which we present a summary of in the table below:\n\n| Model | [-1.00, -0.60) | [-0.60, -0.20) | [0.20, -0.10) | [-0.10, -0.05) | [-0.05, 0.00) | [0.00, 0.05) | [0.05, 0.10) | [0.10 to 0.20) | [0.20 to 0.60) | [0.60 to 1.00] |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| Qwen 2.5 3B | 10 | 4405 | 20005 | 44105 | 135895 | 161200 | 83005 | 61085 | 24765 | 290 |\n| Qwen 2.5 7B | 20 | 7595 | 28110 | 59640 | 221350 | 308105 | 159140 | 133025 | 48125 | 545 |\n| Qwen 2.5 14B | 145 | 19445 | 116280 | 357050 | 2286530 | 3065200 | 799030 | 321755 | 68525 | 1055 |\n| Qwen 3 8B | 30 | 5525 | 40680 | 122205 | 690575 | 853650 | 208925 | 88505 | 21095 | 565 |\n| Llama 3 8B | 185 | 18510 | 85120 | 195675 | 935850 | 933545 | 216330 | 128010 | 53995 | 735 |\n| Llama 3.1 8B | 160 | 17080 | 79440 | 192825 | 914310 | 916240 | 213520 | 121045 | 47350 | 530 |\n| Deepseek Llama 8B | 150 | 14810 | 70610 | 186825 | 985025 | 965665 | 200615 | 100995 | 32755 | 380 |\n| Deepseek Qwen 7B | 60 | 1245 | 6380 | 21075 | 158700 | 288795 | 161495 | 111735 | 26150 | 1335 |\n---\n\n---\n\n## It is especially confusing since in Figure 2 the tokens later in the network seem to adhere strongly to the tags produced by the single discussed head earlier in the network, whereas multiple heads surely were attention sinks.\n\nWe appreciate the reviewer’s observation and agree that further clarification is warranted. While Figure 2(a) illustrates the behavior of a single attention head exhibiting a specific sink structure, in this particular example, **other heads in the layer independently converge on the same sink structure**. This results in the tags from multiple heads reinforcing one another and producing a coherent clustering pattern deeper in the network, as shown in Figure 2(d).\n\nTo clarify this, we have revised the caption of Figure 2(d) to read:\n>Tagged tokens, created by multiple attention heads that independently exhibit the same sink structure, propagate through the residual stream and cluster in a deeper layer based on their associated sink.\n\nHowever, we emphasize that this **alignment across heads is not guaranteed and does not always occur**. To illustrate a contrasting case, we have **added a new figure where different heads create distinct sinks, resulting in more heterogeneous tagging across heads**. We hope this comparison clarifies that the example in Figure 2 is representative of one possible, and particularly clean, case rather than a general rule.\n\n---\n\n## What is the role of multiple attention heads… in the synthetic setups considered in the work?\n\nWe deliberately designed a simple synthetic task that can be solved using a single attention head to isolate and study the catch, tag, release mechanism. A single head is sufficient in this case because it can form one attention sink and tag a single subsequence starting from a specific source token. However, **this setup limits the model to tagging only one subsequence per source token. To tag multiple subsequences that all begin at the same token but end at different positions, multiple attention heads are needed -- each forming its own sink and injecting a distinct tag**. For instance, a more complex variant of the task could require the model to extract and average several subsequences (e.g., tokens 3–7, 3–15, and 3–28, all starting from token 3) and then combine those averages. This would naturally encourage the use of multiple heads, each capturing and processing a different subsequence.\n\n---\n\n## There is little-to-no discussion of applications to the practice of LLMs.\n \nWe have added the following paragraph discussing various applications of the mechanism:\n\n> The presence and utility of the `catch, tag, and release’ mechanism have important implications for practical use of large language models. Section 4 shows the potential for tags to be used to steer activations [1] which can be applied for model alignment and safety [2]. In prior work [3], attention sinks were linked to repeated token phenomena and jailbreaking, suggesting the potential for the mechanism to be used for both exploit detection and mitigation. In model compression, preserving this mechanism may be critical, as we hypothesize it can be captured within a low-rank subspace of the model’s parameters, which must be retained during pruning or quantization to maintain performance [4,5]. Furthermore, recent work, [6], has identified low-entropy tokens as central to LLM reasoning abilities, likely overlapping with the tag tokens we highlight, indicating this mechanism may play a foundational role in LLM reasoning.\n\n---\n\n## What is the role of other non-attention components in the network (MLP, ...) in this mechanism?\n\nWe have **added the following paragraph to discuss the potential role of the MLP in the transformer layers in light of the catch, tag, release mechanism**:\n\n> Prior theoretical work studying transformers from a dynamical systems perspective has shown that groups of token representations tend to collapse to low-dimensional subspaces, or even single points, across layers [7]. This raises the question: which tokens collapse together, and what determines the target of this collapse? We hypothesize that the catch, tag, release mechanism plays a central role in both the grouping and the collapse target. Specifically, tokens attending to the same attention sink receive a common tag, which defines the direction and destination of their collapse. We further **propose that the MLP layers progressively drive the collapse across layers by reinforcing the shared tag structure, effectively denoising the token representations toward the shared tag**.\n>\n> We also envision this mechanism being used in practice by researchers and practitioners to better understand and intervene in model behaviour. For example, one could input a prompt, extract the tags assigned to tokens, and highlight, for each tag, the tokens grouped by it. This would reveal how the model segments and jointly processes subsequences of input, offering a transparent window into its internal computation.\n\n---\n\nWe sincerely thank the reviewer once again for highlighting areas of ambiguity and missing discussion in our manuscript. If the reviewer feels that our revisions have sufficiently addressed these concerns, we would be grateful for any reconsideration of the score.\n>\n\n[1] Turner et al. (2023). Steering Language Models With Activation Engineering.\n\n[2] Wang et al. (2023). Trojan Activation Attack: Red‑Teaming Large Language Models using Activation Steering for Safety‑Alignment.\n\n[3] Yona et al. (2025). Interpreting the Repeated Token Phenomenon in Large Language Models.\n\n[4] Zhang et al. (2024). OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition.\n\n[5] Makni et al. (2025). HASSLE-free: A Unified Framework for Sparse plus Low-Rank Matrix Decomposition for LLMs.\n\n[6] Wang et al. (2025). Beyond the 80/20 Rule: High‑Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning.\n\n[7] Geshkovski et al. (2023). A mathematical perspective on Transformers."}]

Author rebuttal:

Meta-review: The paper argues that attention sinks implement a three-step mechanism: catch, tag, release. Sink tokens attract attention, copy their value vectors as tags into many tokens, and those tags later guide computation by the residual stream. The claim is supported by a variance-explained metric, visualizations, semantic probes, cross‑family coverage, and a minimal constructive theory. Reviewers asked for deeper rigor around sink detection thresholds, multi‑head interactions, and the link to reasoning. The authors responded with additional analyses (threshold sensitivity, sink‑token taxonomy, QK-norm probes on Qwen-3), sentiment probes, orthogonality across heads, and a new multi‑head synthetic experiment. The reviews converged to an overall positive.
Decision: Accept (poster)