Looking Beyond the Top-1: Transformers Determine Top Tokens in Order
Authors: Daria Lioubashevski
Tomer M. Schlank
Gabriel Stanovsky
Ariel Goldstein
Venue: ICML 2025 poster (2025)
Keywords: mechanistic interpretability
transformer
large language model
efficient inference
Primary area: deep_learning->large_language_models

Abstract: Uncovering the inner mechanisms of Transformer models offers insights into how they process and represent information. In this work, we analyze the computation performed by Transformers in the layers after the top-1 prediction remains fixed, known as the “saturation event”. We expand this concept to top-k tokens, demonstrating that similar saturation events occur across language, vision, and speech models. We find that these events occur in order of the corresponding tokens’ ranking, i.e., the model first decides on the top ranking token, then the second highest ranking token, and so on. This phenomenon seems intrinsic to the Transformer architecture, occurring across different variants, and even in untrained Transformers. We propose that these events reflect task transitions, where determining each token corresponds to a discrete task. We show that it is possible to predict the current task from hidden layer embedding, and demonstrate that we can cause the model to switch to the next task via intervention. Leveraging our findings, we introduce a token-level early-exit strategy, surpassing existing methods in balancing performance and efficiency and show how to exploit saturation events for better language modeling.

Looking Beyond the Top-1: Transformers Determine Top Tokens in Order
Daria Lioubashevski
Tomer Schlank? 2
Gabriel Stanovsky' Ariel Goldstein 34
Abstract
(Zhang et al., 2023; OpenAl et al., 2023). This has lead to growing interest in model interpretability. which tries to explain the internal processes that give rise to these remark- able capabilities. In the language domain, investigation into the way the model's predictions are constructed has led to the discovery of saturation events, where the model's top-1 I prediction determined in an early layer and remains fixed in subsequent layers (Geva al., 2022).
Uncovering the inner mechanisms of Transformer models offers insights into how they process and represent information. In this work, we analyze the computation performed by Transformers in the layers after the top-1 prediction remains fixed, known as the "saturation event". We expand this concept to top-k tokens, demonstrating that simi- lar saturation events occur across language, vision, and speech models. We find that these events oc- cur in order of the corresponding tokens' ranking, i.e., the model first decides on the top ranking token, then the second highest ranking token, and so on. This phenomenon seems intrinsic to the Transformer architecture, occurring across differ- ent variants, and even in untrained Transformers. We propose that these events reflect task transi- tions, where determining each token corresponds to discrete task. We show that it is possible to predict the current task from hidden layer em- bedding, and demonstrate that we can cause the model to switch to the next task via intervention. Leveraging our findings, we introduce a token- level early-exit strategy, surpassing existing meth- ods in balancing performance and efficiency and show how to exploit saturation events for better language modeling.
In this work, we investigate the following question what compulation is the Transformer model performing after the saturation event? Taking inspiration from Frydenlund et al. (2022), we treat the model's output as a ranking over the labels instead of a probability distribution. Using the logit lens (Nostalgebraist, 2020), we project the hidden states of intermediate layers onto the vocabulary space extract ranking over tokens and analyze the changes over the layers. For the first time, we show that in all tested decoder-only text Transformers (Llama3-8B, GPT2-XL, Mistral-7B, and Falcon- 7B) saturation events also take place for the top ranking tokens beyond the top-1 (2nd, 3rd, 4th, etc.).
Surprisingly, we find that they happen in order of their ranking, i.e. the second-ranking token is determined only after the first-ranking token, and so forth (see in Figure 1). We then generalize the results across different modalities and Transformer variants, including pretrained Transform- ers for both vision (encoder-only ViT-L/16) and speech (encoder-decoder Whisper-large). Next, we demonstrate that sequential saturation seems intrinsic to the Transformer architecture, occurring even in an untrained randomly ini- tialized model (Llama3-8B)
1. Introduction
In recent years, Transformer-base models (Vaswani et al., 2017) have achieved state-of- of-the-art performance in vari- ous tasks across multiple modalities, including text genera- tion, image classification, and automatic speech recognition
We propose that this phenomenon is due to discrete task- Iransition mechanism, where each task i corresponds to the model determining the i-th token in the final ranking, and the transition between one task and the next happens at the corresponding saturation layer. Furthermore, we claim that the task information is encoded in the layer embeddings and that at each saturation layer, a discrete "switch" is flipped, signaling that the relevant token has been determined, caus- ing the model to move on to the next task while keeping this token fixed in subsequent layers. To support this, we show that it is possible to predict the task index from the layer embeddings using a simple logistic regression classifier, and that we can cause the model transition from the first to
School of Computer Seience and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel Department of Math- ematics. The University of Chicago, Chicago, US Bussinss School, The Hebrew University of Jerusalem, Jerusalem, Israel "Department of Cognitive and Brain Sciences, The Hebrew Uni- versity of Jerusalem, Jerusalem, Isracl. Correspondence to: Daria mai..huji.ac.ll.
Lioubashevski dlaria.lioubasheve
Proceedings of the 4nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Transformers Determine Top Tokens in Order
2
4
5
128K
toy
bell
ribbon
kitten
bell
5
32
8
8
5th saturation layer
31
Task
8
8
8
4th saturation layer
29
28
Task
27
3d saturation layer
26
Task
2d saturation layer
22
Task
1st saturation layer
18
Task
Figure 1. An illustration of the proposed task transsranition mechanism wherein the layers of the Transformer perform a changing number of tasks in order, so that task i is determining the i-th token the final ranking, and the transition between task and task + occurs at the corresponding i-th saturation layer. The transition is akin to switch being flipped "on" and staying "on" for the remaining layers representing the i-th token being fixed from this point onward.
a discrete task-transition mechanism, encoded in the rep- resentation of hidden layers where each task corresponds to determining the next ranking token. We empirically ings of Transformers, also demonstrate the potential applications of our finding. As proof of concept, show that this new understanding of the Transformer lends itself to improving inference efficiency. For this purpose, we introduce a novel token-level early-exit strategy for text generation, where the computation halts after the transition from task to task 2, presuming the top-1 token is finalized. This method outperforms existing early-exit strategies (e.g.. boftmax-responsee Schuster et al., 2022) in balaneing com- putational efficiency and performance. In addition, we show that we can use task information to achieve more accurate language modeling, by demonstrating that in cases where the top-1 prediction is incorrect, the second highest ranking token represents much more accurate prediction when reaches saturation than when does not.
the second task by "injecting" embeddings from either the top-1 saturation layer or of one the subsequent layers.
While our primary focus is on uncovering the inner work-
show that it is possible to predict the task index only from internal activations, and that to switch from one task to the next via an intervention procedure.
cause the model
We demonstrate that our findings have practical implica- tions by introducing a new token-level early-exit strategy for text generation, as well as insights into better language modeling based on saturation.
By identifying ordered saturation as modality-agnostic phenomenon, we take step toward developing inter- pretability frameworks that generalize beyond domain- specific observations.
The code for our experiments is available at: https:://gituub.comm htt.m//drraaaaaaaaiiiiaaaa beyond.topl.
Our main contributions are:
We find that Transformers tend to decide their top ranking tokens in order, so that the top ranking token is fixed first, then the second-ranking token at later layer and We show that this occurs across various modalities and variants of the Transformer architecture, and even in untrained randomly initialized models.
2. Experiments
In this section, we first extend the formal definition of top-1 saturation to account for arbitrary i-th ranking token satura- tion (Section 2.1). Building on this, we formulate two ex- periments to understand what computation the Transformer performs in the layers after the top-1 saturation event. The
We show that sequential saturation can be explained with
2

---

Transformers Determine Top Tokens in Order
Llama3-8B
Projection
Sort
Ranking over vocabulary
toy
2
3
128K
Layer 32
E
toy
ball
ribbon
kitten
toy
Layer 31
E
toy
ball
rbbon
kitten
toy
Layer 30
X
E
toy
bell
ribbon
piece
toy
Layer 29
E
loy
paw
dog
toy
Layer 28
toy
mouse
leash
stray
new
Layer
E
new
huge
little
?
Token and positional encoding
The
cat
played
with
Figure 2. Schematic of our framework and visualization of the ordered saturation of the top-k tokens on Llama3-8B. The hidden states from each layer are projected onto the vocabulary space using unembedding matrix E, then sorted in descending order and treated as rankings. The saturation effect is marked separately for each token in the top-4 of the final ranking, emphasizing the fact that the 2nd token saturates after the token, the token saturates after the 2nd token and so on. The dashed line represents the previously established saturation event of the top-1 token.
first experiment (Section 2.2) leverages definition to develop metric capturing the extent to which top tokens are saturated in order. The second experiment (Section 2.3) uses a probing approach to test whether it is possible to determine the rank of the token currently being determined by the model solely from intermediate layer activations.
following layers l' s.t. 1 < l' S N the token in position k in the ranking induced by that layer remains unchanged. We note that the saturation event defined in (Geva et al., 2022) happens at 1t, For example, in Figure 2, 12 = 28 as that is where the top token ("toy") ") determined; 12 = 28, since the second-mos, probable token ("ball")) determined at layer 29; and similarly 1 30, since the third-most probable token ("ribbon"") is determined layer 30, etc.
2.1. Defining Saturation Layers
Definition 2.1 (Ist Saturation Layer; Geva et al., 2022). The saturation event occurs at layer (from here on referred as the "Ist saturation layer") for index in the input the top-1 prediction of the model remains constant in all subsequent layers after L. Formally, given model with N layers, saturation event occurs at layer| I < N - 1 if for all layers !' s.t. 1 < l' < N the top-1 token in the ranking induced by that layer remains unchanged. For example, the saturation event marked with dashed line in Figure 2 occurs on layer 28, since in subsequent layers the top predicted token ("toy") remains constant.
2.2. Examining the Order of Saturation Layers
We investigate whether the saturation layers of the top-k tokens 1 1", are arranged in order, i.e., whether the saturation of the first token happens before the saturation of the second token, the saturation of the second token happens before the saturation of the third token and so on'. To this end, for each token in the input we first calculate these sat- uration layers for A = I,..., 5 according to Definition 2.2 and then for each compute the rank of the saturation layer of the k-thh top token. We use k = 5 to ensure consistency across models, it is the highest value of k where the k- th token reaches saturation in at least 5% of input tokens for all the models analyzed. Following our example from Figure 2, we have 1 28, 2 = 29, 13 = 30, 2 = 31,2
Definition 2.2 (k--th Saturation Layer). Here, we are inter- ested in examining model behavior beyond the determina- tion of the top ranking token and naturally extend the definition of top-1 Saturation (Definition 2) to capture the layer at which the k-th top token is determined and remains fixed. Formally, the saturation event for the k-th top token at index the input occurs at layer & < N - 1 (from here on referred to as the k-th saturation layer") for all
"I all of our experiments. we only consider tokens in the input where the 1st saturation layer satisfies that 1 < 0.85 N, to ensure that there are enough layers after for meaningful analysis. 2/5 is ill-defined in this case as the 5-th token doesn't 't reach saturation before the last layer.
3

---

Transformers Determine Top Tokens in Order
and their ranking is [1, 4), since 1 < 12 < 1 < 1.. If the tokens reach saturation in order of ranking, as they do in this case, we would expect the average rank of the saturation layers to increase monotonically with k. To sta- tistically validate this phenomenon we use a stricter version of Kendall's t coefficient that treats ties as disagreements, allowing us to quantify the alignment between the token ranking and the ranking of their corresponding saturation layers (see Appendix A.3).
embeddings.
3. Results
To show the robustness of our findings, we test pretrained Transformer models on corresponding datasets from three modalities: text, vision and speech.
Top-1 saturation events are common. Saturation events are quite common, with 10% to 60% of all input tokens reaching top-1 saturation, according to our defintion, in all models tested (see Appendix A.2 for more statistics).
2.3. Probing for Task Transition
We argue that the mechanism underlying the saturation of the top-k tokens in order is one of task transition, such that determining the identity of each token in the final ranking is a separate task, and the model performs them sequentially: first determining the identity of 1st token, then the identity of the 2nd token, and so on, and that the transition from one task to the next occurs at the corresponding saturation layer. Importantly, we do not claim that the model does not process token i before task i, rather it may begin accu- mulating evidence for token i earlier, but the token is fixed at the end of the task at the corresponding saturation layer. Additionally, we claim that the specific task number can be inferred from the model embedding at each layer, and that this information independent of the context or the specific token predicted by the model.
3.1. Text Transformer
We conduct our experiments to examine the order of satu- ration in a wide range of LLMs: GPT2-XL (Radford et al., 2019), 8-bit quantized versions of Llama3-8B (Wendler et al., 2024), Mistral-7B (Jiang et al., 2023) and Falcon- 7B (Almazrouei et al., 2023). We employ a set prompt to present each task's questions, answer choices, and correct answer, ensuring uniform input structure (see Appendix A.1 for prompt details).
Tokens reach saturation in order of ranking. We use IK randomly sampled questions from MMLU test split which represent 60K-100K tokens (depending on the model). In all tested models, we find that tokens tend to reach saturation in order. As an example, Figure 3a shows the average rank of the k-th saturation layer for each k for the Llama3-8B model. This value increases monotonically with k and the difference between each two consecutive token ranks is statistically significant with p < 0.001 based on a pairwise independent samples t-test. All other models show a similar trend, see Appendix A.4.
To test our hypothesis, we perform type of probing in which we train simple one- velsus-lll multi-class logis- tic regression classifier to predict the number of the task the model is "working on" from the hidden layer embed- dings. We collect the data for training by extracting the model's hidden states during inference and categorize them into 5 classes according to the saturation layers of the top-5 tokens for each instance. This means that for given input, embeddings from layers up to (and including) the 1st satura- tion layer are classified as belonging to task 1, embeddings from layers from the next layer until the 2nd saturation layer are classified as belonging to task 2, and so forth.
Task number can be predicted from model embeddings. After extracting embeddings from 500 randomly sampled questions we split the data into train and test using 5-fold cross validation, and report the mean and standard error of the accuracy. Table shows that the logistic regression classifier trained on embeddings extracted from pre-trained Llama3-8B model achieves very high accuracy, while the classifier trained on the random embeddings in the control setting performs approximately at chance level (see Ap- pendix A.7 for accuracy and ROC-AUC scores per class). From this we conclude that the representations of the hidden layers across examples encode task specific information and that the saturation layers as we defined them are the points of transition between those tasks. The same qualitative results are observed for all other models, see Appendix A.5.
For example, in the case of the token "a" as depicted in Figure 2, the embeddings from layers through 28 would be classified helonging to task 1; the embedding of layer 29 would be classified as belonging to task 2; the embedding of layer 30 as belonging to task 3; the embedding of layer 31 as belonging to task 4, and as the model reaches the last layer directly afterward there would be no embedding classified as belonging to task 5.
We balance the training data to ensure equal representation of embeddings from all layers in each class. To confirm that task encoding arises from the model's embeddings rather than the classifier's weights, we create a control setting by generating random vectors for each class, matching the dimensions, mean, and variance of the corresponding layer
'The number of tasks per model is set to the maximum where data balancing yields at least 10 embeddings per class from at least 4 different layers.
4

---

Transformers Determine Top Tokens in Order
saluration layer rank
Salur lian layer nank
(a) Pre-trained Llama3-8B (decoder only)
(b) ViT-L/16 (encoder only)
saluration layer rank
2-
Salure e lion layer rank
rank
Whisper-large (full- colll-Trnsformer)
(d) Randomly initialized Llama3-8B
Figure 3. Average rank of the k-th saturation layer among the saturation layers for k=1....5 with standard error bars. Asterisks indicate statistically significant differences between consecutive token ranks (*** represents < 0.001), based on an independent samples t-test.
3.2. Vision Transformer
3.3. Speech Transformer
Encoder-only image-laasiffiattin ViT' (Dosovitskiy al., 2021) take input sequence of linear projections of equal- sized image patches with added position embedding and special "class token" denoted [CLS]. Following the work of (Vilas et al., 2023) we use version of the logit lens adapted to ViT to project the hidden state representations of each layer in the encoder onto the class embedding space using the output embedding matrix. Importantly this done only for the [CLS] token for each image under the assumption that it best represents the model's prediction, since during ViT's pretraining the only token projected onto the class- embedding space is the [CLS] token from the last layer.
Whisper (Radford al., 2023) encoder-decoder Trans- former model trained many different speech processing tasks, including ASR. Although recently there have been attempts to increase efficiency in ASR, such as Malard et al. (2023), the concept of early exit has yet to be explored in this setting, and to best of our knowledge there has not been work done concerning saturation events in speech models. We adapt the logit lens and apply it only to the decoder stack of Whisper-large under the assumption that representations in the encoder stack are inherently different and projecting them onto the token vocabulary space would not be meaningful. For our dataset we randomly sample 5K audios from LibriSpeech (Panayotov et al., 2015).
For our experiments we use the ViT-L/16 variant pretrained on ImageNet-22 21k and fine-tuned on ImageNet 2012, and run inference 5K randomly sampled images from the CIFAR- 10 (Krizhevsky al., 2009) dataset. Figure 3b demonstrates the high correspondence between saturation layer and token rank, supporting our claim that in this domain well as in text the saturation layers are highly ordered. Furthermore, Table shows that the task index can be predicted from the hidden layer activations with high accuracy well above chance and control setting. We further validate findings in multi-modal setting using the vision-language model LLaVa (Liu et al., 2023) the MMMU (Yue et al., 2024) visual question answering benchmark, see Appendix A.8.
In addition reproducing for the first time in speech the classical top-1 saturation event established in language and vision models in previous work, we also show in Figure 3c evidence for the tendency of the top-k tokens to reach saturation in order in this model as well, albeit only to the third token. We suspect that the deterioration order for later tokens arises from the fact that all decoder layers are conditioned via cross- atteentitt on the final layer of the encoder. This global conditioning may interfere with the task transition mechanism by "blurring the lines" between tasks, effectively entangling task-specific representations. Supporting this hypothesis, in Appendix A.9 we replicate
5

---

Transformers Determine Top Tokens in Order
Table 1. Accuracy of task number logistic regression classifier showing that in all modalities the layer embeddings contain in- formation about the task number. Asterisks indicate statistically significant accuracy (* represents p < 0.001), based on an Bi- nomial Distribution probability test. stands for pre-trained, R stands for randomly initialized.
Model top- artist
Model top- artist
0
32
Model
Layer Emb.
Random Emb.
Chance Level
27
LLaMA3-8B (P)
88.1" + 0.4
33.1 + 0.5
33.3
21
21
LLaMA3-SB (R)
79.2" + 0.3
34,1
+ 0.4
33.3
Set Value cf Activations
ViT-L/16
63.8" 0.1
21.0
+ 0.5
20.0
Whisper-large
52.7" 0.1
0.1
24.5 + 0.4
25.0
20
20
1
0
Britain's most wanted artist .. taken a photo of the artist.
our findings using Qwen-Audio (Chu et al., 2023), an audio model in which the output of the audio encoder is used only as input the first decoder layer, and observe that the ordered saturation persists up to the 4th token. Even so, Table shows that the task index can be predicted from Whisper's decoder layers' embeddings with accuracy much higher than chance or that achieved in the control setting.
Figure 4. By injecting the output from the top-1 saturation layer of "the" as input to the subsequent layer "artist", we trigger a saturation at the injected layer (21) in the post-intervention run, without altering the top-1 prediction. Saturation layers are marked in bold, saturation layer in the original run for "the" token marked with dashes. The use of activations from adjacent layers is not depicted for the sake of clarity.
4. Analysis
4.2. Injecting Saturation Layer Activations Causes Task Switch
We have shown that top-k tokens tend to reach saturation in order of their ranking, well the plausibility of the underlying task transition mechanism over multiple modal- ities and variants of Transformers: decoder only, encoder only and full Transformer; in section 4.1 we argue that this phenomenon is inherent to the architecture itself and sec- tion 4.2 we delve deeper into the way the model transitions between tasks, demonstrating that we can cause the model to switch to the next task using an intervention procedure.
Using the probing analysis, we demonstrated that the tasks, as we defined them, are distinct enough to be separated by a simple classifier, that saturation layers mark the boundaries between them, and that the task index is encoded in the hidden layer embeddings. We argue that in addition, each saturation layer encodes the signal to transition to the next task, and all subsequent layers contain the information that the previous task has been completed and that the relevant token is fixed. This can be thought of as switch being flipped "on" for each token that reaches saturation, and remaining "on" from the saturation layer onwards.
4.1. Untrained Transformers Also Determine in Order
We repeat our experiments an untrained Llama3-8B with randomly initialized weights on the same amount of ran- domly sampled questions from MMLU dataset as with the pretrained model. Surprisingly, Figure 3d shows that the top- k tokens tend to reach saturation in order up to the 3th token. In addition, Table shows that the task transition classifier's accuracy is more than 2r times higher than chance or that of the control setting. The ability of the classifier to extract the task index from the hidden layers' representations in this setting is especially remarkable, demonstrating that despite the randomness of the weights as well of the identities of the predicted tokens, there is still highly ordered information encoded in the model originating only from the constraints of the architecture.
To causally validate this claim, taking inspiration from Stolfo et al. (2023), we perform an intervention (visual- ized in Figure 4) in which we "inject" the output from the 1st saturation layer of sample $1 as input into the subsequent layer in the run on sample 82 and check how this affects the 1st saturation layer of s2- If these activations contain the signal switch to the next task, we expect this intervention to cause the model to switch task 2 at the injected layer in the post-intervention run, i.e. in the new post-intervention run the 1st saturation layer should be the one on which the intervention is performed which is I' (s1) + 1. To minimize the effect of confounding factors, we choose pairs of sam- ples 81 and 82 that share context and where the original
6

---

Transformers Determine Top Tokens in Order
% of examples with task switch
50
50-
50
50
40-
30
30
30
20
20
10
-3 -2
2
Distance of injected layer from original saturation layer
(a) Pretrained Llama3-8B
(b) VIT-L/16
(c) Whisper-large
Figure Plipping the Top-/ Switch, The percentage of examples where the top-1 saturation occurred at the injected layer after the intervention, shown function of the layer from which the injected activations taken, relative to the original saturation layer (e.g.. -2 means activations were taken from two layers before the original saturation layer).
top-1 prediction of the model is the same, but there is big difference in their 1st saturation layers s.t. L' (s1) < (52).
stark difference in the effect the injected activations have on the 1st saturation layer post-intervention when the activa- tions are taken from the st saturation layer in the original run or one of the following layers, compared to the layers before it. When the injected activations are taken from an earlier layer, the new top-1 saturation almost never occurs at the injected layer, whereas when the injected activations are taken from the saturation layer or a later layer the top-I saturation occurs at the injected layer in high percentage of cases. This drastic change resembles step function, and is in line with our description of a switch being flipped "on" at the 1st saturation layer and remaining on in all subsequent layers, indicating to the model to switch to the next task and keep the top-1 constant. These results are consistent across range LLMs, shown in Appendix A.1I.
In the example depicted in Figure 4 82 "the", and for both the model's top-1 next-word prediction is "artist", but (s1) = 20 while 1' (s2) = 27. Injecting the output of layer (s1) into the subsequent layer (21) in the run for 82 should cause the model to switch task 2, resulting in layer 21 being the 1st saturation layer post intervention.
"wanted" and
Moreover, we would expect the same thing to happen when injecting activations from a layer I after the 1st saturation layer, i.c. (s1), since they should contain the infor- mation that the top-1 token fixed. On the other hand, activations from layer l' before the 1st saturation layer, i.c. P' < 1 (s1) should not result in saturation the injected layer the switch is still "off" in our analogy, indicating to the model that it still working on task 1. To test this, we repeat the same steps with activations from 3 layers before and after the 1st saturation layer [ (s1) - 3, 1 (s1) + 3] each time injecting them as input into the subsequent layer.
5. Practical Implications
While the primary focus of this work is on understanding the mechanisms underlying saturation events, we demon- strate how these insights can inform practical strategies for improving computational efficiency and language modeling.
Figure 5(a) shows the results of this procedure performed using pre-trained Llama3-8B 200 token pairs taken from 10 randomly sampled texts from the CNN/DM dataser (Hermann et al., 2015), Figure 5(b) shows similar results reproduced using ViT-L/16 on 200 pairs of images from CIFAR-10 dataset, and Figure 5(c) shows the results of the intervention on Whisper-large on 200 token pairs from 100 randomly sampled audios from LibriSpeech'. There
5.1. New Early-Exit Strategy
We propose a new token-level dynamic inference method based on the bask-transition classifier described Section 2.3, where the early exit layer for each token is defined the earliest layer which is predicted belong task 2 by the classifier. The idea being that once the model has transitioned into the second task, it has finished with the first task of determining the top-1 token. To demonstrate the viability of this method, we compare it to two other local confidence measures introduced by Schuster al. (2022): softmax response (the difference between the top two values
*We use texts from CNN/DM not MMLUU this exper- iment as they tend to he longer and have more pairs that fit our criteria for intervention
"See Appendix A.10 for a formal description the procedure as well as adaptations for vision and speech modalities

---

Transformers Determine Top Tokens in Order
of the logits after softmax) and hidden-state saturation (co- sine similarity between two consecutive layer embeddings), both recently found to be competitive with other early exit- ing methods (Zhou et al., 2024). Since dynamic decoding is not the focus of this paper, we calculate the metrics for each measure while propagating states from the layers after the "early exit" as in regular inference. 6
Table 2. Highest accuracy and corresponding speedup-ratio achieved by each early-exit strategy. Asterisks indicate statis- tically significant accuracy (" represents p < 0.001). based on an independent samples t-test.
Softmax Re-
State Satura- tion
Ours
sponse
Table 2 shows our results on a pre-trained Llama3-8B model and 100 randomly sampled texts from CNN/DM dataset, showcasing the ability of our probing classifier to generalize across datasets as it was trained on texts from MMLU bench- mark (see Section 3.1). We evaluate the model on next-word prediction, and compute the speedup ratio for each method as the total number of layers in the model divided by the number of layers it uses for each token." 7 For the two other confidence measures we caleulate these metries at various thresholds (see details in Appendix A2) while in our mea- sure the class is selected based on the highest predicted score among all classes. Our strategy outperforms the other two when considering the trade-off between next-word pre- diction accuracy and speedup ratio, and requires no training besides that of simple logistic regression classifier on relatively small amount of data. We find the difference in speedup between our strategy and the other methods to be statistically significant (p < 0.001) using an independent samples t-test.
Accuracy Speedup Ratio
40.6+t 0.6 1.07 0.002
24.3 0.4
43.3 + 0.7
1.001 0.000
1.100
cates statistically significant difference between the groups (p < 0.001) even when correcting for multiple comparisons for 2 S i S 6 in the 2nd and 3rd tokens, and for and 2 H i I 5 in the 4th token. This supports our claim that top-k tokens that reach true saturation are more plausible predictions than those that are determined in the last layer, which has potential implications for decoding strategies which consider tokens beyond the top-1.
6. Related Work
There are many ways of thinking about the role of intermedi- ate layers in Deep Neural Networks (DNNs) in general, and Transformers in particular. The iterative inference hypothe- sis interprets each layer as an iteration from an iterative and convergent process (Simoulin & Crabbé, 2021), suggesting that each layer incrementally refines the hidden representa- tion by gradually shaping the next token prediction (Geva et al., 2022; Belrose et al., 2023; Rushing & Nanda, 2024). We argue that our findings challenge this view, given the discrete nature of the tasks in the proposed task-transition mechanism and the sharp transitions between them.
5.2. Improved Language Modeling
Popular decoding methods in language generation such as top-k (Fan et al., 2018) or top-p (Holtzman et al., 2020b) sample the next token from the top ranking tokens. Based on our task-rrassition mechanism and the assumption that the tasks represent relevant computation, we argue that top ranking tokens that are determined in the last layer repre- sent less meaningful predictions, since the model only had enough layers for the first task in these instances. To test this hypothesis, we compare the accuracy of the second, third, and fourth ranked tokens in the next-word prediction task across two conditions: (I) the token in question reaches saturation i layers before final layer (with 2 < i S 6); (2) the token does not saturate, and is determined only in the last layer. In all cases, we restrict the analysis to examples where the top-1 prediction is incorrect.
Pruning is another approach, focused on mitigating the re- dundancy inherent to large machine learning models by removing unnecessary parameters. Recent work has applied structured pruning methods to Transformer based LLMs, dropping whole modules, from self-attention layers (Ben- Artzy & Schwartz, 2024; He et al., 2024) to full Transformer blocks (Sun et al., 2024; Men et al., 2024). These studies often focus on the middle layers of the model, and claim to reduce memory and computation costs without degrading performance. However, these works evaluate the accuracy before and after pruning based only on the top-1 prediction of the model, even though stochastic generation methods such as top-p (Holtzman et al., 2020a) and top-k (Fan et al., 2018) are preferable to deterministic decoding in certain settings such open-ended tasks, as they produce more coherent and varied text (Shi et al., 2024). In light of this and of our results regarding the sequential saturation of top ranking tokens, we suggest that future work takes this into account, since what may seem as redundancy is actually necessary computation not reflected in the measured metric.
Using 100 randomly sampled texts from CNN/DM dataset and pre-trained Llama-8B predictions, we show in table 3 that in the first condition the accuracy is much higher than in the second condition. A two proportion z-test indi-
"This is an informative comparison between the measures, as the effect of a state copying mechanism for skipped layers on model's performance is negligible (Schuster et al., 2022).
We apply all strategies starting from layer 15, since our classi- fier was trained only on the intermediate layers of the network.
8

---

Transformers Determine Top Tokens in Order
probing results indicate that the relationship between layer depth and computation is more dynamic: the same layer can participate in qualitatively different mechanisms depending on the structure of the task. challenging the view of fixed layer specialization.
Table LLMMA388B: Accuracy (percentage) of next-word predic- tion for the 2nd, 3rd, and 4th ranked tokens when the top-1 1 token incorrect
(comparing no saturation saturation i layers before out- put).
In addition, the emergence of ordered saturation an un- trained Transformer raises fundamental questions about the architectural biases baked into the model. We observe that this behavior not only appears prior to training (potentially influenced by weight initialization) but also persists after pre-training, suggesting a constraint on the model's compu- tation. This opens the door to future work examining how such structural biases influence representational dynamics and probability distributions in language models.
Token rank
No saturation
Saturation layers before output
i
i
i
=
i
=
i
=
2
3
4
5
6
2nd
25.2
38.1
48.6
37.6
37.6
33.0
3d
17.0
23.3
26.9
28.9
29.2
31.0
4th 13.4
16.8
19.7
20.5
19.5
18.3
The logit lens has also been used to study intermediate lay- ers in a wide variety of interpretability papers (Yang et al., 2024; Wendler et al., 2024; Halawi et al., 2024). Despite this, (Belrose al., 2023) claim that it can produce implau- sible results due the difference in representations between layers. To address this issue they introduce the "tuned lens", in which an affine transformation is learned for each layer in the model with distillation loss, so that its image under the unembedding matrix matches the final layer logits as closely as possible. Although this method may be better at approximating final top-1 prediction from intermediate layers, our work highlights why this might actually be disadvantage when attempting to gain insights into the com- putational process of the Transformer, as it could obscure the changing dynamics of the lower ranked tokens.
Limitations While our analysis sheds light on the high-level mechanism behind the ordered saturation of top-k tokens, several questions remain open. First, the precise architec- tural components responsible for this phenomenon are still unclear. Targeted ablation studies could help localize the origin of the effect within the Transformer architecture. See- ond, a more concrete explanation is needed for how the model keeps the "chosen" tokens constant in the layers af- ter the corresponding saturation event. Third, we did not consider whether the model had seen the experimental data during training and how this might affect ordered saturation. Furthermore, since we focused solely on Transformer archi- tectures, it remains to be explored whether other DNN types also determine their top-ranking tokens in order. Recurrent Neural Networks might be of particular interest due to their mathematical equivalence to decoder only Transformers (Oren et al., 2024), and based previous work success- fully applying the logit lens to them to extract meaningful predictions from intermediate layers (Paulo et al., 2024).
7. Discussion
This work contributes to a deeper understanding of how Transformer models construct predictions over layers. We find that the top-k tokens (for k >> 1) go through saturation events in the order of their ranking. This phenomenon is inherent to the Transformer architecture, occurring in untrained models and is robust over multiple modalities. We provide evidence in support of task transition as the underlying mechanism, showing that we can predict task index from the hidden layers' embeddings, as well as cause the model to switch from the first task to the second via intervention procedure. Our findings also show promise in improving LLMs' efficiency and performance as suggested by the preliminary results in Section 5.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There many potential societal consequences of our work, none which we feel must be specifically highlighted here.
A Acknowledgments
We would like thank Nir Rosenfeld, Omri Abend and Mariano Schain for the invaluable insights, as well as Noam Dahan and Timna Wharton Kleinman for their helpful feed- back.
Our findings also carry broader implications for how we understand the internal structure of Transformer models. A common line of work in interpretability of LLMs aims to assign distinet computational roles to different layers, often proposing hierarchical progression of feature com- plexity (Tenney et al., 2019; van Aken et al., 2019; Geva et al., 2021; 2022), suggesting that earlier layers specialize in shallow or syntactic features, while later layers capture more semantic or task sask-spec information. However, our
References
Almazrouei, E., Alobeidli, H., Alshamsi, A., Cappelli, A.,
Cojocaru, R., Debbah, M., Goffinet, É., Hesslow, D., Launay, J., Malartic, Q., et al. The falcon series of open

---

Transformers Determine Top Tokens in Order
language models. ArXiv preprint, abs/2311 2023. URL http:::/p//........ httLhttts:////rae http::////xxvv..rr////p abwpr/gg//b/////// 231 1.1666677.
doi: 10.
URL https: 4446.
/ //acllanthol.gy..rrr22211111111
Belrose, N., Furman, Z., Smith, I.., Halawi, D., Ostrovsky, I,, McKinney, L., Biderman, S., and Steinhardt, J. Eliciting latent predictions from transformers with the tuned lens. ArXiv preprint, abs/2303.08 2023. URL ht https: // //arx/l.crrg/aa//233...000
Geva, M., Caciularu, A., Wang. K., and Goldberg, Y. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Gold- berg, Y., Kozareva, Z., and Zhang, Y. (eds.), Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 30-45, Abu Dhabi, United Arah Emirates, 2022. Association for Compu- tational Linguistics. doi: 10. 10.18653/1//0222.mmmm main.3. URL fttpss//aclanthologyy.. 2022.emm 1 p-mapmmain n.33.
Ben-Artzy, A. and Schwartz, R. Attend First, Consolidate Later: On the Importance of Attention in Different LLM Layers. arXiv, 2024. doi:
Chu, Y., Xu, J., Zhou, X., Yang, Q.. Zhang, S., Yan, Z., Zhou, C., and Zhou, J. Qwen-audio: Advancing uni- versal audio understanding via unified large-scale audio- language models. ArXiv preprint, hts//311.009919 2023. URL https: / www.i..oggaas.///oooooo.... att.abbb.//..........
Halawi, D., Denain, J., and Steinhardt, J. Overthinking the truth: Understanding how language models process false demonstrations. In The Twelfth International Conference Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open 2024. URL https: / / //oppneeview.eee..e wttp://wwwwt//eooueeeeeeeeo -iteiiiddiiigir
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenbomn D., Zhai, X., Unterthiner, T., Dehghani, M., Min- derer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Trans- formers for image recognition at scale. In 9th Interna- tional Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net, 2021. URL https:/ forum? i d=Yi cbFaNTTy.
He, S., Sun, G., Shen, Z., and Li, A. What matters in transformers? not all attention needed. ArXiv preprint, hls/2406.117886 2024. URL atttts:///axivv abs/2406.15786.
Hermann, K. M., Kociský, T., Grefenstette, E.,, Espeholt, I.,, Kay, W., Suleyman, M., and Blunsom, P. Teaching machines to read and comprehend. In Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M., and Garnett, R. (ods.), Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pp. 1693-1701, 2015, URL https::/pproceeddigss.cet paper/2015/hasee a Abstract.html.
Fan, A., Lewis, M., and Dauphin, Y. Hierarchical neural story generation. In Gurevych, I. and Miyao, Y. (eds.), Proceedings of the 56th Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Papers), pp. 889-898, Melbourne, Australia, 2018. As- sociation for Computational Linguistics. doi: 10.18653/ 1MP18-1082. URL. https::: / /acll www.llathhhhoooooooooooo P18-1082
Frydenlund, A., Singh, G., and Rudzicz, F. Lan- guage modelling via learning to rank. In Thirty- Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourte Conference on Innovative Applica- tions of Artificial Intelligence, IAAI 2022, The Twel- veth Symposium on Educational Advances in Artifi- cial Intelligence, EAAI 2022 Virtual Event, February 22 March 1, 2022, pp. 10636-10644. AAAI Press, 2022. URL www.hhttps//t/m /ttp::::////js..a...ooooooo AAAI/ MAAT/articcleevieww///ao http://aaaa....nn........aaa index.pho/
Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y.
The curious case of neural text degeneration. In 8th Inter- national Conference 2020, Addis Ababa, Ethiopia, April 30-30, 2020. OpenRe- view.net, 2020a. URL https: http///ppppeeeeeee...... Dorum?id=ryyG@yyrFe
Learning Representations, ICLR
Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y.
The curious case of neural text degeneration. In 8th Inter- national Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26ril view.net, 2020b. URL https:: //pp/eeeeeeeeeeeeeeeeeeeeeeeeeee ww.et/ forum? d
Geva, M.. Schuster, R., Berant, J., and Levy, O. Trans- former feed-forward layers are key-value memories. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.- t. (eds.), Proceedings of the 2021 Conference on Em- pirical Methods in Natural Language Processing, pp- 5484-5495, Online and Punta Cana, Dominican Repub- lic, 2021. Association for Computational Linguistics.
26-30, 2020. OpenRe-
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. L., Bressand, F., Lengyel, G., Lample, G.,, Saulnier, L.,, et al. Mistral 7b, ArXiv preprint,
10

---

Transformers Determine Top Tokens in Order
abs/2310.06825, 2023. URL SoLttts//axxxiv..rrr abs/231 0.06825.
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. Robust speech recognition via large-scale weak supervision. In Krause, A., Brun- skill, E., Cho. K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), International Conference on Ma- chine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Pruceedings of Machine Learning Research, pp. 28492-285111 PMLR, 2023. URL httpst:s:/ http:////ppooeeee.e.....eeeeeee radford23a.html.
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.
Liu, H., Li, C., Wu, Q., and Lee, J. Visual instruction tuning. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA,, USA, December 10- 2023, 2023. URL http:
Rushing, C. and Nanda, N. Explorations of self-repair in language models. In Forty-first International Conference Machine Learning, ICMI. 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https: openreview.net/Eee tii
p http:::/paaprs..is..com 6acc2777a32c3388551ffa33886888 Abstrract-Conferencelll
Malard, H., Zaiem, S., and Algayres, R. Big model only for hard audios: Sample dependent whisper model selection for efficient inferences. ArXiv preprint, Abs/2309.12712 2023. URL httpp:://axxi..rrt. abs/2309.12713
Schuster, T., Fisch, A., Gupta, J., Dehghani, M., Bahri, D., Tran, V., Tay, Y., and Metzler, D. Confident adaptive language modeling. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Infor- mation Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November December 9, 2022, 2022. URL http: http://papers.niis..t paper_ harrffllee/paper2222//baas (Eccle3llalaae35e2224deee
Men, X., Xu, M. Zhang, Q. Wang, B., Lin, H., Lu, Y., Han, X., and Chen, W. Shortgpt: Layers in large language models are more redundant than you expect. ArXiv preprint, abs/2403.03853, 2024. URL https: //arxiv.crg/abbbea 200.ala/b.222000005.533
Nostalgebraist. Interpreting gpt: the logit lens, 2020.
Shi, C., Yang, H., Cai. D., Zhang, Z., Wang. Y., Yang. Y., and Lam, W. A thorough examination of de- coding methods in the of llms. ArXiv preprint, abs/2402.06925, 2024. URL htttth:://aaiiweeee abs abs/2402.06925.
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L.., and et al. Gpt-4 technical report, 2023. URL https: //arxiv.org. abs 2303008774.
Oren, M., Hassid, M., Adi, Y., and Schwartz, R. Transformers are multi-state rnns. ArXiv preprint, abs/2401.06104,, 2024. URL https:/ http:://axxvvvvroeee abs/2401.05104.
Simoulin, A. and Crabbé, B. How many layers and why? An analysis of the model depth in transform- ers. In Kabbara, J., Lin, H., Paullada, A., and Vam- vas, J. (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1th International Joint Conference on Natural Language Processing: Student Research Workshop, pp. 221-228, Online, 2021. Association for Computational Linguis- tics. doi: 122116553/112021.aclll 23. URL https:
Panayotov, V.,
Chen, G., Povey, D., and Khudanpur, S. Librispeech: An ASR corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2015, South Brisbane, Queensland, Australia, April 19- 24, 2015, pp. 5206-5210. IEEF, 2015. doi: 10.1 ICASSP 201....7905.7778666 URL ptteps:/ddo.orr/ M.L1109/ICASSO 20155.178854..
/ //aclaatthoooory.rrg//00.....
Stolfo, A., Belinkov, Y., and Sachan, M. mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis. In Bouamor, H., Pino. J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7035-7052, Singapore, 2023, Association for Com- putational Linguistics. doi: 10. (0.18653/v1/20233.cm main.435. URL https:
Paulo, G., Marshall, T., and Belrose, N. Does trans- former interpretability transfer to rans? ArXiv preprint, abs/2404.05971,, 2024. URL https: htttps://arxiv.or/ arxiv.org/ abs/2404.05971.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language models are unsupervised multitask
learners. OpenAl blog, 1(8):9, 2019.
2023.emm i

---

Transformers Determine Top Tokens in Order
Sun, Q., Pickett, M., Nain, A. K., and Jones, L. Transformer layers painters. ArXiv preprint, abs/2407.002988, 2024. URL ptttps://axii...t
abs/2402.16837, 2024. URL https:://aaxii.rrrt abs/2402.16837.
abs/ 2ttps/2407.000 2407.09298.
Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B.. Yang. Z., Liu, Y., Huang, W., Sun, H., Su, Y., and Chen, W. MMMU: A massive multi-discipline multi- modal understanding and reasoning benchmark for ex- pert AGI. In IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pp. 9556-9567. IEEE, 2024. doi: 10. 10.1109CCVPP3773.2224000000 URL https: / //00l..rg/10.1100////773333332
Tenney, I., Das, D., and Pavlick, E. BERT rediscovers the classical NLP pipeline. Korhonen, A., Traum, D., and Mårquez, L.. (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4593-4601, Florence, Italy, 2019. Association for Computational Linguistics. doi: 10. 11186661111111155555 URL httph:tpp///cmc htt///aaaantcogoy.yooooom
van Aken, В., Winter, B., Löser, A., and Gers, A. How does BERT answer questions?: layer-wise anal- ysis of transformer representations. In Zhu, W., Tao, D., Cheng, X., Cui, P., Rundensteiner, E. A., Carmel, D., He, Q. and Yu, J. X. (eds.), Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019, pp. 1823-1832. ACM, 2019. doi: 10.114553357344.355585 URL hwwph:ttps..tt. //tpsoor///o./gg///// 10.1145/3357384.0
Zhang, Y., Han, W., Qin, J., Wang. Y., Bapna, A., Chen, Z., Chen, N., Li, B., Axelrod, V., Wang. G., et al. Google usm: Scaling automatic speech recognition beyond 100 languages. ArXiv preprini, abx/2303.01037, 2023 URL betps://arxiv..r
abs /ttp///aaab/33300333.3
Zhou, Z., Ning. X., Hong. K., Fu, T., Xu, J., Li, S., Lou, Y., Wang, L.,, Yuan, Z., Li, X., et A survey on efficient inference for large language models. ArXiv preprint. abs/2404.14294, 2024. URL abs abs/2404.14299
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, V. N., and Garnett, R. (eds.), Advances Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA. USA, pp. 5998-6008 2017. URL https: https://prooeecings..errr.ppp
paper/2017/hasi/ 3tte222$5776e2111o003aacaa434333 Abstract.html.
Vilas, M. G., Schaumlöffel, T., and Roig, G. Analyzing vision transformers for image classification in class embedding space. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Con- ference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL http: httt:::/paaprs.miis...
p
7dd309d 7eC009#f0a554430088444444222 Destrrcct-Conffrencee.htt
Wendler, C., Veselovsky, V., Monea, G., and West, R. Do
laaaas work in english? on the latent language of multilin- gual transformers. ArXiv preprint, abs/2402. 10588, 2024. URL htttps://arxivat wttoorg/bbb//////ee abs/ /bb...20588.8
Yang, S., Gribovskaya, E., Kassner, N., Geva, M., and Riedel, Do large language models latently perform multi-hop reasoning? ArXiv preprint,
12

---

Transformers Determine Top Tokens in Order
A. Appendix
A.I. Prompt Format
When using questions from MMLU dataset in our experiments we employ the following prompt to present each task's questions, answer choices, and correct answer, ensuring uniform input structure.
Prompt format:
Question: <QUESTION>
A. <CHOICE A>
B. <CHOICE B>
C. <CHOICE C>
D. <CHOICE D>
Answer: <ANSWER>
A.2. Saturation statisties
Table 4 shows the percent of input tokens which reach top-1 saturation in the first 85% layers of the model across all models we tested. For all fext-Transformers we use 1K randomly sampled questions from MMLU dataset with the prompt format described in A.I, for ViT we use 1K random images from CIFAR- 10 dataset, and for Whisper we use IK random audios from LibriSpeech dataset.
For Llama3-8B, for example, find that 31.2% of all input tokens reach top-1 saturation defined it. Additionally, show in Figure 6 that the tokens that reach top-1 saturation belong to all different parts of speech (POS) including content words, with over 30% of them being nouns.
A.3. Stricter Kendall's tau
We define version of Kendall's tau coefficient measuring the ordinal association between two tanking, where one-sided ties are considered discordant unlike the regular metric, where ties are typically either ignored or handled neutral, meaning they neither count as concordant nor discordant. This is done to discount cases where two or more tokens reach saturation at the same layer. The coefficient takes values in the range [-1, 1] where values close to 1 indicate strong agreement, and values close to 11 indicate strong disagreement between the rankings.
Formally, given two rankings (I1. .2..... and y (y/1 y2..... Un). let pair(i,)) be pair of indices where 1 KiijjS
We define the pair as concordant if the rankings in both sequences agree, meaning:
(x; > xy and > yj) or (n, < xy and f/ < リj) or
(x; = ny and y/ = yj)
Table 4. Percent of input tokens that reach top-1 saturation.
Model
% tokens
Llama3-8B
31,24
Mistral-7B
43.36
Falcon-7B
11.75
GPT2-XI.
58.42
ViT-I/16
44.70
laise
10.92
13

---

Transformers Determine Top Tokens in Order
5000
of tokens
0000
2000
MOUNVERS CONJ
PRT ADP
ADJ
DET PRON ADV NUM
POS
Figure 6. POS of samples that reach top-1 saturation in first 85% of layers of Llama3-8B
The pair is discordant if:
(x; > xy and y < }j) or
(x; > zj and y > yj) or
(æ; = xy and y # yj)
(x; + Iy and = Uj)
The coefficient Taricts is computed as:
C
Tstrict
C+
where C' is the number of concordant pairs, and D is the number of discordant pairs (including ties), ranging in values between 1].
To check whether the sequence of saturation layers of the top-k tokens (l",.., (*) is strictly increasing, we use that sequence as one ranking, and the sequence (1, ..,,,..k) k) as the other. & independently for each token in the input be the largest token index such that this token's 's reaches saturation by definition i.e. 1 N. Table 5 summarizes the results of this metric across the different models discussed in the paper. We find that for all models, the average coefficient indicates moderate agreement between the rankings, and is larger than all values over 1K permutations, where the saturation layers sequence were randomly shuffled for each instance, resulting in p < 0.001.
A.4. Reproducing Ordered Saturation in Various Text Transformers
We extend the results from Section 3.1 by reproducing the findings with additional decoder-only LLMs: GPT2-XL, and 8-bit quantized versions of Mistral-7B and Falcon-7B, Figure 7 presents the high correspondence between saturation layer and token rank observed consistently across all three models.
Using IK randomly sampled questions from the MMLU dataset for each model and using the prompt format described in Section A.2, find that the average rank of the k-th saturation layer increases monotonically with k (up A 5), with statistically significant differences between each consecutive token rank < 0.001, pairwise independent samples t-test). Similarly, observe that Mistral's 's top-k tokens reach saturation in order of their ranking up and including the 5th token, while Falcon's tokens follow this pattern up to the 4th ranking token.
Table 5 shows additional validation of this agreement between token ranking and saturation layer ranking using a stricter version of Kendall's 7 metric as described in Section 3.1, with statistically significant average 7 values for all models confirmed by a random permutation test.
14

---

Transformers Determine Top Tokens in Order
saturation layer rank
saturation layer rank
(a) GPT2-XL
(h) Mistral-7B
(c) Falcon-7B
Figure 7. Average rank of the k-th saturation layer among the saturation layers for k=1...,5 with standard error bars. Asterisks indicate statistically significant differences between consecutive token ranks (*** represents p < 0.001, represents p < 0.01), based on an independent samples t-test.
A.5. Reproducing Task Probing in Various Text Transformers
In support of our proposed task transition mechanism, using embeddings extracted during inference over 500 questions randomly sampled from MMLU dataset, we demonstrate that task index can be predicted from GPT2, Mistral and Falcon hidden layers' embeddings with high accuracy. We report full results and control settings in Table Asterisks indicate statistically significant accuracy (*** represents p < 0.001), based on Binomial Distribution probability test.
A.6. Training Data for Task Transition Classifier
Table 7 shows from which layers we took embeddings to train the task-transition classifier for each model.
A.7. Per Class Metrics for Task Transition Classifier
Table 8 shows accuracy scores per-class for each model, while Table 9 shows ROC-AUC scores per-class for each model.
A.8. Reproducing Ordered Saturation and Task Probing in LLa
We extend the results from Section 3.2 to multi-modal models by reproducing the findings with vision language model LLa Va-1.5-7B model (Liu et al., 2023).
Using IK randomly sampled questions from the visual question answering MMMU dataset (Yue al., 2024) and using the prompt format described in Section A.2, we find that the average rank of the k-th saturation layer increases monotonically with k (up to & = 5), with statistically significant differences between each consecutive token rank (p 0.001, pairwise independent samples (-test), See Figure 8.
Table 5. Stricter Kendall's tau coefficients and p-values for each model. signifies statistical significance with Pealue < 0.001
Model
Tstrict (avg + ste)
Llama3-8B (pre-trained)
0.249* + 0.004
Llama3-8B (random initialization)
0.283* + 0.004
ViT-L/16 (pre-trained)
0.149* + 0.007
Whisper-large (pre-trained)
0.210* + 0.009
GPT2-XL (pre-trained)
0.187* = 0.004
Mistral-7B (pre-trained)
0.312* = 0.002
Falcon-7B (pre-trained)
0.234* = 0.005
15

---

Transformers Determine Top Tokens in Order
Table Accuracy of task number logistic regression classifier showing that in all modalities the layer embeddings contain information about the task number.
Model
Accuracy
Layer embeddings
Random embeddings
Chance
GPT2-XL (pretrained)
84.2 "M 0.4
20.4 0.5
20.0
Mistral-7B (pre-trained)
85.8 0.1
27.1 = 0.1
25
Falcon-7B (pre-trained)
91.0" + 0.1
24.6
25
LLaVa-1.5-7B (pre-trained)
89.0 + 0.7
2.1
25
Qwen-Audio (pre-trained)
95.1 T** + 0.3
25.0 = 0.5
25
Table Task tramsition probing data.
Model
Layers
Dataset size
Llama3-8B (pre-trained)
19 27
KK
Llama3-8B (random initalization)
21 27
4K
GPT2-XL (pre-trained)
27 - 40
10K
Mistral-7B (pre-trained)
22 27
4K
Falcon-7B (pre-trained)
19 27
5K
VIT-L/16 (pre-trained)
16 - 21
2K
Whisper-large (pre-trained)
29 32
4K
LLLVa-1.5-7B (pre-trained)
22 27
4K
Qwen-Audio (pre-trained)
25 30
5K
In support of our proposed task transition mechanism, using embeddings extracted during inference over 500 questions randomly sampled from MMMU dataset, we show in Table 6 that task index can be predicted from LLaVa's hidden layers' emheddings with high accuracy.
A.9. Reproducing Ordered Saturation and Task Probing in Qwen-Audio
We extend the results from Section 3.3 to another variant of Transformers with Qwen-Audio (Chu et al., 2023) multimodal speech model trained for speech recognition, audio captioning, and audio-text retrieval.
Using 1K randomly sampled audios from the LibriSpeech we find that the average rank of the k-th saturation layer increases monotonically with (up to A 4), with statistically significant differences between each consecutive token rank (p 0.001, pairwise independent samples t-test). See Figure 9.
In support of our proposed task transition mechanism, using embeddings extracted during inference over the same audios, we show in Table that task index can be predicted from Qwen-Audio's 's hidden layers' embeddings with high accuracy.
A.10. Intervention Procedure Additional Details
Formally, this procedure consists of the following steps:
1. Given an input sequence =< #1...., >> we first pass it through the model as in regular inference while storing the activation values at all hidden layers, i. hf for all 1 < i 1 N.
2. We calculate the saturation layer 1/ of the 1st token for each token in the text.
3. We sample pairs of token indexes i,j in the text that the satisfy the following conditions:
(a) The distance between and j is no more than 40 tokens, i.c. | - j| < 40.
This a precaution to minimize the effect of the difference context on the model's predictions after intervention.
16

---

Transformers Determine Top Tokens in Order
Table Task transition probing per-class accuracy scores.
Model
Task
Task 2
Task 3
Task
Task
Llama3-8B (pre-trained)
0.82
0.89
0.93
Llama3-8B (random initalization)
0.69
0.79
0.89
ViT-L/16 (pre-trained)
0.65
0.62
0.60
0.64
0.68
Whisper-large (pre-trained)
0.52
0.53
0.48
0.58
499
saturation layer rank
2
token rank
Figure 8. LLaVa-1.5-7B: Average rank of the k-th saturation layer among the saturation layers for k=1,...5 with standard emror bars. Asterisks indicate statistically significant differences between consecutive ranks (*** represents p 0.001, ** represents p < 0.01), based on an independent samples t-test.
(b) The model's top-1 prediction (in the final layer) for both indexes is the same token y, meaning y cargmar(sofftnn) (
The goal here is to avoid conflict in the top-1 predictions which could be confounding factor.
(c) There a difference of at least layers between the 1st token saturation layers of i and j. such that 14 - ensure that the change in saturation layer after intervention is significant.
For convenience's s sake we will assume in the remainder of the procedure description that It < 1). that the saturation layer of the first index in the pair is smaller then that of the second index (even though both cases allowed by our conditions).
4.
We perform additional forward passes, each time injecting the output from layer l' in range !1 3, + at position as input into layer l' + at position j. The goal here is to to quantify the difference effect between layers preceding the saturation event and those after it.
Table 9. Task transition probing per-class ROC-AUC scores.
Model
Task 1
Task 2
Task
Task
Task 5
Llama3-8B (pre-trained)
0.94
0.97
0.97
Llama3-8B (random initialization)
0.87
0.90
0.96
ViT-L/16 (pre-trained)
0.86
0.82
0.81
0.83
0.87
Whisper-large (pre-trained)
0.78
0.77
0.69
0.77
17

---

Transformers Determine Top Tokens in Order
saturation layer rank
1
2
token rank
Figure Qwen-Audion Average rank k-th saturation layer among saturation layers k=1...,5 with standard error bars. Asterisks indicate statistically significant differences between consecutive token ranks (+#* represents < 0.001, ** represents < 0.01). based on an independent samples t-test.
5. We measure the causal effect of the intervention by calculating the percent of examples where the saturation layer of the 1st token after intervention 2) the layer which we intervened, i.e. i3 l
For example, in the setting depicted in 4 we would take the indexes of the marked tokens "wanted" and "the" as our pair, where the original top-1 prediction both is "artist". The top-1 saturation layer the clean run for "wanted" is layer 20, so we would inject activations from layers 17 to 23 one at a time as inputs into the corresponding subsequent layers the run of token "the" (24), and check for each one if the injected layer became the new top-1 saturation layer after the intervention.
A. 10.1. INTERVENTION PROCEDURE ON VIT
To adapt the intervention procedure described in section 4.2 to ViT-L/16 and the image classification setting made the following modifications:
1. Since each image is processed independently by the model there is need for two images to share context, the only requirements for two images to be chosen relevant pair were: distance of at least 5 layers between the top-1 saturation layers, and the same top-1 class prediction in the final layer.
2. For each image, as in all experiments conducted on this model we only consider the prediction at index 0 corresponding to the [CLS] token in the input.
Figure 5 shows that the results for this model follow a similar step function pattern to the ones for Llama3-8B, where injecting embeddings from the top-1 saturation layer or one the subsequent layers causes the model to "immediately" (at the injected layer) switch to the second task high percentage of cases, when compared to injecting embeddings from one of the layer before the top-1 saturation which almost never has the same effect.
A. 10.2. INTERVENTION PROCEDURE ON WHISPER
We made the following adjustment to run the procedure described in section 4.2 on Whisper-large and 200 token pairs from randomly sampled 50 audios from the LibriSpeech dataset: Since the average audio in LibriSpeech is 10 seconds long there are not enough tokens in one sample to find relevant pairs, so we wave the requirement for a pair to share context and only leave two conditions: a distance of least 5 layers between the top-1 saturation layers, and the same top-1 prediction the final layer.
18

---

Transformers Determine Top Tokens in Order
8
of examples with task switch
80
40
60-
30
40-
20
10
Distance of injected layer from original saturation layer
(a) GPT2-XL
(c) Falcon-7B
(b) Mistral-7B
Figure 10. Flipping the Top-I Switch. The percentage of examples where the top-1 saturation occurred at the injected layer after the intervention, shown as function the layer from which injected activations were taken, relative to original saturation layer (e.g.. -2 means activations were taken from two layers before the original saturation layer).
Figure 5 shows that the results for this model follow a similar pattern to the other two models, even though the effect increases the following layers after the saturation event.
A.11. Reproducing Task Switch in Various Text Transformers
show in Figure the results of the intervention procedure described in Section 4.2 using GPT2-XL, Mistral-7B and Falcon-7B models over 200 token pairs (each) extracted from 10 randomly sampled texts from CNN/DM dataset. As with the other models, when the injected activations saturation happens at the injected layer much more frequently than when injecting activations from earlier layers, indicating that these layer contain the signal switch to the next task and keep the top-1 constant.
taken from the top-1 saturation layer later layers, the new top-1
A.12. Additional Details for Token-Level Early Exit Measures Comparison
Table 10 shows the accuracy and speedup ratio of the Softmax Response token-level early-exit strategy various thresholds.
Table 11 shows the accuracy and speedup ratio of the Hidden-state saturation token-level early-exit strategy at various thresholds.
Figure 11 visualizes the performance--fffccinng trade-off of both methods comparison to our novel carly-cxit strategy, as well as baseline and oracle.
Table 10. Softmax Response accuracy & speedup ratios at various confidence thresholds
Thresholds
0.5
0.6
0.7
0.8
0.9
0.92
0.94
0.96
0.98
Accuracy Speedup Ratio
33.149
34.861
36.437
37.930
39.334
39.717
40.026
40.340
40.648
1.326
1.284
1.243
1.199
1.145
1.132
1.117
1.099
1.073
19

---

Transformers Determine Top Tokens in Order
baseline
oracle
ours
softmax
0.4
state
Accuracy
0.3
0.2
0.1
11
12
Speedup ratio
Figure Performancc--ffiiienng trade-off comparison different confidence measures against a static baseline (where all layers are used for each token) and local oracle measure (where the early exit is at the top- saturation layer). The graph shows softmax and state confidence measure results at different thresholds. Our method achieves the highest next word prediction accuracy out of all early-exist methods while providing significant speedup compared the baseline.
Table 11. Hidden-sate saturation accuracy & speedup ratios various confidence thresholds
Thresholds
0.9
0.92
0.94
0.96
0.98
0.99
Accuracy
11.267 1.682
15.183
21.396
24.250
24.252
24.252
Speedup Ratio
1.552
1.361
1.130
1.001
1.001
20


Reviews:
[{"review_id": "V33QnC7zsc", "reviewer": "Reviewer_2s7d", "summary": "This work analyses the saturation event observed in transformer layer, where a layer's top-1 prediction for the next best token remains the same from some layer onwards. This paper expands the analysis beyond that of top-1, illustrating how the argument continues for top-k and analyses a specific component from the activations which causes the top-k switch from the embeddings from which point onwards all layer's top-k remain the same. The main idea of the paper revolves around moving from top-1 layer saturation task to top-k'th layer saturation. \n\nThe paper's empirical results include 1) illustrating how the top-k token ranks for LLM output mostly follow a similar trend to that of the position of their respected saturation layer in the network, 2) This observation is classified as a property of transformers and not task-specific as it appears on untrained randomly initialized, 3) Train a linear regression model to classify the k'ranking for each layer, 4) Use their discovery to improve the top-k token prediction for LLMs (compared to previous methods) by implementing their early exit strategy using the linear classifier.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review.\n\n**Claims And Evidence**\n\n>I don't believe results provided in Figure 3 .. are convincing enough beyond the 3rd top token.\n\nBeyond the top-3 tokens, saturation order is indeed less consistent in some models. However, our results are statistically significant across models and modalities.This variability may be partly due to model depth—fewer layers may limit lower-ranked tokens from reaching saturation. For instance, GPT-2 XL (48 layers) maintains order up to the 5th token, unlike Llama3-8B (32 layers) (Appendix A.4). We will add this observation to the paper.\n\n>With respect to the results on classifier accuracy… I'm afraid the same can't be said about the Image and speech transformers.\n\nWhile the classifier accuracy in Vision and Speech Transformers is lower than in text models, it remains statistically significant (Table 1) and over twice of chance level. Though this may limit practical applications, this experiment validates the task transition mechanism.\n\n>I believe the model's accuracy on specific k-top layer is probably more informative than the overall accuracy.\n\nTable 8 shows no significant accuracy differences across tasks, with ViT even performing slightly better for Task 5 than Task 1. The classifier was trained on balanced data representing all tasks. We didn't investigate these differences, but agree this is an interesting avenue for future research.\n\n>With regards to the layer activation injection, I have some questions on what the activation layer is.\n\nWe inject the first saturation layer’s output from sample s1 into sample s2’s subsequent layer and observe its effect on s2’s first saturation layer. As a control, we repeat this with activations from three layers before and after. Please see Appendix A.8 for details. We will clarify this in the camera-ready version.\n\n>However, just extending the results to 2-tokens might not be significant enough for the paper.\n\nFollowing this comment we analyzed the 3rd and 4th top tokens using Llama3-8B on 500 CNN/DM texts, assuming higher-ranked tokens were incorrect, and found the results generalize.\n\nFor the 3rd top token, accuracy increased from 17.01% (no saturation) to 23.3%, 26.9%, 28.9%, 29.2%, and 31.0% when saturation occurred 2 to 6 layers before output, respectively.\n\nFor the 4th top token, accuracy increased from 13.4% (no saturation) to 16.8%, 19.7%, 20.45%, 19.5%, and 18.3% when saturation occurred 2 to 6 layers before output.\n\nA two-proportion z-test confirms statistical significance (p < 0.001) for 2 ≤ i ≤ 6 in the 3rd token and 2 ≤ i ≤ 5 in the 4th token. We will include this analysis in the camera-ready version.\n\n**Questions**\n\nQ1: We report in our response to Reviewer J6pR that other Vision and Speech models achieve high accuracy, comparable to text models. In Whisper, we speculate that order deteriorates in later tokens because all decoder layers are conditioned on the encoder’s last layer, potentially blurring task boundaries. In contrast, in Qwen-Audio, where the encoder only generates input features for the decoder, our task classifier achieves high accuracy.\n\nQ2: Non-linear probing could hurt our claim, as powerful probes may learn arbitrary mappings rather than reveal the model’s inherent representations [1, 2]. \n\nQ3: We computed standard error as std/sqrt(n) [3]; large n results in small values .\n\nQ4: Omitted values indicate that the probing classifier was trained on a different number of tasks per model. For Llama3-8B for example, it was trained on three tasks, so accuracies for Tasks 4 and 5 are not reported. We will clarify this in the caption.\n\nQ5: We did not explicitly test task classifier transferability across models, but we indirectly demonstrate cross-dataset transferability by applying a classifier trained on MMLU to CNN/DM in our early-exit experiment (Section 5.1). \n\nQ6: In our analysis of the ordered saturation of top-k tokens, we set k = 5 to ensure consistency across models while maintaining sufficient examples (>4% of inputs for all models). As k increases, saturation cases decline. The 5% value was a typo. We will clarify this and add detailed statistics in the appendix.\n\nQ7: As noted in Appendix A.3, stricter Kendall’s tau p-value was computed via a permutation test of saturation layer sequences (1K random shuffles per instance). In all cases, real tau values exceeded shuffled results (p < 0.001). We will further clarify this in the paper.\n\nQ8: Table 3 analyzes the second token’s accuracy when the top-1 token is incorrect, depending on its saturation layer (2 ≤ i ≤ 6 layers before output). Variability in saturation layer is an open question, but our main goal was to show a statistically significant difference between saturation and no-saturation cases.\n\nWe welcome further discussion if needed. If our clarifications are satisfactory, we kindly ask the reviewer to consider raising their rating.\n\n---\n[1] Hewitt et al., 2019\n\n[2] Belinkov, 2022\n\n[3] Casella, G., & Berger, R. L. 2002", "claims_and_evidence": "While I do believe the main claims of the paper, which is that that layer saturation happens beyond just the top-1 token, I don't believe results provided in Figure 3 regarding the rank of k'th top token following the same as their order in the network are convincing enough beyond the 3rd top token. Even though the rankings look quite monotonic for the first top-1,2,3 tokens, for 3 onwards I find it alarming that the average is not close to the actual layer rank. Further I would like more information on the way $p$ was calculated and I would like some more discussion regarding the $\\tau_{\\text{strict}}$ definition. I will ask further questions in the appropriate section. \n\nWith respect to the results on classifier accuracy, while I find the results on language and text models acceptable, I'm afraid the same can't be said about the Image and speech transformers. I think the accuracy there is somewhat low for me to consider reliably using the classifier for further practical applications. I will discuss my questions on this topic as well. \n\nAdditionally, when considering Table 1, I believe the model's accuracy on specific k-top layer is probably more informative than that if the overall accuracy. It could be that the classifier is doing a great job at detecting the first  top-1,2 tasks as they are evidently more distinct from Fig 3, but could fail on the layer ranks top-3,4,5. Looking at Table 8 in appendix, I see that this happens for task 4-5 on some of the models. To be clear, I am not saying the argument doesn't hold, I am saying this should be discussed and is more important that the total accuracy. \n\nWith regards to the layer activation injection, I have some questions on what the activation layer is ? but regardless, I see that visuals were only provided for probing the top-1 layer ? Again I see Table 9 provide some accuracies but the probing task but I would like to have a more clear understanding of how these numbers were generated. \n\nWith regards to the speedup and early exit, I am not as familiar with other early exit methods and I taking the authors results at face value.  However, just extending the results to 2-tokens might not be significant enough for the paper. I don't find in appendix any results on accuracy for tasks above the 2nd top token. I believe its important for them to be reported even if they don't have a positive outcome. I understand however that this could be hard given the constraints of the experimental setup (top 2 tokens have to be wrong for the LLM for the task 3 evaluation)\n\nFinally I am not sure I understand the conclusion drawn from Table 3. I will have a question regarding it below.", "methods_and_evaluation_criteria": "I believe most of my concerns regarding this have been discussed above. I would like more clarity on the evaluation of $p$ for statistical significance. I do think more results could be provided for the speed-up argument of the paper, however, I understand that this is not the main focus of the paper.", "theoretical_claims": "Does not apply to this paper.", "experimental_designs_or_analyses": "Here I will ask the main questions I have regarding the paper as it is mostly reliant on empirical observations: \n\nQ1: Why do authors believe the rank accuracy drops noticeably on non-text transformers ? Could it have something to do with certain discoveries regarding the sparsity of attention activations or weights when dealing with LLMs? \n\nQ2: Have the authors attempted to use non-linear model ? I think it could help\n\nQ3: For Figure 3, I don't understand why the error bar is so small. I assume every rank is an integer, and I would expect an integer to exists within in the range of mean +/- error bar. Am I missing something ? \n\nQ4: Can the authors explains the missing values in Tables 8-9. I think I missed the explanation for them. \n\nQ5: Is the task classifier transferable from one model to another assuming the dimensions across transformers are the same ? For example for the same architecture but trained on different datasets. \n\nQ6: Could the authors please expand on the \"$k=5$ as the highest value token which reaches 5% of the input tokens \" ? I am not sure I understand. \n\nQ7: Could the authors expand on the calculations of $p$ using the $\\tau_{\\text{strict}}$.\n\nQ8: Why do the accuracies of the second layer change beyond the task 2 saturation layer in Table 3 ? I assume all these layers are post task 2 saturation.", "relation_to_broader_scientific_literature": "This paper extends the top-1 saturation argument to further top-k tokens. I believe assuming this is the first paper which extends the idea, specially the discovery w.r.t untrained transformers, could have impact on better understanding of the transformer's functions.", "essential_references_not_discussed": "I am not aware of any. I am not as familiar with early exit strategies and it is possible that works from that field are excluded in the paper's discussion, however as the authors mention, this is not the main target of the paper.", "other_strengths_and_weaknesses": "All discussed above.", "other_comments_or_suggestions": "I believe there might be a type on line 681-682 for the discordant. \n\nI am open to change my score depending on the future discussion.", "questions_for_authors": "Mentioned above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ISr8u1fs6F", "reviewer": "Reviewer_9Kne", "summary": "This paper shows that at the intermediate layers, model will sequentially determine the top-k token, rather than just determine the top-1 token. And this observation maybe some special property of transformer architecture. The author calls this as task transition, and show that this is a general phenomenon on different kinds of scenarios (ViT, SpeechTransformer). They also design some new early-exit strategy for accelerating inference.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful comments and helpful suggestions. We appreciate that you found the sequential determination of top-k tokens to be an interesting and well-supported phenomenon. We're also glad that you recognized the broader relevance of our findings in relation to existing literature on token determination in Transformers.\n\n**Claims And Evidence:**\n>I believe the obvious order of token determination doesn't necessarily result the task transition. There is possibly another possible modeling: if the weight of all top tokens is increasing together but with different scale, we can also observe sequential saturation of top tokens starting from random weights.\n\nWe agree that the sequential saturation of top tokens could emerge from a different underlying mechanism, such as the one suggested—where all top token probabilities increase together but with different scales due to residual connections. This is an interesting alternative perspective, which we will add to the paper.\n\nTo clarify, our proposed interpretation is not that the first task corresponds strictly to processing only the first token, the second to the second, and so on. Rather, what we can say with confidence is that by the time the model transitions to the second task, it is no longer focusing on the first token in the same way. This distinction is supported by the probing classifier results, which show high accuracy in distinguishing the tasks as we define them, indicating that these transitions are not simply a gradual rescaling but rather discrete shifts in model behavior.\n\nAdditionally, the intervention experiments provide further support for our interpretation. Specifically, when we inject layer activations from later stages, the model abruptly shifts its behavior, ceasing to focus on the top-1 token and instead progressing to the next stage. This causal evidence suggests that the observed transitions are more than just a continuous adjustment of weights. That said, we agree that further investigation is needed to rigorously rule out alternative explanations, and we appreciate the reviewer’s suggestion to refine the claim for greater clarity.\n\nGiven the opportunity we will incorporate those points into the discussion section in the camera-ready version.\n\n**Other Comments Or Suggestions:**\n\nWe will follow your suggestion to expand on previous work on extracting predictions from intermediate layer in the introduction section in the camera ready version of our paper.\n\n**Questions For Authors:**\n>I think this phenomenon will be quite related to the existence of residual connection, because this results that every layer will have minor modification to the previous layer, i.e., for x_{n+1} = x_n + f(x_n), actually many works show that |x_n| >> |f(x_n)|, I think this may be related?\n\nYou raise a very interesting question about the relevance of residual connections to the ordered saturation phenomenon. We address this indirectly in our Related Work section when considering the iterative inference hypothesis. This approach interprets each layer as an iteration from an iterative and convergent process [1], suggesting that each layer incrementally refines the hidden representation by gradually shaping the next token prediction [2-4].\n\nAt first glance, the existence of residual connections should make it more difficult to distinguish layer-wise task transitions, as they enforce strong similarity between consecutive layers. However, our results show that saturation events still occur in a structured and discrete manner, rather than as a continuous drift. This suggests a phase transition-like dynamic: while each layer makes small refinements, at a certain point, a critical threshold is reached, leading to a sharp shift in representation and task focus.\n\nMore work is needed to determine which components in the Transformer architecture give rise to this phenomenon, and the residual connections is certainly one to examine alongside self-attention and MLP layers. One possible way to do so is through various ablation studies. \n\n\nThank you again for taking the time to review the paper and providing helpful feedback! \nDo the above actions address your concerns with the paper? If not, what further clarification or modifications could we make to improve your score?\n\n---\n[1] Simoulin, Antoine, and Benoit Crabbé. \"How many layers and why? An analysis of the model depth in transformers.\" \n\n[2] Geva, Mor, et al. \"Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space.\"\n\n[3] Belrose, Nora, et al. \"Eliciting latent predictions from transformers with the tuned lens.\" \n\n[4] Rushing, Cody, and Neel Nanda. \"Explorations of self-repair in language models.\"", "claims_and_evidence": "I think the claim of sequential determination of top-k tokens are convincing and supported by various experiments on different downstream tasks, and the claim itself is interesting.\n\nHowever, I'm confused about the claim about \"task-transition\". Because it sounds like a modeling for explaining sequential determination of top tokens, but the later experiments (like the logistic regression classifier) don't really support this claim, but only re-support that embeddings from different layers can corresponding to different stage of token determination. \n\nI believe the obvious order of token determination doesn't necessarily result the task transition. There is possibly another possible modeling: if the weight of all top tokens is increasing together but with different scale, we can also observe sequential saturation of top tokens starting from random weights. And considering the existing of residual connection, the weight modification of each token can be minor across layers. Then the probability of all these top tokens can increases slightly with different scale, but the tokens with larger increasing scale will appear to be top-k earlier. I think the modeling way is not unique, need more investigation or change the claim for soundness.", "methods_and_evaluation_criteria": "I think the evaluation metrics (like Kendall's tau) are reasonable and we can see some statistical trend of the phenomenon.", "theoretical_claims": "This paper doesn't have theoretical claims.", "experimental_designs_or_analyses": "I think the experiments on convincing the sequential saturation phenomenon is relatively completed, consider different downstream tasks and different models. The improvement from the additional algorithm (like Table 2) looks incremental for me, but this should not be the main contribution of this paper.", "relation_to_broader_scientific_literature": "I think the broader scientific literature shows the top-1 token will be determined at the very last layers of transformer (like [1]), and this paper shows that this phenomenon also happens, and sequentially, for top-k (like k=4 or 5) tokens.\n\n\n[1] Nostalgebraist. Interpreting gpt: the logit lens, 2020.", "essential_references_not_discussed": "I don't find there is some missing key references.", "other_strengths_and_weaknesses": "Strengths:\n\n+: The observation is interesting, and the experimental result for this phenomenon is supportive and sufficient.\n\nWeaknesses:\n\n-: I think the main concern is about the claim of \"task transition\" we mentioned above. And I think this make the readers more difficult to understand the main takeaway of this paper. And following this, maybe Figure 2 is more understandable than Figure 1 for authors' idea, and should be put earlier.", "other_comments_or_suggestions": "Can introduce more about logit lens or previous works in the main paper for the background introduction? For example how people obtain and use the projection for intermediate layers.", "questions_for_authors": "I think this phenomenon will be quite related to the existence of residual connection, because this results that every layer will have minor modification to the previous layer, i.e., for x_{n+1} = x_n + f(x_n), actually many works show that |x_n| >> |f(x_n)|, I think this may be related? How do authors see this and how to prove this holds or not.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QywGGmW5hp", "reviewer": "Reviewer_J6pR", "summary": "This paper studies the calculation process of the hidden layer of the Transformer model after completing the top-1 prediction, and find that the model will determine the subsequent tokens (such as top-2, top-3, etc.) in order of token ranking. \n\nThrough experiments across modalities (text, vision, speech) and model variants (pre-trained/untrained Transformers), it is proposed that this phenomenon originates from the discrete task transformation mechanism.\n\nFinally, the authors introduce a token-level early exit strategy that can improve efficiency while maintaining performance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and positive assessment of our work! We appreciate your recognition of the novelty of the ordered saturation phenomenon and its implications for understanding Transformer models. Your acknowledgment of both the theoretical contributions and practical applications, such as our early exit strategy, is encouraging.\n\n**Other Strengths And Weaknesses:**\n\n> However, the applicability of complex tasks/models is still unclear, such as some of the more advanced LLMs, Qwen Audio series, LLaVA series, and it is not clear whether they are universal for these complex tasks/models\n\nWe agree that exploring the applicability of our findings to more complex tasks and advanced models is an exciting and important direction.\nFollowing your suggestion we replicated our results using LLaVA-1.5-7B model on 1K questions from MMLU dataset treating it as a language model. We found the average rank of the k-th saturation layer for each k increases monotonically with k (up to k=5) and the difference between each two consecutive token ranks is statistically significant with p < 0.001 based on a pairwise independent samples t-test (average saturation ranks in order: 1.01, 2.14, 2.66, 2.87, 3.00). We also trained a classifier to predict task number from hidden layer representations, and achieved an 82.4% average accuracy over 5 classes (chance level accuracy being 20%).\n\nWe then re-run the analysis on 500 questions from MMMU dataset [1], utilizing this model’s multi-modal capabilities on the more complex task of Visual Question Answering, where the input contains both image and text. We found the average rank of the k-th saturation layer for each k increases monotonically with (k up to k=5), and the difference between each two consecutive token ranks is statistically significant with p < 0.001 based on a pairwise independent samples t-test (average saturation ranks in order: 1.01, 2.38, 2.91 3.14, 3.33).\nWe also trained a classifier to predict task number from hidden layer representations, and achieved an 82.4% average accuracy over 5 classes (chance level accuracy being 20%).\n\nAdditionally, we ran Qwen Audio model on 500 audio samples from LibriSpeech dataset and found that the average rank of the k-th saturation layer for each k increases monotonically with k (up to k=4) and the difference between each two consecutive token ranks is statistically significant with p < 0.001 based on a pairwise independent samples t-test (average saturation ranks in order: 1.01, 1.85, 2.73, 2.93). We also trained a classifier to predict task number from hidden layer representation and achieved an 85.5 average accuracy over 4 classes.\n\nThese results further emphasize the robustness of the ordered saturation phenomenon over models, tasks and modalities.\n\nThank you again for taking the time to review the paper and providing helpful feedback! \nDo the above actions address your concerns with the paper? If not, what further clarification or modifications could we make to improve your score?\n\n---\n[1] Yue, Xiang, et al. \"Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi.\" IEEE/CVF 2024.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "The paper does not involve theoretical proof, but mainly relies on experimental verification.", "experimental_designs_or_analyses": "Yes.\nThe experimental design of this paper is clear, verifying the universality of saturation events through cross-modal (text, vision, speech) and multi-model (pretrained/untrained Transformer)", "relation_to_broader_scientific_literature": "This paper mainly inherits the saturation event research of [1] and extends it to top-k tokens. At the same time, it echoes the logit lens analysis of [2,3], and methodologically, the early exit strategy of [4] is compared.\n\n[1] Geva, M., Caciularu, A., Wang, K., and Goldberg, Y. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 30–45, Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguistics.\n\n[2] Nostalgebraist. Interpreting gpt: the logit lens, 2020\n\n[3] Yang, S., Gribovskaya, E., Kassner, N., Geva, M., and Riedel, S. Do large language models latently perform multi-hop reasoning? ArXiv preprint, abs/2402.16837, 2024\n\n[4] Schuster, T., Fisch, A., Gupta, J., Dehghani, M., Bahri, D., Tran, V., Tay, Y., and Metzler, D. Confident adaptive language modeling. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "This paper systematically reveals for the first time the \"ordered saturation\" phenomenon of top-k prediction in the Transformer model in order of ranking, and verifies its universality through cross-modal experiments, which provides a new perspective for understanding the internal mechanism of the model.\n\nIn addition, practical applications, such as an early exit strategy to accelerate the next token prediction, without introducing too many additional calculations, are also explored, striking a balance between speed and accuracy\n\nHowever, the applicability of complex tasks/models is still unclear, such as some of the more advanced LLMs, Qwen Audio series, LLaVA series, and it is not clear whether they are universal for these complex tasks/models\n\nOverall, given the outstanding performance of the paper in terms of innovation and practical value, I suggest accepting", "other_comments_or_suggestions": "No.", "questions_for_authors": "see other strengths and weakness", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xHEL4yTyh6", "reviewer": "Reviewer_petj", "summary": "This paper studies the \"saturation event\" in transformers, where the model’s top $ k $ predictions are determined in the early layers and remain fixed in the later layers. They show empirically that the top-ranked tokens saturate in order, with higher-ranked tokens saturating in earlier layers. They also demonstrate that probing the embeddings at each layer allows for predicting the transition to the next saturation event. Based on these findings, they propose an early-exit strategy for text generation to improve model efficiency. Their experiments across different modalities—text, image, and audio—using various transformer-based architectures support their results.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's helpful feedback. \n\n**Methods And Evaluation Criteria**\n>A simpler and more direct option could be reporting the percentage of inputs where the order of the saturation layers matches the order of the tokens.\n\nWe calculated this percentage for Llama3-8B on 1K MMLU questions for top-k tokens, where top-1 refers to cases where the 1st tokens reach saturation in order, top-2 includes the 1st and 2nd tokens, and so on:\n\n Top-1 | Top-2 | Top-3 | Top-4 | Top-5 |\n|------|-------|-------|-------|-------|\n| 99.7% | 82.4% | 77.0% | 76.0% | 75.8% |\n\nThese results further validate the ordered saturation phenomenon, and we will include this analysis in the camera-ready version.\n\n> it’s unclear whether the data used in the experiments was part of the pre-training set.\n\nTo mitigate the concerns of data pollution, we reproduced our findings on datasets which came out after the pretraining, and hence less likely to be contaminated. For text, in Appendix A.4, we report results for GPT2-XL model on the MMLU dataset—a benchmark introduced after the model’s release. \n\nFor vision, we reproduced our results using ViT on 1K images from ImageNet-D[1], a newer synthetic benchmark. \nThe average rank of the k-th saturation layer increases monotonically (up to k=3), with statistically significant differences between each two consecutive token ranks (p < 0.001, average ranks: 1.0, 2.0, 2.39). \n\nFor speech, we reproduced our results using Whisper on 200 English audios from the newer Emilia [2] benchmark. Here we also found the average k-th saturation layer rank to increase with k (up to k=3), with the differences being statistically significant (average saturation ranks in order: 1.01, 1.94, 2.42).\nFurthermore, since we demonstrate this order phenomenon in a randomly initialized Transformer, our findings suggest robustness to both training and dataset choice.\n\n>For the early-exit strategy .. it may be more appropriate to use a metric that assesses the model's alignment with the language distribution rather than just the top token prediction/accuracy.\n\nFollowing your suggestion, we calculated the perplexity metric for each early-exit strategy using probabilities from the early-exit layer. For Llama3-8B and 500 CNN/DM texts, our method significantly outperforms Softmax Response in speedup-performance tradeoff and performs similarly to State Saturation, both of which are considered SOTA. See results in table below:\n\n| Strategy           | Speedup | Perplexity |\n|-------------------|---------|------------|\n| Oracle           | 1.09    | 31.13      |\n| Ours             | 1.12    | 100.2      |\n| State Saturation | 1.12    | 107.93     |\n| Softmax Response | 1.15    | 153.96     |\n\n**Weakness**\n>If this implies that the architecture plays a major role in unlocking this phenomenon (line 289), then it would be useful to verify this observation across other architectures.\n\nIn this work we are the first to notice this phenomenon, and focus our work on developing measures of quantifying its extent. We agree that other architectures can also lead to ordered saturation. We will refine this claim in the paper to make it clear that this is an hypothesis raised by our findings.\n\n**Questions:**\n\n1. We extract embeddings for training only when saturation layers are in order, as our goal is to explain this mechanism. \n2. When evaluating our early-exit strategy we also used cases where this order doesn’t hold, i.e., we exit when the first token saturates in all cases. This demonstrates the robustness of our probing classifier and method.\n3. Chance level varies as it is inversely proportional to the number of tasks/classes, which is set to ensure data balancing. We will clarify this in the table caption.\n4. i. Saturation events occur fairly often for other ranks. In LLaMA3-8B on 1,000 MMLU questions: top-1 saturates in 65.7% of tokens (77K), top-2 in 28.3% (33K), top-3 in 15.2% (18K), top-4 in 9.9% (12K), and top-5 in 7.2% (8.5K).\nii. The strict Kendall’s coefficient values reported in the paper, suggesting moderate agreement between token rank and saturation order, are calculated only on the subset of samples where top-1 reaches saturation. \n5. As noted in Section 5.1, dynamic decoding is not our focus, so we propagate states from layers after early-exit as in regular inference. This allows later tokens to use these computations if needed, and the effect of a state-copying mechanism on performance is negligible [3].\n6. Yes, in LLaMA3-8B on 500 MMLU questions, the first saturation layer ranges from 2 to 30. Analyzing its correlation with token position, we found a weak but significant negative correlation (r = -0.1, p < 0.001), suggesting later tokens tend to saturate earlier. We will add this to the camera-ready version.\n\nWe welcome further discussion if needed. If our clarifications are satisfactory, we kindly ask the reviewer to consider raising their rating.\n\n[1] Zhang et al. 2024\n\n[2] He et al. 2024\n\n[3] Schuster et al. 2022", "claims_and_evidence": "The authors test their claims on datasets from text, image, and audio domains, covering a wide range of modalities, which strengthens the support for their findings.\n\nIn the current format, some claims and results, when considered together, might convey mixed/incomplete messages that might need further clarification. See Question 4 and \"*Other Strengths And Weaknesses*\".", "methods_and_evaluation_criteria": "1. Since the key point is that top tokens saturate in order for each input, the average rank of the saturation layers over a large number of inputs may not effectively capture this pattern. The Kendall metric reported in the appendix seems more aligned with the message. A simpler and more direct option could be reporting the percentage of inputs where the order of the saturation layers matches the order of the tokens (which should correlate with the Kendall metric).\n\n2. As also acknowledged by the authors in the limitations section, it’s unclear whether the data used in the experiments was part of the pre-training set. This could slightly affect the results depending on whether the model was directly optimized on these inputs or if they were unseen. Using more diverse input sequences could have reduced this issue.\n\n3. For the early-exit strategy and the comments on improved language modeling, it may be more appropriate to use a metric that assesses the model's alignment with the language distribution rather than just the top token prediction/accuracy. Top-1 accuracy alone is not a perfect measure of a text generator. Nonetheless, the observations in Table 3 are still interesting.", "theoretical_claims": "(No theoretical claims)", "experimental_designs_or_analyses": "See Questions", "relation_to_broader_scientific_literature": "The paper could contribute to the extensive body of interpretability research aiming to understand the inner workings of language models. It also suggests initial methods for leveraging these findings to improve the efficiency of language models during inference, offering an alternative to pruning techniques.", "essential_references_not_discussed": "None that I'm aware of.", "other_strengths_and_weaknesses": "The observations reported in the paper are interesting and are supported by experiments in different setups. They also suggest practical implications for these observations.\n\nHowever, the observation that randomly initialized models also exhibit this in-order saturation event makes the message and implications of this observation a bit vague. If this implies that the architecture plays a major role in unlocking this phenomenon (line 289), then it would be useful to verify this observation across other architectures. \n\nAlso see the clarification issues raised in the \"*Questions*\" field,", "other_comments_or_suggestions": "Minor typos:\n1. In Def 2.2: $l^2=29$. \n2. The running header of the pages doesn't reflect the title of the paper", "questions_for_authors": "The questions below are for clarifying the methodology used in the paper:\n\n1. In section 2.3, for the training phase of the probing task, how do you label the embeddings if the saturation layers are not in order? The tKendall's coeffs reported are around 1.2, which corresponds to only 60% of inputs having in-order saturation layers. Will this cause an issue?\n2. Same question above for the early exiting strategy. Don't you need a guarantee that the second saturation layer is later than the first one?\n3. In Table 1, why is chance level accuracy different in different experiments? Are you predicting a different number of top tokens in each experiment? \n4. Based on Table 4, it appears that in many cases, top-1 saturation occurs in fewer than 50% of inputs (with two models even showing only 10%).\n\n    i. In this context, would the saturation event for other ranks still be meaningful?\n\n    ii. A Kendall coefficient of 1.2 (Table 5) suggests that 60% of inputs show higher rank saturation events in order. Could you clarify how to reconcile this with the observation that less than 50% of inputs show top-1 saturation?\n5. In the early exit strategy (results in Table 2), do you measure top-1 accuracy on all positions in the context window? In this case, in order to skip the computations in a layer, we need all the tokens in a context window reaching their first saturation event before this layer. Is there high variations in the first saturation layer across different positions for a given input sequence?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: After considering all reviewer comments, author responses and the internal reviewer discussion, I recommend conditional acceptance. 

**Key Points**
Reviewer petj found unclear descriptions of measurements and inconsistencies in reported numbers. While mostly addressed, they raised a serious concern (detailed below) but had no strong objections to acceptance provided clarifications are made.

Reviewer 2s7d's concerns were mostly addressed. Their remaining concern about accuracy drops in top-4 and beyond was balanced by their recognition of the paper's contributions: 1) identification of beyond top-1 saturation and 2) introduction of top-k saturation injection. They agreed to raise their score to 3, but share the concern below.

**Shared Concern**
Reviewers petj and 2s7d noted inconsistencies in reported results for Llama3-8B on 1K MMLU questions:

* 31% of tokens reach top-1 accuracy (in Table 4 of the paper)
* top-1 saturates in 65.7% of tokens (response to Q4 of review petj)
* 99.7% of tokens reach top-1 saturation (the first table in the author's response to reveiwer petj)

This concern was brought up after the author response deadline, during internal discussion

**Recommendation**
Since the authors addressed most concerns and two reviewers recommend acceptance, **I recommend acceptance while giving the benefit of the doubt to the authors regarding the reporting inconsistencies.** This acceptance is conditioned on:
* Authors must clarify the numerical inconsistencies noted above
* Authors should use the extra page to include all promised clarifications: Better descriptions of metric settings, more detailed explanations of the injection method, the additional reported experimental results and discussions.
Decision: Accept (poster)