The emergence of sparse attention: impact of data distribution and benefits of repetition
Authors: Nicolas Zucchet
Francesco D'Angelo
Andrew Kyle Lampinen
Stephanie C.Y. Chan
Venue: NeurIPS 2025 oral ()
Keywords: emergence
sparse attention
in-context learning
induction head
Primary area: deep_learning
TL;DR: We show that learning sparse attention is prone to emerging behaviors during training, and study (theoretically and empirically) how data and model design influence emergence speed.

Abstract: Emergence is a fascinating property of large language models and neural networks more broadly: as models scale and train for longer, they sometimes develop new abilities in sudden ways. Despite initial studies, we still lack a comprehensive understanding of how and when these abilities emerge. To address this gap, we study the emergence over training of sparse attention, a critical and frequently observed attention pattern in Transformers. By combining theoretical analysis of a toy model with empirical observations on small Transformers trained on a linear regression variant, we uncover the mechanics driving sparse attention emergence and reveal that emergence timing follows power laws based on task structure, architecture, and optimizer choice. We additionally find that repetition can greatly speed up emergence. Finally, we confirm these results on a well-studied in-context associative recall task. Our findings provide a simple, theoretically grounded framework for understanding how data distributions and model design influence the learning dynamics behind one form of emergence.

The emergence of sparse attention: impact of data distribution and benefits of repetition
Nicolas Zucchet" ETH Zürich
Francesco D' D'Angelo EPFL
Andrew Lampinen Google DeepMind
Stephanie Chan Google DeepMind
Abstract
Emergence is fascinating property of large language models and neural networks more broadly: as models scale and train for longer, they sometimes develop new abilities in sudden ways. Despite initial studies, we still lack comprehensive understanding of how and when these abilities emerge. address this gap, we study the emergence over training of sparse attention, critical and frequently observed attention pattern in Transformers. By combining theoretical analysis a toy model with empirical observations small Transformers trained linear regression variant, we uncover the mechanics driving sparse attention emergence and reveal that emergence timing follows power laws based task structure, architecture, and optimizer choice. We additionally find that repetition can greatly speed up emergence. Finally, confirm these results well-studied in-context associative recall task. Our findings provide a simple, theoretically grounded framework for understanding how data distributions and model design influence the learning dynamics behind one form of emergence.
Scaling has been central to the recent success of large language models [1-5], with scaling laws [6, 7] describing how increased model size, data, and training consistently improve average performance. However, beneath this macroscopic predictability, model performance on specific tasks often reveals capabilities that appear suddenly beyond critical scaling thresholds phenomenon known as emergence [8-11]. While recent studies have begun to characterize how emergence appear after a critical training time [12-18], a comprehensive scientific understanding remains elusive.
This work explores sparse attention a lens understand emergence during training. The formation of sparse attention where Transformers' attention layers focus on a small subset critical tokens coincides with sudden performance improvements in many emergent behaviors, including in-context learning abilities [12, 15] and factual recall [18]. We investigate why the development of sparse attention lead to abrupt performance improvements and reveal how characteristies of the training data influence the speed of emergence. We make two key contributions:
We design variant of linear regression that specifically requires Transformers learn focus on few tokens within the context (Section 2). This analytically tractable task allows us to mathematically characterize, in a toy model, the mechanics behind the emergence of sparse attention and precisely quantify how shorter sequences and repetition accelerate emergence.
We apply our sparse attention framework to explain the emergence of in-context learning in an associative recall task (Section 3). Our theoretical predictions successfully account for how data influences the emergence speed of the induction head that solves this task.
Overall, our results suggest that sparse attention may provide unifying perspective for understanding seemingly diverse emergence phenomena in large language models. They additionally highlight the potential practical benefits repetition for accelerating the formation of specific neural circuits.
*Correspondance to nzucchet@ethz. ch. Advisory capacity only.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Motivation
Can emergence be predicted? Emergence large language models not only challenges our scientific understanding of how they acquire new skills but also poses Al safety issues [19] due to its unpredictable nature, while additionally complicating frontier model development. To address this unpredictability, researchers have proposed several progress metrics under which scaling has more predictable consequences, including validation loss [20], metrics that reward partial progress [21, 22], and those that employ mechanistic interpretability-monn mottivted measures [23]. However, significant limitation is that these metrics typically can only be derived post-emergence [24]. An alternative certain tasks can predict whether
approach demonstrates that fine-tuned models' performance the ability to solve the task will emerge in larger models [17]. Our work explores predictability from different angle: understanding how the data distribution influences the learning time at which emergence occurs.
Is there link between emergence and sparse attention? The driving hypothesis underlying this work is that learning of sparse attention patterns is particularly prone to produce sharp transitions in behavior during training- i.e., the sudden emergence of new capabilities. There is statistical argument behind this intuition: when the target a Transformer has to predict depends only on a few tokens within the context, these tokens initially have very low weight in the prediction of the model, as attention typically starts uniform. Initial progress is therefore slow and the more targeted (thus sparse) attention is, the faster learning becomes. Multiple empirical observations strengthen our hypothesis. The induction head [12), whose formation coincides with the sudden emergence of certain in in-context learning abilities, fundamentally relies the combination of two attention layers with sparse patterns. Similarly, the mechanisms underlying factual recall in large language models [23, 25] demonstrate sparse attention properties and show emergent learning dynamics [18]. These specific emergent phenomena appear to be linked to the development of sparse attention mechanisms, suggesting the existence of potential causal relationship between the two.
The intriguing interplay between repetition and emergence. While data diversity is generally considered a gold standard in machine learning, growing body of evidence suggests that repetition can actually accelerate emergence over training. Chan et al. [13] demonstrated that showing some tasks more often favors the emergence of in-context learning. Charton and Kempe [16] revealed that repeating a subset of examples more frequently than others dramatically accelerates Transformers' ability to solve certain arithmetic tasks. This pattern extends beyond mathematical reasoning, as Zucchet et al. [18] (and to some extent Allen-Zhu and Li [26)) showed that repeating biographies of specific individuals speeds up the development of circuits critical for factual recall from model parameters. Collectively, these results establish repetition as a fundamental property of data distribu- tions that can systematically influence emergence timing, thus justifying integrating it in our model to better understand its role.
2 Theoretical insights on an attention-based linear regression task
To illustrate how emergence can arise from sparse attention learning, we introduce a variant of linear regression that additionally requires selecting relevant token from the input sequence. We introduce this task alongside a minimal attention based toy model for solving (Section 2.1), theoretically analyze its learning dynamics both without (Section 2.2) and with repetition (Section 2.3), and empirically show that our findings extend to more realistic Transformers (Section 2.4).
2.1 The single-location linear regression task
We consider the following supervised learning task. We are given an input sequence (xx))=1 length T in which each token 2; Rd is drawn i..dd from zero-mean normal distribution with variance 1 1/d and aim to predict the target y" given by
y
W' FT
(I)
Here, the target weight matrix W* Bddd a predetermined matrix and y E IRd, To successfully solve this task, an attention-basee model must learn to attend to the last token only and learn the ground-truth target weights W* We deliberately incorporate a sparse attention target mechanism to study the relationship between sparse attention and emergence. This task shares similarities with the

---

Task. Single-location linear regression
b.
10a
d
W'x
10
64
10a
10
10-1
lass
10-3
T tokens in context
10
10-2
10a5
10-1
In-context repetition
10-6
#11
a-3
10k
time
the relevant token appears B times
10a
Cross-sample repetition
256
plateau length
512
#1
22
333
24
PTG
ZGG
10"
1024
2048
4096
24
276
Z6
10a
T plasteaa
1.51 0.9s0 a aa.
data points
scaling law
#1
x2 #a
24
#66
104
107
the same appears with probability
Figure I: A simple task to study the emergence of sparse attention. (left) We introduce variant of linear regression task that analytically tractable and which Transformers-lie models need learn sparse attention. The model must identify which token (here the last one, xr) is relevant for the target output y"- We incorporate two realistic forms of repetition in the data: in-context repetition, where the relevant token appears multiple times within the context, and cross-sample repetition, where an input sequence contains a special token T (here colored in green) at the relevant position with probability p. See Section 2.1 for details. (right) a. As desired, the reduced learning dynamics of our simplified Transformer (Eq. 2) exhibit multi-phase behavior including an initial plateau, on the task without repetition (T 512). Mechanistically, the weights w begin learning before attention to the relevant token (T = 512, d = 64). Dashed lines represent optimal values. The duration of the initial plateau increases as function of sequence length T and input/output dimension d, closely following power law scaling relationship (R² 0.999) that can be accurately predicted by linearizing the dynamics around initialization (Fquation 6). See Section 2.2 for details.
single-locas -lcction regression task of Marion al. [27], although our case, the relevant token location is always the same. The left panel of Figure summarizes this task.
We model two forms of repetition that ubiquitous in natural language:
In-context repetition. This occurs when specific token groups (c.g., a person's 's name) repeatedly appear within single context window. In our framework, model this property by repeating the relevant token 2r multiple times in the sequence (xa (rz)T=1 always at the same positions for simplified model to be able use it. Following Chan al. [13], who termed this property "burstiness", we denote B as the number of times the task-relevant token appears in the context.
-Cross-sample repetition. This form of repetition comes from having certain information (such as biographical details of specific individuals) overrepresented in the overall training data. We implement this by first sampling the input sequence normally, but then occasionally (with probability p) replacing the relevant token øт with special predefined token T.
For the purpose of our theoretical analysis, we introduce simplified attention layer, defined by
y W Σ soft
with T the attention scores vector and W E Rddd the weight matrix. Unlike in standard Transformer attention [28], our model does not use any semantic information determine where to attend. This simplification facilitates theoretical analysis and implicitly assumes that the attention

---

layer has already learned to filter irrelevant contextual information. The model's 's parameters (a, W) learned by minimizing the expected mean square error between predictions and targets y".
2.2
The learning dynamics of the simplified Transformer exhibit emerging abilities
The performance of our simplified model on this task exhibits a characteristic learning pattern: an initial plateau where the loss minimally decreases, followed by sharp phase transition towards significantly lower loss values. The analysis we detail below reveals that this emergent behavior arises from the interaction between feedforward weights and attention during learning, and that the duration of the initial plateau increases with the sequence length T and data dimensionality d.
Reduced learning dynamics. We analyze the gradient flow dynamics of the simplified model. The assumptions and mathematical details of this analysis can be found in Appendix A. Under these mild assumptions, the learning dynamics of the entire model reduces to two key scalar variables: Aa == 4y a for any < T (all attention scores to non-relevant tokens stay the same under our assumptions) and the scalar projection' of W on W These variables are initially both equal to 0 and they evolve according to the following system of ordinary differential equations:
al a(vd aw)
(1 2
d
d(T -T)
(w(Vd. aw) 11-a1 a) d(T 1)
ww(
Aa a(1 aa)
with a a= = (1+ 1) exp( (x)(= the attention given to the final token. In these two equations, the first term is the signal coming from the relevant token at position T. and the second term the noise coming from the T - 1 non-relevant tokens. This decomposition follows from the derivation in Appendix A.3, which groups token contributions by their relevance to the target prediction.
These equations enable us to elucidate the roots of emergence in this task. Initial learning is slow, as Aa does not receive any teaching signal as w and w slowly increases as attention initially uniform (a = 1/T). As a consequence, the feedforward weight W must first align with the target weights W before attention can learn. positive feedback loop is then progressively established: increased attention improves the learning signal for w, and better-learned w further reinforces the correct attention pattern. This dynamic, similar to the one found in deep linear networks [29], leads to sharp decrease in loss and the sudden emergence we observe in Figure 1.a.
Predicting when emergence arise. To estimate how long takes to escape the initial loss plateau, we linearize the dynamics around the initial conditions and obtain
V
VaT O
Aa
Aa
VTT
This linearization provides two key insights: First, it confirms that feedforward weight learning precedes and drives attention learning, as evidenced by the initial gradient and the top-right entry of the matrix in the equation above, and corroborated by the simulations of Figure 1.b. Second, it enables us to estimate the escape time from initial conditions, defined as the time required to reach (1 e) of the initial loss value. It is equal to
T- VdT In (EVAT)
(6)
This formula succinetly demonstrates that both longer sequences and nigher-dimensional inputs increase the time spent on the plateau and delay emergence. This theoretically derived scaling closely matches the one obtained in simulations, as depicted in Figure 1.c (e 0.8 in these simulations).
2.3 Repetition speeds up emergence
Now that we have thoroughly examined the vanilla case, we focus on understanding the effects of repetition. Our analysis reveals that in-context repetition makes the attention pattern to be learned less
ww formally defined (W*, W)r / IR with (+, .)r the Froebenius inner product.

---

In-context repetition
Cross-sample repetition
10
10
10
plateau length
10-1
p plateau length
10
10
0.2
10-3
10-3
10a
10a
0..0
10:5
10-5
.2
.16
5k
10k
10a
10a
5k
10k
10a
10a
time
time
Figure 2: Repetition speeds up emergence in the linear regression task in a theoretically predictable way. (left) Increasing in-context repetition through B reduces the initial plateau, and the length of the plateau well captured by the power law Tplatean 1.51 7.999 B B-0.99 0 (R²a = 0.999). (right) Cross-sample repetition, modulated by the repetition probability p. exhibits similar effects, even when evaluating the model on a test loss without repetition (i.c., = 0). The length of the plateau follows Tplateau = 2.15(vdT) Vpdd+ + (1 - p)3)1.02 (R² = 0.992). The plateau length in both cases closely follows theoretical predictions. See Section 2.3 for details.
sparse, thereby simplifying the task, while cross-sample repetition accelerates feedforward weight learning in specific directions, which subsequently increases overall learning speed by increasing the attention of the model to relevant tokens earlier. We present the key findings below.
In-context repetition. The role of in-context repetition is rather simple. When relevant information repeats multiple times within the context, it becomes more correlated with the token to predict. This can be understood as increasing the signal to noise ratio or as effectively reducing the sequence length from T to T/B. Since learning time scales with sequence length, this makes the task fundamentally easier. Theoretically, we can show, cf. Appendix A.3, that this intuition precisely holds and that the escape time from the initial loss plateau becomes proportional to Tvd/B. Replacing T by T/ B in the scaling law we obtained in Figure 1.c yields an almost perfect empirical fit, Figure 2.b. This result highlights that B reduces the sparsity of the target attention pattern and thus accelerates emergence.
This analysis ignores, by design, any ability of the attention mechanism to flexibly use semantic or positional information, as we directly parameterized the attention scores. We argue that our findings will extend to the more general case. Indeed, both the attention scores, which would now be the output of some function, and the feedforward weights receive larger gradients with in-context repetition, and thus learning will overall be faster. We will confirm that the same conclusion holds empirically for more realistic architectures and optimizers in the next sections.
repetition. The role of cross-sample repetition is more intricate. Repeating one token more frequently, that is increasing p. causes input covariance matrix of the relevant token, E. to become anisotropic. The different components of the weight matrix W/ are then Iearned different speeds. While this difference learning speed traditionally leads to slower convergence rates in vanilla linear regression [30], it turns out to be beneficial in our attention- based version of the task. Indeed, following similar principles weights in the repeated direction will lead to the attention to the relevant tokens increasing faster, and this then speeds up the learning of the non-repeated dimensions. As consequence, model escapes the initial learning plateau faster. Importantly, this also holds when measuring the model's performance on non-repeated data, as seen in Figure 2. The effect of sample repetition can also be understood increasing the signal- coaaaal-tonoiisss noinei ratio. Repeating the same token increases the amount of signal coming from the relevant token while keeping the amount of noise received from other tokens fixed, at the price of losing some information about the relevant signal.
the ones detailed in the previous section, learning
Theoretical analysis requires the introduction of an additional variable, so that the learning speed in both the repeated dimension and the other dimensions can be tracked independently. This model, which we introduce in Appendix A.4, enables us to justify the mechanistic insights mentioned above, as well as to derive that the plateau length scales as vdT/ / pdd + (1 p)) which accurately
describes empirical behavior (cf. Figure 2). The repetition probability p thus implicitly interpolates
5

---

placeau length
104
plateau length
plateau length
104
20
107
10'
T
P
16
32
64 128
16
64
128
16
32
128 256
Figure 3: Theoretical insights on the linear regression task transfer to more realistic versions of the task, the model, and the optimizer. Transformer-basee architectures exhibit similar phase transitions
toy model and its corresponding plateau length follows similar trends to the ones derived in theory. (left and middle) Evolution of the plateau length as function of d, when varying T and (by default T
256 and B 1). The lines corresponds to the power law platean = 0.76 d 1.25 TO.80 B 0.80 (R² = 0.995). (right) Same plot, this time varying the cross- sample repetition probability p. The lines correspond the evolution of the average plateau length as d increases, for the different values. See Section 2.4 for details.
between the d-dimensional case and the 1-dimensional case, highlighting that the learning of the feedforward mapping becomes less of a bottleneck.
However, cross-sample repetition is not free lunch: it only provides temporary advantage. First, these dynamics only have a smaller test loss in the medium term. In the long run, learning is slower for similar reasons for the standard linear regression. Second, there is additional overfitting problem. While it does not appear in this simplified setting as anisotropic input covariance does not bias the final solution, starts appearing for more realistic architectures. This phenomenon has been observed in practice: examples include [31, 16, 18) for synthetic tasks that have properties similar to the one we focus here, and [32, 33] for the pretraining of large language models.
2.4 The theory qualitatively captures learning dynamics of full-fledged Transformers
We conclude our linear regression analysis by examining how our theoretical predictions extend to more realistic task versions, optimizers, and models particularly those with standard attention that varies with inputs. Our findings, detailed in the remainder of this section, indicate that learning dynamics behave qualitatively similarly, showing sharp phase transitions, with emergence time maintaining similar dependencies on data properties. However, the precise dependencies differ, which we investigate in more depth.
For this investigation, train standard 2-layer 4-heads Transformer with the Adam optimizer [34], using a constant learning rate of 10-4, Our only deviation from standard architectures is removing layer normalization, which makes solving the task more challenging. To enhance task realism, we randomly sample the positions of relevant tokens and incorporate an additional feature into the input indicate whether a token is task-relevant. Further experimental details are provided in the appendix.
The learning dynamics of Transformers align qualitatively with our theoretical predictions. First, the loss evolution exhibits sharp phase transition (see Figure 11 in the appendix for example). Second, emergence time depends similarly on the data properties identified in our theory, with repetition accelerating emergence. However, this result comes with several nuances.
For scenarios with no repetition or with in in-context repetition, power laws still accurately capture the dependency of emergence time sequence length T, dimension d and burstiness B, though with significantly different exponents. Ablations (see Appendix B.3) reveal that optimizer, architecture, and task specifics all influence these exponents. Notably, replacing Adam with SGD substantially slows emergence, both in absolute terms and by increasing dependency on task difficulty. This finding illustrates broader observation: Adam is crucial for efficient Transformer learning [e.g.. 35).
Regarding sross- repetition, power laws no longer accurately describe the empirical relation- ship. partly because measuring plateau length becomes more difficult (see the learning dynamics

---

106
1.0
number pairs
test accuracy
0.8
plateau length
20a
0.6
number
16
10
pairs
32
10a
0.2
number pairs
10a
8
16
32
64
0.0
50k
100k
150k
200k
0
50k
100k
150k
200k
32
128
256
steps
steps
vocab size
Figure 4: In-context learning emerges in the associative recall task, and emergence time grows with the number of pairs in the context and the vocabulary size. (left and middle) The ability solve the task emerges through training, with emergence time increasing as the task gets harder (by increasing the number of pairs here, the vocabulary size being fixed to 256). This is qualitatively consistent with the theory developed for sparse attention. (right) Systematically investigating the relationship between emergence time and data properties reveals that it follows the power law plateean 0.55 N7.79 N2.25 (R² 0.982). Results are only shown when the number of pairs is paars
"tokens
larger than the vocabulary size, to ensure that the query does not appear multiple times in the context (we do not consider repetition here). More details, particular a hypothesis for why the Naairs exponent is so high, can be found in Section 3.2.
under cross-sample repetition in Figure 12 in the appendix for an example). Nevertheless, the trends identified in our theory still hold: this form of repetition accelerates emergence, with the effect becoming more pronounced as d increases.
3 Emergence of in-context learning, through the lens of sparse attention
We conclude by investigating what extent the insights developed on our simple linear regression task extend to more realistic learning problems, in particular one in which in-context learning emerges. To this end, we examine an in-context associative recall task, which can be solved by learning an induction head
well-studied circuit strongly implicated in in-context learning [12, 15), which necessitates at least two attention layers with sparse attention patterns. Our theory qualitatively captures both the learning dynamics and the impact of the training distribution on learning speed.
3.1 The in-context associative recall task
The in-context associative recall task serves as standard benchmark for testing language models' ability to access information in their context and to perform a simple form of one-shot learning. This task requires abilities that correlate with models' capacity for in-context learning [12], and variations of it have been extensively studied to better understand how in-context learning emerges [13, 15, 36- 40]. It has also been used as a testbed for recently proposed linear recurrent architectures [c.g. 41-43], as it acts as an important differentiator between attention- and recurrent architectures [44].
We implement this task as follows: each sequence consists of Npairs key-value token pairs (5 such pairs the example below) followed by a query token (Z below), as shown:
Y
I A
UR
Z Y CA
Z?
The query corresponds to of the keys in the context, and the model must output the corresponding value - in the example above (since the query Z matches the key in the pair Z Y). practice, we work with total of Ntokens unique tokens provided the network via one-hot encodings. We train the model using a cross-entropy loss comparing the model output to the target value.
The number of pairs N Npaite controls the sequence length T as 7 2N 2N query token), and the vocabulary size Ntokens plays role comparable to the input dimension d in the linear regression task. To model in-context repetition, average times the context (as a key). To model cross-sample repetition, we select a subset of tokens that will appear more often? and vary p. the probability to sample the query from this subset.
pairs + (accounting for the
ensure that the query token appears
22 is arbitrary choice that have not ablated.
7

---

The precise description of the task is provided in the appendix. The results we report in the main text are testing the learned models on data without any repetition, thereby testing the generalization abilities of models learned on repeated data.
Transformers typically solve this type of task by implementing an induction head circuit that combines an attention layer responsible for concatenating the representation of the current token with the previous token, and a selection attention layer responsible for retrieving the relevant information from the context. Both attention layers focus on a sparse subset of tokens (usually just one), making this task particularly well-suited to test our theory. Unlike the linear regression task where sparse attention was hard-coded in the target input-ouiput mapping, there is no inherent constraint forcing models to develop sparse attention here. We note that our toy model (Equation 2) cannot directly implement an induction head, as it lacks the relative positional information for copying and semantic similarity mechanisms for token matching. This task thus serves as the perfect testhed to show that our theoretical insights extend to more realistic, yet still finely controlled, learning scenarios in which their simplifying assumptions do not hold.
3.2
The emergence of in-context learning depends on data as in the theory
As with the linear regression task, we investigate the learning dynamics of Transformers on this task and compare the qualitative findings with those of our developed theory. The experimental setup being closely related to the one in Section 2.4, we defer its thorough description the appendix and note that we use layer normalization and MLPs for this set of experiments.
Overall, we find that theory accurately describes both the learning dynamies (in-context learning emerges in this task) and the dependency of emergence timing on data distribution properties.
Longer sequences and larger vocabulary size delay emergence. We begin by verifying that the intuition described above applies and that the learning curve exhibits the sharp phase transitions characteristic of emergence. This is indeed the case for sufficiently long sequences and/or large vocabularies, as illustrated Figure 4. Importantly, there is only one phase transition despite two attention layers being needed to learn sparse patterns; Figure 17 shows that they are learned simultaneously. This observation is consistent with the results of Reddy [36] and Singh et al. [15].
In Figure 4 (right), we report how the emergence time, arbitrarily defined as the time needed to reach 5% accuracy, evolves power law, as in all our other experiments, albeit with exponents that are relatively low for Neokens (0.79) and extremely high for N Npains (2.25). We hypothesize that the lower exponent for Niokeus stems from the fact that most of the feedforward processing can be handled by residual connections, and given that the dimension of the data primarily influences the speed of feedforward learning, data dimension does not have such large effect. For the higher exponent for N Npairs. our hypothesis is that this occurs as two sparse attention patterns must be learned jointly. the toy model we analyze theoretically, similar coupling cxists between the attention and the feedforward weights, resulting in a multiplicative interaction. We posit that a similar mechanism may be at work here, with each attention layer contributing additively to the N Npairs exponent. We leave more thorough investigation of how different circuits interact to influence emergence time to future work.
function of sequence length and data dimension. It accurately follows a
Repetition speeds up emergence but comes with overfitting risks. We next investigate the benefits of in-context and gross-sample repetition. From the high sensitivity of the emergence time on sequence length revealed by previous analysis, we expect in-context repetition to have a stronger effect than the cross-sample one. This is what we observe. The results reported in Figure 5, which evaluate the model performance on test data that has no repetition, demonstrate that repetition can significantly speed up emergence, by a factor of 4 for in-context repetition and a factor of 2 for cross-sample repetition. The attention sparsity perspective thus explains why in-context repetition has been found to favor in in-context learning vs. in-weight learning [13, 36, 45]: the induction head needed to solve this kind of task is formed earlier with such form of repetition.
However, this benefit comes with an overfitting risk: while repetition consistently accelerates learning (repetition always speeds up learning on the train loss, cf. Figure 16 in the appendix), too much repetition leads to learning strategies that do not generalize well. For example, selecting the most frequent value is valid strategy for this task whenever the query appears two or more times in the context. Interestingly, we also observe some grokking-like patterns [46] as the test accuracy eventually starts to increase late in training for large amounts of repetition. We argue that this occurs
8

---

In-context repetition
Cross-sampll repetition
1.00
1.00
test accuracy
test accuracy
bast loss
best loss
4
0.25
2
0.25
2.
0.00
0
0.00
50k
100k
150k
0
50k
100k
150k
50k
100k
150k
50k 100k
150k
steps
steps
steps
steps
0.B
Figure 5: Repetition speeds up emergence in the in-context associative recall task but comes with overfitting risks. (left) We vary the amount of in-context repetition B, that is the number of times the query appears as key in the context on average, and find significant benefits of small amount of repetition. Larger amounts of repetition lead to overfitting, but learning for long enough eventually leads to grokking. (right) Cross Cross-sample repetition, more precisely the probability p that the query one of the repeated tokens, has similar effects. Results are obtained for Npairs 32 and Nooeess 256.
because the no-repetition case still represented in the training data, albeit with very little weight. This trade-off between learning speed and generalization is consistent with what Park et al. [31] reported on another in-context learning task. The overfitting we observe here may appear inconsistent with our theory but is not: our theory addresses learning speed (performance on the training loss) rather than generalization ability.
4 Discussion
Connection with other theoretical work. Through the lens of sparse attention, we provide a unifying perspective set of empirical results highlighted in the motivation (Section 1). While our focus on sparse attention and the effect repetition is unique (to the best of our knowledge), there multiple closely related works. First, the interaction between the feedforward weights and attention we study is closely related to the one between two consecutive feedforward linear layers [29, 47, 48). Linear networks also display incremental learning with abrupt transitions characterizing each phase change [49-52]. This line of research on linear networks has since then been extended linear [45, 53] and softmax [54] Transformers, in particular study the emergence of in-context learning. On the more conceptual side, there exist other theories of how emergence could arise from longer training, such as grokking [46, 14] singular learning theory [55, 56). While rarer, other theories focus on emergence from increased model size [57]. Compared to these, our theory focuses on learning dynamics (emergence over the course of training) and is directly tied to the internal mechanisms of the Transformers.
How much emergence comes from sparse attention? We find that the learning of sparse attention prone producing emergence over the course of training. Given that sparse or concentrated attention is common emergent feature of Transformers (e.g.. induction heads [12), task/function vectors [58, 59], high-norm tokens in vision transformers [60]), may ask how much currently known emergent behaviors can be attributed to the learning of sparse attention patterns, and whether there are many more emergent behaviors than what is currently reported. However, these points must be weighed. As noted above, even simpler MLPs can give rise to abrupt emergence over training [c.g. 29]; thus, learning other circuits within Transformers might contribute to sudden transitions in their performance. Furthermore, emergence results at large scale mostly show sharp phase transition of the model as the number of FLOPs [e.g., 8]. which roughly corresponds to the model size times the number of training steps, reaches certain threshold. It is therefore unclear whether these examples of emergence are rooted in longer training, models with higher capacity, or a complex interplay between the two. More empirical and theoretical work is needed to better understand the causes of emergence large models and the possible complex relationship between training-basen and based emergence.

---

When does repetition help? Overall, we expect repetition to be beneficial when training on tasks prone to performance plateaus, as there is a clear benefit in trading off learning speed for generalization in these cases. Cross-sample repetition should be particularly helpful in tasks where learning feedforward transformations challenging (high
our analysis). This includes learning many factual associations as in Zucchet et al. [18] and the reasoning-heavy arithmetic tasks of Charton and Kempe [16], which require learning complex non-linear transformations. In-context repetition should be most beneficial when learning from very long sequences, as it effectively reduces the sequence length the model must process. Repetition, both in-context and cross-sample, is a natural touch
feature of language that may accelerate some parts of language learning. The principles upon could already be more directly at play in the current training pipeline of large language models, for instance when code, which typically has lower syntactic diversity than other types of text [61], is included at specific moments during training.
Data diversity as a path towards active learning? A key result of our work is that low data diversity can actually improve performance. This contrasts with classic machine learning principles, which state that low diversity hurts generalization. These two apparently conflicting statements are actually compatible. As our results highlight, low diversity initially accelerates the learning of sparse attention, but high data diversity is better training time goes to infinity. We hypothesize that data diversity might be a powerful lever towards enabling active learning [62]. Specifically, our findings suggest a simple active learning algorithm: whenever agent detects it is stuck on task, it can decrease data diversity to accelerate learning, then increase diversity after the critical transition occurs to improve generalization. This dynamic adjustment of diversity provides practical mechanism for agents to actively control their own learning trajectories, with the ultimate goal of letting the learner decide what it wants to learn on and reaching human-level sample efficiency that current deep learning systems crucially lack (63-65), Humans also encounter varying levels of data diversity over development. For example, infants receive progressively increasing data diversity during their early years [66]. Our theory suggests that it may be an important factor in accelerating our development. While promising, this thread requires more extensive analysis, both at larger scale and on more realistic data distributions.
Acknowledgments
The authors thank Jay McClelland for discussions that inspired this work. F.D. thanks Aditya Varre for the useful discussions.
References
[1] Yoshua Bengio. Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model. Journal of machine learning research, 3, 2003.
[2] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and others. Language models are unsupervised multitask learners. OpenAl blog, 18))9, 2019.
[3] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subhiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and others. Language models are few-shot learners. Advances in neural information processing systems, 2020.
[4] OpenAL, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and others. GPT-4 technical report, arXiv preprint arXiv:2303.0877 2023.
[5] Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Cean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and others. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312. 11805, 2023.
[6] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.088 00330000000.0..00.00000000000610 2020.
[7] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, and others. Training compute- optimal large language models. arXiv preprint 220.:222223155566 2022.
10

---

[8] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogalama. Maarten Bosma, Denny Zhou, Donald Metzler, and others. Emergent abilities of large language models. Transactions on machine learning research, 2022.
[9] Deep Ganguli, Danny Hernandez, Liane Lovitt, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova Dassarma, Dawn Drain, Nelson Elhage, and others. Predictability and surprise in large generative models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1747-1764, 2022.
[10] Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shocb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga- -ilnso,,,oo,,...... and others. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint 246.412206.04 2022.
[11] Philip W Anderson. More different: Broken symmetry and the nature of the hierarchical structure of science. Science, 177(4047), 1972.
[12] Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, and others. In-context learning and induction heads. Transformer circuits thread, 2022.
[13] Stephanie Chan, Adam Santoro, Andrew Lampinen, Jane Wang, Aaditya Singh, Pierre Richemond, James McClelland, and Felix Hill. Data distributional properties drive emergent in-context learning in transformers. Advances in neural information processing systems, 2022.
[14] Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacoh Steinhardt. Progress measures for grokking via mechanistic interpretability. International conference on learning representations, 2023.
[15] Aaditya K Singh, Ted Moskovitz, Felix Hill, Stephanic CY Chan, and Andrew M Saxe. What needs to go right for an induction head? a mechanistic study of in-context learning circuits and their formation. International Conference on Machine Learning. 2024.
[16] François Charton and Julia Kempe. Emergent properties with repeated examples. arXiv preprint arXiv:2410.0704), 2024.
[17] Charlic Snell, Eric Wallace. Dan Klein, and Sergey Levine. Predicting emergent capabilities by finetuning. In Conference Language Modelling, 2024.
[18] Nicolas Zucchet, Jörg Bornschein. Stephanie Chan, Andrew Lampinen, Razvan Pascanu, and Soham De. How do language models learn facts? Dynamics, curricula and hallucinations. arXiv preprint arXiv:2503.21676. 2025.
[19] Usman Anwar, Ahulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, and others. Foundational challenges assuring alignment and safety of large language models. arXiv preprint arXiv:2404.0993: 2024.
[20] Zhengxiao Du Aohan Zeng, Yuxiao Dong, and Jie Tang. Understanding emergent abilities of language models from the loss perspective. Advances in neural information processing systems, 2024.
[21] Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are emergent abilities of large language models a mirage? Advances in neural information processing systems, 2023.
[22] Rosie Zhao, Tian Qin, David Alvarez-Melis, Sham Kakade, and Naomi Saphra. Distributional scaling laws for emergent capabilities. arXiv preprint 20XIv:2502.17356 2025.
[23] Neel Nanda, S Rajamanoharan, J Kramár, and R Shah. Fact finding: Attempting reverse-engineer factual recall the neuron level. In Al alignment forum, 2023, 2023.
[24] Boaz rage,
abilities and grokking: https://vindossothherryy wtt.heerrrrr.rr///3////o org/2023/12/22/ emergent mmergent-abilitien and honddgrooking--- fundamental -irage----------- or both/.
Barak.
Emergent
Fundamental, Mi-
both?. 2023.
URL
[25] Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. Dissecting recall factual associations
in auto-regressive language models. Conference on empirical methods in natural language processing, 2023.
[26] Zeyuan Allen Zhu and Yuanzhi Li. Physics of language models: Part 3.1, knowledge storage and extraction. arXiv preprint arXiv:2309. 2023.
11

---

[27] Pierre Marion, Raphaël Berthier, Gérard Biau, and Claire Boyer. Attention layers provably solve single- location regression. In International conference learning representations, 2024.
[28] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, 2017.
[29] Andrew M Saxe, James L McClelland, and Surya Ganguli, Exact solutions to the nonlinear dynamies of learning in deep linear neural networks. International conference learning representations, 2014.
[30] Francis Bach. Learning theory from.frst principles. 2024.
[31] Core Francisco Park, Ekdeep Singh Lubana. Itamar Pres. and Hidenori Tanaka. Competition dynamics shape algorithmic phases of in-context learning. arXiv preprint arXiv:2412.01003 2024.
[32] Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burie Burohh and Nicholas Carlini. Deduplicating training data makes language models better. In Annual Meeting of the Association for Computational Linguistics, 2021.
[33] Danny Hemandez, Tom Brown, Tom Conerly, Nova DasSarma, Dawn Drain, Sheer El-Showk, Nelson Elhage, Zac Hatfield- Dodds, Tom Henighan. Tristan Hume, and others. Scaling laws and interpretability of learning from repeated data. arXiv preprint arXiv:2205. 10487, 2022.
[34] Diederik P. Kingma and Jimmy Ba. Adam: a method for stochastic optimization. International Conference on Learning Representations, 2015.
[35] Yushun Zhang. Congliang Chen, Tian Ding, Ziniu Li, Ruoyu Sun, and Zhiquan Luo. Why transformers need Adam: A Hessian perspective. Advances in neural information processing systems, 37, 2024.
[36] Gautam Reddy. The mechanistic basis of data dependence and abrupt learning in-context classification task. International conference on learning representations. 2024.
[37] Alberto Bietti, Vivien Cabannes, Diane Bouchacourt, Herve Jegou, and Leon Bottou. Birth of transformer: A memory viewpoint. Advances in neural information processing systems, 2023.
[38] Ezra Edelman, Nikolaos Tsilivis, Benjamin Edelman, Eran Malach, and Surhhi Goel. The evolution of statistical induction heads: In-context learning markow chains. Advances in neural information processing systems, 2024.
[39] Eshaan Nichani, Alex Damian, and Jason D Lee. How transformers learn causal structure with gradient descent. International conference on machine learning, 2024.
[40] Francesco D' Angelo, Francesco Croce, and Nicolas Flammarion. Selective induction heads: How trans- formers select causal structures in context. In International conference on learning representations, 2025.
[41] Soham De, Samuel L Smith, Anushan Fernando, Aleksandar Botev, George Cristian-Muraru, Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, and others. Griffin: Mixing gated linear recurrences with local attention for efficient language models. arXiv preprint arXiv:2402.19422 2024.
[42] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. Conference on language modeling, 2024.
[43] Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, and Yoon Kim. Parallelizing linear transformers with the delta rule over sequence length. In Advances in neural information processing systems, 2024.
[44] Simran Arora, Sabri Eyuboglu, Aman Timalsina, Isys Johnson, Michael Poli, James Zou, Atri Rudra, and Christopher Ré. Zoology: Measuring and improving recall in efficient language models. In International conference on learning representations, 2023.
[45] Aaditya Singh, Stephanie Chan, Ted Moskovitz, Erin Grant, Andrew Saxe, and Felix Hill. The transient nature of emergent in-context learning in transformers. Advances in neural information processing systems, 2024.
[46] Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Generalization beyond overfitting on small algorithmic datasets. arXiv preprint 2200072201.02.17,, 2022.
[47] Andrew M Saxe, James L McClelland, and Surya Ganguli. A mathematical theory of semantic development in deep neural networks. Proceedings of the National Academy of Sciences, 116(23), 2019.
12

---

[48] Aditya Vardhan Varre. Maria-Luiza Vladarean, Loucas Pillaud-Vivien, and Nicolas Flammarion. On the spectral bias of two-layer linear networks. Advances in neural information processing xystems, 2023.
[49] Arthur Jacot, François Ged, Berfin Şimçek, Clément Hongler, and Franck Gabriel. Saddle-to-saddle dynamics in deep linear networks: Small initialization training, symmetry, and sparsity. arXiv preprint arXiv:2 106, 15933, 2021.
[50] Emmanuel Abbe, Enric Boix Adsera, and Theodor Misiakicwicz. The merged -erreaestaircase property: a necessary and nearly sufficient condition for sgd learning of sparse functions two-layer neural networks. In Conference on learning theory, 2022.
[51] Scott Pesme and Nicolas Flammarion. Saddle-to so-ddeee dynamics in diagonal linear networks. Advances in neural information processing systems, 2023.
[52] Emmanuel Abbe, Euric Boix Adsera, and Theodor Misiakiewicz. SGD learning on neural networks: Leap complexity and saddle-to-saddle dynamics. In Conference on learning theory, 2023.
[53] Yedi Zhang, Aaditya K Singh, Peter E Latham, and Andrew Saxe. Training dynamics of in-context learning in linear attention. arXiv preprint arXiv:2501.1626 2025.
[54] Siyu Chen, Heejune Sheen, Tianhao Wang, and Zhuoran Yang. Training dynamics of multi-head softmax attention for in-context learning: emergence, convergence, and optimality. In Conference on learning theory, 2024.
[55] Sumio Watanabe, Algebraic geometry and statistical learning theory. Cambridge University Press, 2009.
[56] Jesse Hoogland, George Wang. Matthew Farrugia-Roberts, Liam Carroll, Susan Wei, and Daniel Murfet. The developmental landscape of in-context learning. arXiv preprint arXiv:2402.02.66 2024.
[57] Sanjeev Arora and Anirudh Goyal. theory for emergence of complex skills in language models. arXiv preprint ar Xiv:2307. 15936, 2023.
[58] Roce Hendel, Mor Geva, and Amir Globerson. In-context learning creates task vectors. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the association for computational linguisties: EMNLP 2023, 2023.
[59] Eric Todd, Millicent L Li. Arnab Sen Sharma, Aaron Mueller, Byron C Wallace, and David Bau. Function vectors in large language models. International conference on learning representations, 2024.
[60] Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In International conference on learning representations, 2024.
[61] Abram Hindle, Earl T Barr, Mark Gabel, Zhendong Su, and Premkumar Devanbu. On the naturalness of software. Communications of the ACM, 59(5), 2016.
[62] Burr Settles. Active learning literature survey. University of Wisconsin-Madison, Department of Computer Sciences, 2009.
[63] Brenden Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 350(6266), 2015.
[64] Volodymyr Mnih, Koray Kavukeuoglu, David Silver, Andrei A. Rusu, Joel Veness, Mare G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, loannis Antonoglou, Helen King. Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning. Nature, 518(7540), 2015.
[65] Alex Warstadt, Aaron Mueller, Leshem Choshen, Ethan Wilcox, Chengxu Zhuang, Juan Ciro, Rafael Mosquera, Bhargavi Paranjape, Adina Williams, Tal Linzen, and others. Findings of the BabyL. challenge: Sample-efficient pretraining developmentally plausible corpora. arXiv preprint arXi::2504008165. 2025.
[66] Linda Smith, Swapnaa Jayaraman, Elizabeth Clerkin, and Chen Yu. The developing infant creates curriculum for statistical lcarning. Trends in cognitive sciences, 22(4), 2018
[67] Jeremy M Cohen, Simran Kaur, Yuanzhi Li. Zico Kolter, and Ameet Talwalkar. Gradient descent on neural networks typically occurs at the edge of stability. In International Conference on Learning Representations, 2021.
13

---

[68] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary. Dougal Maclaurin, and Skye Wanderman-Milne. JAX: composable transformations of Python+ NumPy programs, 2018. URL http://github.. com/google/jar.
[69] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf. Edward Yang. Zachary Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: an imperative style, high-performance deep learning library. In Advances in neural information processing systems, 2019.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract enumerates the principal contributions (the toy linear-regression analysis and the associative-recall study) and these are precisely the contributions developed in Sections 2 and 3 of the paper, with additional claims beyond that scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contribu- tions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The assumption of the theory are thoroughly discussed Appendix A.I. The limitations of repetition are mentioned throughout the paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not diseussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to vi- olations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. general. empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that inffuence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images taken in low lighting. Or speech-to- -extexxt system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by review- ers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions favor transparency play an important role in developing norms that preserve the integrity the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification: Appendix A outlines the modeling assumptions, derives the ODEs and supplies full proofs for both the no-repetition and repetition cases.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by
formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
Experimental result reproducibility
Question: Does the paper fully disclose all the information needed reproduce the main experimental results?
Answer: [Yes]
Justification:
The Appendix specifies training details, and the code used perform experiments is provided as supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible verifiable.
Depending on the contribution, reprodueibility can be accomplished various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, if the contribution is a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case a large language model), releasing of model checkpoint, other means that appropriate to the research performed.
While NeurlPS does require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (c.g., with open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-son dlosed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but should be possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code?
Answer: [Yes]
Justification: Code is provided as supplementary material.
16

---

Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines
Saddeee/CodSuumiia ssioommmiss for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips. Stalc//guieee//oeeeeeeee
osmoiiiiiiclccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
Experimental setting/details
Question: Does the paper specify the training and test details necessary to understand the results?
Answer: [Yes]
Justification:
All experimental details are specified in the appendix, as well as the code base.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars or other statistical-sinnificann information?
Answer: [Yes]
Justification: Figures report point and R fits to power-laws, but do not include confidence intervals, standard deviations or hypothesis tests.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not show tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
17

---

If error bars are reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the computer resources used?
Question:
Answer: [Yes]
Justification: These details are provided in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experi- mental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
Code of ethics
Question: Does the research conform with the NeurlPS Code of Ethics?
Answer: [Yes]
Justification: The work is purely theoretical/simulttion based, uses only synthetic data, and raises no privacy or human-subject concerns; no deviations from the Code of Ethics are indicated.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consideration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive and negative societal impacts?
Answer: [NA]
Justification: This work aims gaining fundamental insights the training of large language models and has therefore no direct societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g.. deploy- ment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative upplications, the authors should point out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
18

---

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question: Does the paper describe safeguards for responsible release of high-risk data or models?
Answer: [NA]
Justification:
No models or datasets with foresecable misuse potential are released.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are licences for external assets properly cited?
Answer: [Yes]
Justification: The paper reports the frameworks used to train neural networks.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL..
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi www.whhthod....... com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented?
Answer: [Yes]
Justification:
The code is well documented, in particular, on how reproduce experiments.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL include anonymized zip file.
14. Crowdsoureing and research with human subjects
19

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text instructions given to participants and screenshots, applicable, as well as details about compensation (if any)?
Answer: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non- standard component of the methods in this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Question:
Answer: [NA]
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: : h/t:://neurreeee http:pss.cccoooeeeeeeeeeeeeeeee cc/Connfrrences//2222 200p:///cccc/2/2/2LLL/ for what should should not be described.
20

---

Appendix
Table of contents
Details of the theoretical results
22
A.1 Overview of the assumptions
22
A.2 Derivation of the expected loss and its gradients
22
A.3
Analysis of the dynamies without cross-sample repetition
23
A.3.1
Reducing the learning dynamics to two dimensions
23
A.3.2
Approximate dynamics around initialization
25
A.3.3
Phase transition and late phase dynamics
27
A.3.4
Empirical validation
28
A.4
Analysis of the dynamies with cross sample repetition
28
A.4.1 Reducing the learning dynamics three dimensions
29
A.4.2 Approximate dynamics around initialization
30
B Details and additional analysis for the linear regression task
31
B.1
Implementation the task
31
B.2 Examples learning curves
32
Additional ablations on the impact task variation, model optimizer on plateau length
32
Details and additional experiments for the in-context associative recall task
35
C.I Implementation of the task
35
C.2 Learning dynamies and additional analysis
35
D Methods
37
D.1 Measurement of the plateau length
37
D.2 Fitting power laws on plateau length Architectural and training details
37
D.4 Reproducibility
38
21

---

A Details of the theoretical results
A.1 Overview of the assumptions
Throughout theoretical analysis, we introduce few assumptions to simplify it. We summarize them here:
Asm. Gradient flow dynamics. To study learning dynamics, we focus on the gradient flow of the expected loss. Compared to the standard deep learning regime, this removes stochastic noise induced by sampling, uses gradient descent as optimizer instead of optimizer with adaptive learning rates such as Adam, and requires an infinitely small learning rate. Due to the last point, phenomena such as the edge of stability [67] are out of the picture. Yet, this is very standard assumption (e.g. [29]) and will enable us to use tools from dynamical systems to understand how the behavior of the model changes over the course of learning.
Asm. 2 Uniform attention at initialization. At initialization, the attention patterns of Transform- ers generally display remarkable uniformity at initialization. We take advantage of this observation in our model by initializing the dynamics of the attention part of the model to a single scalar.
vector of zeros. This enables us reduce
A W' has norm 1 columns. We assume that the columns of W* all have norm 1. While this is not strictly necessary for performing our analysis, simplifies the formulas that we obtain and helps to keep the notation concise. Additionally, this is the expected behavior when drawing the entire of W i.i.d. from a zero-mean normal distribution with variance 1/ when d goes to infinity.
Asm. 4 Initialize the weights W at O. When drawing the entries of W i..d. and independently of those of W W is almost surely orthogonal to W as + co. The components of W not aligned with W* exhibit fast decay towards 0, so this assumption corresponds to having already completed this process. With such an assumption, we can reduce the dynamics of the weights single scalar, w. which will be the projection of W on normalized W
Asm.
Large sequences and large dimension. Finally, we assume that we are the large T limit and B is negligible in front of T. This makes sense in our context as we are interested modeling the learning of sparse attention patterns. We additionally assume that d is large, which is reasonable assumption given that are interested in modeling the high-dimensional data neural networks have to process. These assumptions will enable us to ignore some negligible terms and keep our calculations more concise.
A.2 Derivation of the expected loss and its gradients
Recall that the loss is given by
L
2 Ez [ll y 1-1
with
y W
softmax Zt
and
5" W*TT. ET-
For notational clarity, we define attention patterns as a, == softmax(a). Without loss of generality, we assume that the tokens repeated in tokens appear at positions -T- B ... T) so that the last B tokens are always the same, and that the repeated input ă is the first vector of the canonical basis of Fd that is ã [1,0,-... For the sake of conciseness, we denote by
Σ, = En [
the input covariance of each token. Note that for 1 < T B, it is equal to Σ, = Lld.
This amounts to right multiplication of the weights W by orthogonal matrix.
22

---

Using the insights mentioned above and the properties of the data distribution, the loss reduces to:
Bz
[1y
y/ [|a]
2
Bz
Σ a2 Wx
W' zT
1 B
Σ
af W x/|12
Σ
aj aWWr; 24 W' FT
2
BST-B
1>T-BB
1
B2
2 W 2
+
Σ
WWWTT W 2T
1>T-B
((
or (WE,WT)) + tr
Σ
o,W W
Er
Σ
oeW W
2
KTT-B
1>T-B
1ST-B
WI tr ((I
Σ
Σ KST-B
a.W - W
ΣT
Σ
o.W W.W- W*
2
(TT-B
In the third line, we used the fact that the first T - B tokens are independent of each other and independent of the last ones, in the fourth line that the last B tokens all equal to 24 and in the fifth line the equality [x Az|l| tr( (r(AR[zxx AAA).
Leveraging this formula, we get the following gradients for the loss:
at Er Σ
VwL= = Σ at Σ
a.W SaW W
W
(7)
d
1>T-B
1ST-B
KT--B
a-W--W - w: )
tr WET
Σ M-T-B
a.W
if T B
V a.
L
(8)
at
||]]2
otherwise.
Note that we have not calculated the gradients with respect to a here, but only with respect to a.
A.3 Analysis of the dynamics without cross-sample repetition
In this section, we study the dynamics without cross-sample repetition, that is, with Prepent = 0.
A.3.1 Reducing the learning dynamics to two dimensions
In general, the parameters evolve in a high-dimensional space. However, with few reasonable assumptions, it possible to show that they evolve in a 2-dimensional subspace:
Attention initialized uniformly. The first assumption we make is that attention is perfectly uniform (Assumption from Section A. that a₂ = + or a, O. Given that the gradients of these parameters are the same for t < T B and t > 7 - B, cf. the calculation in the previous section, all attention values will have two possible values. If additionally leverage the fact that E, 1. everything is captured by single scalar Aa, that is defined as or Indeed
exp(a:)
01 Σ, explay)
(T exxx((1 -a)) + Bexp(ar at)
so that, for t T B,
04 = (T- Bexxpepppppp + B
(9)
23

---

and fort ST B.
(1 Bag)
ot
(10)
(T B)
Weights are initialized at O. We assume that W = 0 at initialization, following Assumption 4 of Section A.I. As En .jId, we have
VwL= E Ot Σ a. - w²) Σ
ai
W.
1>T-B
1ST-B
IST-B
This implies that whenever is aligned with W*, which is the case when W = 0, it remains aligned for the rest of learning. Yet, we not entirely done as the precise parametrization important to ensure that the dynamics match. Such a property is achieved when = wW- //| as, under the gradient flow dynamics on W, we have
WW,W), w
||
(W*, W)
(W*, Ww*,WW). L)F
||*||
and under the gradient flow dynamics directly on w. we get
- ddW
(W*, VWLLF
dw VWL.
I
F
In the calculations above, we used )) to denote the dlement-wise dot product (i.e.. F) = Eij Ar, Biy). The corresponding norm is the Froebenius norm ||-|| Each column of the target weights W has norm 1. It follows that the Froebenius norm of W* satisfies ||W* 113 = d. This is Assumption from Section A.1.
Under these assumptions, studying the gradient flow dynamics on the original loss therefore reduce to study the gradient flow dynamics on the simplified loss
1
Ba)²
2
(Baww
Baw
-1) tr (W-W.T)
1' + d |
(II)
2
d(T - B)² |
1ST B
BBaw Va
(1- 2w?
(12)
2
d(T - B)
d
with the attention given to token T being equal to
1
a T B) exp(e
Note that we have dropped the T subscript for notational conciseness. We plot this reduced loss landscape in Figure as well as how the parameters evolve under the gradient flow dynamics. As d = a(1 Ba), the gradient of this loss with respect to w and Aa are
T. (1- Ba + Ba(Bow Vd)
(13)
d(T -B)
d
B(1 - Balw²
+ Bw (Bow - Vd)
VaLL = a(l - Ba)
(14)
d(T BB)
d
The entries of the Hessian are
L
(1- Ba)²
B²a²
+
d d(T - B)
d2L
2B(1 Ba w
B(2Baw Vil
a(1 Ba)
+
dw dAa
d(T - B)
dLL
1 2Ba
d(T -IT-BB) + B²
B2w²
VaaL a(1 Ba)
dAnz
a(1 - Ba)
d
24

---

9.0
0.03
8.5
Loss
8.0
7.5
10
6a
10
0.64
0.56
0.48
0.24
0.16
0.08
5
10
15
Aa
Figure 6: Loss landscape for the reduced model (7 256, 64, without any repetition). The white line corresponds the gradient flow on this loss, initialized in (0, 0) (white cross).
A.3.2 Approximate dynamics around initialization
Despite having reduced the dynamics to two dimensions, is still too complex to be analyzed mathematically. In order gain insight into the early behavior of the dynamics, linearize the dynamics around initialization and compute the time it will require to leave the initial loss plateau.
Linearized dynamics at initialization. Linearizing the dynamics at initialization corresponds to considering the gradient flow on the quadratic approximation to the loss:
Aa H(O,0) (
L(w, Aa) L(0,0)+ VZ(0.))
W +
W
Aa
Da
with VL(0, 0) the gradient of the loss and HO,,0) 0) the Hessian of the loss at initialization. Plugging in the values the different variable take at initialization gives
V L B
VdT
V ==
and
d
T B B
d
dT2
dT
dLL
T B B
dw dAa
T
Vd
L
0.
dAa²
Under Assumption 5 from A.1, T co, B stays negligible compared T, and d is large enough so that V is negligible in front of d. that T-1 becomes $. di becomes 1 and it is therefore
T7
25

---

100
attention takon
10-3
0.4975
T
0.4950
128
10
512
0.4925
2048
10-1
0
50k
100k
50k
100k
0
50k
100k
time
time
time
Figure 7: Comparison of the early dynamics (plain lines) with their corresponding linear approximation around linear conditions (dashed lines). d is here fixed to 512.
negligible in front of the cross-term second-order derivative. The gradient flow dynamics around initialization becomes
V ET
VdT
Aa + o(||w]] + [|aa]).
(15)
Aa
B
0
VIT
Figure provides visualization of how well these approximate dynamics match the original one.
The Hessian matrix has positive eigenvalue vdr and one negative one var This implies that the initial parameters are in the vicinity of an unstable fixed point and that the negative eigenvalue Hessian will dictate how fast we are escaping these initial conditions. Its eigenvalues are
B
t = =
Vdr
with cigenvectors
22 +1 (1)
0+
Let P be the change of basis matrix defined by these eigenvectors. This matrix P transforms the original coordinates (w, Aa) to the eigenbasis coordinates, (24, 2_). The dynamics in the new basis
2+
B 2dT
1
0 1
2+ z- )
2
dT
This yields
2+ = (exp(At) - 1)
V2
2_ v2
(1 -expp-XXp
and
sinh(At)
Aa = (At) -
Initial evolution the loss. of the quadratic approximation of the loss:
We can now derive a close-form equation for the temporal evolution
L(w, Aa) = 0.5 Aw AwAa + o [||||| ||
get that the loss is initially approximately equal to
L ~z
ssnnh(At) - ssin(AA)(coos((Ac. -1)
2
1
Asinh(At) (At)
2
1
(1 A Asinh(2Xt))
2
using the identity sinh(2) cosh(x) | sinh((2))
26

---

Time needed to escape the initialization plateau. We can now compute the time T. it will take for the (approximate) loss be equal (1 --) 2) its initial value. From this definition, we get that Tę satisfies
[1 ssinh(2Xe)) = 1-E
2
2
that is
VdT
eVVT
Tg
aresinh
2B
Given that we are in the regime of large and d, we finally get
VdT
2e
Tg 2B
log
B
by using the fact that arcsinh(x)- 1 log(z) as
A.3.3 Phase transition and late phase dynamics
Once we escape initial conditions and attention to the relevant token(s) starts increasing, the impact of the other tokens on the loss starts becoming negligible, the loss approximately becomes
1 2d Bow vd
= L
(16)
This loss features multiplicative interactions akin one-hidden-layer neural network, and therefore it comes as no surprise that the loss exhibits similar sharp phase transitions as the ones observed when learning these networks, cf. Saxe et [29] for depth analysis of these dynamics (one needs to linearize the mapping Aa + a to get the exact mapping to this set of results).
The learning dynamics exhibits interesting behavior after the phase transition: the projection of on W' starts decreasing, as seen on Figure for example. In the following, we will argue that w is close to equilibrium in that phase and that learning consists mainly in slowly pushing Aa to infinity and that the corresponding equilibria decrease. We will show this by investigating the structure of the Hessian when
equilibrium.
First, let us compute the value that w takes at equilibrium, which requires solving the equation V O. Using Equation 13, it gives
(1 Ba)2
(Ba)²
Ba Vd
d(T - B) d
that is
((1 (1-Box)² Bax2
(Ba)²
-1 Ba
(a)
d(T - B) d
Vd
We plot 8 as a function of a in Figure 8.left.
Let us now consider the case in which Ba converges to 1, that is Ba = 1 - with s - O. This gives
Bad
vd 1-8 o(e2).
(Ba)2Vd o(6))
W
We can then plug this value into the different components of the Hessian using the second derivatives we calculated for the simplified loss:
L =
2x
+ o
dw
d
d
d
Be
+ +
dw
AAa
Vd
d
B + B2
+ B2
dAa²
T B
27

---

20
0.04
150
15
Hessian eigenvalue
largest
512
0.02
100
2048
10
0.00
-0.02
-0.04
10
10-2
10"2
10
5k
10k
5k
10k
time
time
Figure 8: (left) w function of for different T values (d 64, B = 1). (middle) Evolution of w on the gradient flow dynamics (T' 512, = 64, B = 1) and comparison with its instantaneous equilibrium value w² (right) Evolution of the smallest and largest eigenvalue of the Hessian of the loss along the gradient flow dynamics.
The Hessian is thus equal
-d Va
Hww Da) =
+ o(=2)
Be
B+B²
+ B30
vd
T-B
when = is at optimality and Ba close to 1 (i.e. Aa + oc). As goes to O, the loss becomes increasingly flat the Aa dimension, highlighting that aa will be the bottleneck in terms of learning and w will always be at its equilibrium values o We confirm this in simulation in the middle panel of Figure 8.
is now worth comparing the loss landscape structure in this late phase compared to the one early on during learning. In the early stages, the eigenvalues are small (B VdT) and w and A both contribute them. At the end of learning, the loss is much sharper in the direction (1 (1/d) than in the Aa direction dimension (s), because of the softmax within the attention. From this analysis of the two extremes of the learning dynamics, in discrete time, w would be the bottleneck terms of the learning rate, due to the final sharpness of the loss. A complete picture is hard to obtain analytically because calculations become more involved in the middle of learning, so we resort to simulations and plot the results in Figure 8.right. We find that the behavior we described analytically holds for reasonably long in the early and late dynamics. In the middle of the dynamics, the geometry of the loss landscape changes drastically and the loss is the sharpest at this time (and thus the maximum learning rate we can take in discrete time will be dependent on geometry the loss around the phase transition).
A.3.4 Empirical validation
In these sections, our aim is to verify whether our theory still captures the behavior of the model when the assumptions made not met. In particular, will relax Assumption 3 (W* has unit norm columns), (W initialized to 0), 5 (long sequences) and partially Assumption 1 (gradient flow dynamics). Indeed, sample W and W from (0, 1/d), take T to be 256 (thus finite), d to be 256 and consider stochastic gradient descent dynamics with batch size 32 and learning rate 1. We report the comparison of the evolution of the loss, the attention and the projection w of the weights W on W* in Figure for both the reduced dynamics (Theory line, as Equations 13 and 14) and for simulations (with the parameters described above). Overall, we find that the simplified dynamics is able to depict the ones of the actual model quite accurately.
the relevant to the T-th token,
A.4 Analysis of the dynamics with cross-sample repetition
When considering cross cross-sample sample repetition. Σr will have larger magnitude in the direction of the token 7 that appears more frequently during learning. As result, weights learn faster that direction and having a single scalar to capture the behavior of the entire weight matrix is no longer possible. In the following, we show how to reduce it to two scalars using similar techniques as before, as well as proceed to the analysis.
28

---

10a
Attention token 7
10a
Projection of W on W
Loss
10-4
10-1
10
10-6
10
10-2
Theory Simulation
0k
5k
10k 0k
10k
5k
10k
Step
Step
Step
Figure 9: The gradient flow dynamics on the simplified loss of Equation 12 match the stochastic gradient descent dynamics of the original objective. We report the dynamics of 5 different seeds for the simulation lines. Changing the seed modified the target mapping, the sampling process, as well as the model initialization. Additional details are provided in Section A.3.4.
6
10"
10-1
test loss
10-2
10-1
0.2
10-3
0.5
2
10-2
10-4
10-5
10-1
10k
20k
30k
10k
20k
30k
10k
20k
30k
time
time
time
Figure 10: Cross-sample repetition speeds up emergence. This is because the repeated component (dashed lines in the middle plot) learns faster, which leads to an earlier increase of attention a to the relevant token (right) and faster learning overall (left). The results reported here were obtained by simulating the gradient flow on the loss of Equation for 512 and d 64.
A.4.1 Reducing the learning dynamics to three dimensions
Cross-sample repetition does not affect attention directly, so we can still capture there entire behavior of attention with
1
a (T' exp( Aa) +1
Note that do not consider any in-context repetition here for simplicity, but all our results can be extended to this case. Let us now look at the weights. Without loss of generality, we can assume 01......, which yields
1 1-
1- d
1
1-p
Σr = diag p+
The gradient of the loss with respect the i-th column W., of the weight matrix therefore becomes
V L Σ an (Min 1 1PP) BBT-B
Σ
W.
W
+ Σ IST
W..
d
VST
Importantly, when initialized at O, each column of will evolve on the line spanned by the corresponding column of W* However, they will not move at the same speed the first column has different dynamics than the other columns. We therefore introduce two scalars i and w such that
W., = wi and W₂ = vύ=i W;- As for the analysis in the previous section, the vd scaling factor is here to ensure that the learning speed coincide. To keep the notation as concise as possible, we introduce == + _ and σ = l.p.
Under these assumptions, the loss simplifies to
1 (1 al=a)?(w² + w2) + (arδ 1)² + aw Vd -V
?
(17)
2 d(T 11)
29

---

The gradients flow dynamics on this loss are reported on Figure 10. The gradients are
VuL = (1- 11-a) a)
+ aá 1)
(18)
d(T - 1)
V V (1 1-aa)2
a(1 - ((1-p)(aw- -Vd-1
(19)
d(T 1)
d
(-11
(1 (1- a)( + :
VooL = a(1 a)
-wo 1)
(20)
d(T' -1)
+ wow vd -1)
(21)
The entries of the Hessian
d (1-a)
+ a²2
a d(T 1)
d² - (1-a)
+ 2
dw
d(T
LL
dw
LL
(1 (1- a)w
a(1 a)
+ (2aw 1)
dw dAa
-1)
d
( ([11- -/)
a(1 a)
(2aw vd-1) 0
dw dAa
d(T 1)
d 1 20
(w² + +
V Da L a(1 aa)
+ (ow' + ow?)
dAa²
a(1 - a))
-1)
A.4.2 Approximate dynamics around initialization
Around initialization we get
T-1
a(T-1)
d
i
dT
T
i
w
cyd-1
+
0
T
oT
Diva-I
W
di
Aa
a(T-1)
Aa
T
Under Assumption 5, this dynamics becomes
frr
(
w
dt
d
Aa
+
0
P
1-p
0
VaT
Aa
w
T
ver
The characteristic polynomial of the Hessian is
(1
p)²
p²
33- x
dT
T
which means that the eigenvalues of the Hessian are 0 and which provides scaling factor for the exit time of
pdd (1-p)²
VdT
ddT
pd 21
As sanity check, we can remark that when there is no cross-sample repetition, we recover the same scaling factor as our previous analysis. As p increases, progressively removes the effect of the dimension d in the escape time.
30

---

101
10"
32
loss
10-1
16
10
M
10-4
7.5k
step
Figure 11: The learning dynamics of Transformers exhibit sharp phase transitions in the single- location task. Here, T 128 and the architecture and training details the one described in Appendix D.3.
d 32
64
d 128
10
10
10
test loss (ID)
10
109
10P
10-1
10-1
10-1
10-2
10-2
10-2
10
10
10
10-4
10-4
0k
5k
10k
15k
20k
10k
15k
20k
0k
5k
10k
15k
20k
step
step
step
107
10
10
test less (OOD)
10P
10a
10a
10:1
10:
10 1
10:2
10-2
10:2
0.0
0.2
10-1
10-1
0.1
0.4
10-4
10-4
10-4
0k
5k
10k
15k
20k
10k
15k
20k
0k
5k 10k
15k
20k
step
step
step
Figure 12: Effects cross-sample repetition on the learning dynamics of Transformer in the linear regression task. The top line corresponds the test loss on the same data distribution than the one on which the network is trained, that is with repetition. The bottom line is data without any repetition. The results are obtained for T 256. See Appendix D.3 for the rest of the training details.
B Details and additional analysis for the linear regression task
B.1 Implementation the task
For experiments, implement slightly different version of the task than the introduced in the main text and that we used for our theoretical analysis. The difference lies in where the relevant token and whether there exists a feature indicate it:
Random relevant token position. In the simple version of the task, the relevant token always presented at position for convenience, as the output of the toy attention model we consider does not depend the specific position. For experiments with more realistic Transformer models, make this position random to avoid skip connections playing a specific role, and we resample for every new sequence. The only exception this is the task specifies ablation of Figure 14, in which we sometimes fixed throughout the entire training run (the relevant position being still randomly sampled the beginning of training).
Relevant token feature. As the position of the relevant token is resampled for every new sequence, we need to provide information to the network about which token is relevant for the
31

---

task. To this end, we append an extra feature to each input token that is 1 whenever the token is relevant. As in the previous point, the only experiment which this feature not added the input is the ablation of Figure 14.
B.2 Examples of learning curves
Figures 11 and 12 provide examples of the learning dynamics of Transformers on the variation of the single-location linear regression task described in the previous section.
B.3
Additional ablations on the impact of task variation, model size and optimizer on plateau length
In the main text, we claim that the architecture, the details of the task, as well as the optimizer change the dependence of the plateau length on the different data parameters (here we study d and T). These changes collectively explain why the theoretical exponents of the power laws we obtained in our simplified regime do not directly translate practice. This section provides the ablation underlying this claim; we detail them by increasing importance.
Architecture.
The toy model we consider has single head and a single layer whereas the Transformer architecture we consider has 2 layers and 4 heads. In Figure 13, we ablate the number of layers and number of heads. We find that they have some importance, by slowing the learning process (note that we did not tune the learning rate to each architecture size). and T captured through the power However, they do not significantly alter the dependency law exponents.
Task specifics. In the more realistic version of the task, the Transformer is provided with an extra input feature which indicates whether the token relevant for the task, and the relevant token position is random. However, the toy model does not have access to this feature (it cannot use it) and the position is fixed. In Figure 14, we ablate these different choices and find that randomizing the position significantly increase the dependence in T and, to smaller extent, the dependence on d. Interestingly. the power laws are almost the same for fixed position when the relevant feature is given to the model or not. We argue that this may be because the relevant feature provides weaker statistical signal, likely because of the initial embedding layer from dimension d to 256, than the positional encoding. Learning to use would therefore be slower, and the network eventually ignores this feature.
Optimizer. Our theoretical were obtained by analyzing the gradient flow dynamics. While gradient flow has tight links with gradient descent, and thus the emergence time under the gradient flow is approximately proportional the number of steps before emergence (assuming small enough learning rates), this does not hold for Adam. To test how much Adam changes emergence time, we ablate the choice of the optimizer in Figure 15. We find that Adam to be significantly faster than SGD (up to almost two order of magnitudes for the hard versions of the task), both in absolute terms and sensitivity to the difficulty of the task.
32

---

layer, head
layer, heads
1 layer, heads
plateau length
plateau langth
plateau length
10a
102
103
32
64
128
10a
10a
10a
2 layers, head
2 layers, 2 heads
layers, 4 heads
10
10
10a
plateau length
plateau length
plateau length
10a
10a
10a
10a
102
10a
16
32
64
128
16
32
64
128
16
32
64
128
d
4 layers, head
layers, heads
layers, heads
plateau length
plateau lengeh
plateau length
10a
10a
10a
10a
10a
10a
Figure 13: Changing the width of the network (by increasing the number of heads) increases time spent on the initial plateau more significantly than increasing the depth of the network. We plot the evolution of the plateau length, for Transformers with different number of layers and number of heads. We obtain the following scaling law: T Tplatean 1.03 . 7 N70.06 16 layers heads
(R² 0.995), which corresponds to the lines in the plots above.
33

---

relevant token feature random relevant token position
relevant token feature fixed relevant token position
relevant token feature fixed relevant token position
plateau length
platnau length
10*
plateau length
104
101
10a
10a
T
T
32
64
128
32
64
128
32
64
128
10a
10a
10a
16
64 128
16
32
64
128
16
64 128
Figure 14: Emergence is faster when positional information is available than when the network has use semantic information. this set of experiments. we vary the nature of the task. The network can be given a "relevant token feature" indicating whether the token is relevant for the task is given (that is semantic information about the nature of the token). The position of the relevant token for the task can be either fixed or random. If it is random, the network must use semantic information solve the task. If not, it (may) use positional information. In particular, this means that the network cannot solve the task when it has no access to the relevant token feature and the positions random; this is why we do not report results in this configuration here. The different scaling laws plotted are:
(leff) Tplateau 1.43 d 70.58 R² 0.998 (relevant feature, random position).
(middle) Tplatean = 4.20 d2.22 20.299 R² 0.996 (relevant feature, fixed position).
(right) plattauu = 4.61 d 20.30 R² 0.999 (no relevant feature, fixed position).
SGD
Adam
10f
10a
plateau length
105
length
20a
104
101
10a
10a
32
64
128
256
64 128
10a
128
Figure 15: Switching SGD for Adam leads to important acceleration of learning. Here, train a single head single layer transformer with (left) SGD with a learning rate of 5- 10 and (right) Adam with learning rate of 10-1 (these learning rates manually tuned). Changing the optimizer has a huge effect on emergence time, both in terms absolute time but also of scaling as the problem gets harder: Tplateau 6.42 /1 71.07 (R² 0.997) for SGD and T plaateau 1.45 d 70.57 for
Adam (R²² 0.998).
34

---

Details and additional experiments for the in-context associative recall task
C.1
Implementation of the task
Recall that each sequence consists of Naaiis key-value token pairs (5 such pairs in the example below) followed by query token (Z below), as shown in the following example:
AX UR ZY
CA
2?
The number of possible tokens is the vocabulary size Ntokens and each of these tokens has corresponding one-hot encoding. Both keys and values are sampled from the same pool of tokens.
The generative process behind the task is as follows:
Set up query distribution. We define a query distribution Pquery which corresponds to uniformly sampling one of the 2-repeated tokens with probability and uniformly sampling all the tokens with probability 1 p (recall that p the sross- repetition probability). Said otherwise, this probability distribution has mass 2 rest of the tokens.
1-p_ the repeated tokens and mass 1-2_ on the
Neokens
Niokens
Sample the in-context mapping. For each new sequence, sample the mapping between keys and queries. This mapping is injective, meaning that two different keys will always be associated with different values. In practice, we sample random permutation for this mapping.
Sample the query. For each new sequence, we sample the query following the probability distribution Pquery defined when creating the task. The target output for the task, will be the value indicated by the in-context mapping.
Fill the context. For each new sequence, we finally fill the context as follows. We first decide whether we put the query (and its corresponding value) in the Npairs possible positions by sampling Bernoulli variable with success probability N (on expectation the query thus appears B times in the context). We then fill the rest of the context by sampling keys from the Niokens 1 remaining tokens without replacement, and appending their corresponding values directly after.
C.2 Learning dynamics and additional analysis
We here provide two additional results: the learning dynamics on the training data, which show that repetition accelerates emergence in similar ways what is predicted in our theory (Figure 16) and the evolution of the attention scores over the course of learning, which confirms that emergence occurs jointly with attention to relevant tokens getting sparser (Figure 17). We can make additional comments regarding the last point: it is interesting to see that all heads in the same layer have similar attention relevant tokens, and that the copying mechanism (to group together the key and the value) only occurs in layer and the selection mechanism only in layer 4.
In-context repetition
Cross-sample repetition
1.00
1.00
train accuracy
0.75
0.75
train loss
0.50
train
0.00
50k
100k
150k
50k 0
100k
150k
50k
100k
150k
50k
100k
150k
steps
steps
steps
Figure 16: Repetition accelerates emergence in the associative recall task per the theory. This plot is the training counterpart of the plots of Figure 5. As we measure the performance on the training data, there no overfitting and repetition only speeds up emergence.
35

---

layer
layer
accuracy
0.75
0.50
0.50
0.50
0.25
0.25
attention
0.25
0.00
0.00
0.00
100k
200k
100k
200k
100k
200k
steps
steps
steps
Figure 17: The emergence of associative recall co-occurs with an increase of attention to relevant tokens. (left) Evolution of the performance of the model over time, in Figure 4 left (dVpairs 32 and Nrokens 256). There is no repetition in this experiment. (middle) Evolution of the attention from the relevant value token to the relevant key token (the first layer of two-layers manually constructed attention head would have this attention value equal to 1). (right) Evolution of the attention from the query token to the relevant value token (the second layer of the induction head would get this value equal to l).
36

---

D Methods
D.1 Measurement of the plateau length
To quantify the emergence time of sparse attention, we measure the duration of the initial plateau in the learning dynamics. We define the plateau length the number of training steps required for the model to reach a specific threshold performance from initialization. This performance threshold depends on the task consider:
For the linear regression task with the toy model, we define the plateau length as the time required for the test loss to decrease to 0.2 of its initial value when following the gradient flow dynamics, that is 0.1. The test loss is the loss without any form of repetition for the vanilla dynamics (Figure La) or for the dynamics with B value as during training for in-context repetition. Note that we use test loss with repetition for the latter the toy model cannot generalize to the no repetition case.
sample repetition (Figure 2 right), and with the same
For the linear regression task learned with a Transformer, except for the cross-sample repetition where we used a threshold of 0.4. This threshold being different partially explains why the lighter lines in Figure 3 middle and right do not map (the other reason being that we additionally tuned the learning rate for the right plot). The reason behind this change comes from the shape of the learning curves: after emergence, the learning speed can be drastically different for different values (see Figure 12, and hence we need to be closer to 0.5, the initial loss value, accurately capture the emergence time.
the same threshold as above,
For the in-context associative recall task, we pick accuracy threshold, here 5%, as the random guess loss varies as a function of the vocabulary size.
It is important to note that in the first case, the plateau length corresponds to a physical time, whereas for the last two cases it corresponds to a number of optimization steps.
D.2 Fitting power laws on plateau length
To quantify the relationship between plateau length and various factors (sequence length T, input dimension d, in-context repetition B, cross-sample repetition probability p). we fit power laws to our empirical measurements using least squares regression on log- taggrranfformed data. We also fit power laws the number of heads and the number of layers in 13 according to similar principles what detailed below.
For the general form of our scaling laws, assume:
T B
8 f(p dy
T Tplateau- = dd
(22)
where C' is a constant, a, B, and are the scaling exponents, and = v p) is function of the repetition probability (which is rooted in our theoretical analysis of Appendix A.4. Note that when considering the associative recall task, the vocabulary size is replacing d and the number of pairs replacing T.
We perform the fitting by linearizing this relationship through a logarithmic transformation:
T
log T plateau log C' + a log + log
+ 7og f(p d)
(23)
B
and performing linear regression find the missing coefficients. We report the score (R²) of the linear fit as a measure of fit quality for all scaling laws. Most reported scaling laws had R² > 0.98, indicating excellent fit to the empirical data. For the cross-sample repetition results of Figure right, where we could not fit accurately any power law on the obtained results.
More precisely, the exact parameters fit for each plot are:
Figure I.c: C, a and / (B is fixed to 1).
Figure left: C, and 8 (I' is fixed to 4096).
Figure right: C, a, B and y, such that 2a 3 = (I'ss fixed to 4096).
37

---

Figure left and middle: C, and B. The same power law is fitted on the data of the two plots. Figure right: C, a and B (B is fixed to 1).
D.3 Architectural and training details
We report the default hyperparameters use in experiments in Table and report the deviations we make to this setup below:
In Figure right, additionally tune the learning rate (in (10-1,3 13-11))) based on the shortest average plateau length as we found cross-sample repetition to significantly change the learning rate the Transformers considered need.
In Figure 4 right, additionally tune the learning rate (in {10-5,3 1,-5, 10---000-11 1)= large vocabulary sizes and number of pairs need smaller learning rates.
In Figure 13, we ablate the number of layers and the number of heads in the Transformer architecture.
-In Figure 13, we ablate the choice of Adam as optimizer and replace it by stochastic gradient descent. Note that here consider a single layer and single head.
It should be additionally noted that we adapt the number of iterations depending on the difficulty of the task and roughly aim end training significantly after emergence. However, for the most challenging versions of the task (e.g.. large T values in Figure 3), the phase transition sometimes occurs after the maximum number of training steps we attribute the task (in that case 50k steps).
Parameter Type
Theory
Linear regression
Associative recall
Architecture Parameters
Model
Toy model
Transformer
Transformer
Layers
2
4
Number of heads
N/A
4
4
Embedding dimension
N/A
256
256
QK dimension
N/A
64
64
MLP factor
N/A
4
Positional encoding
N/A
sinusoidal
sinusoidal
Layer normalization
No
No
No
Skip connections
No
Yes
Yes
MIP between layers
No
Yes
Yes
Training Parameters
Optimizer
GD
Adam
Adam
Learning rate
1.0
10
10-
Scheduler
None
None
None
Batch size
Full
32
32
Seeds
N/A
5
3
Table Default hyperparameters for the different types of experiments.
D.4 Reproducibility
Our theoretical simulations Torch [69]. All our experiments were run on Nvidia RTX 3090 GPUs. The typical train- ing run takes one hour, although training on shorter sequences can be significantly shorter. The code is publicly available at the following url: /ttps://github /gggtttt..b.... com//Vicclllsszcccchtt// asparse-attention.
in JAX [68] and our Transformer experiments in Py-
The- the-emerrgene--
38


Reviews:
[{"review_id": "f8gyhGKbfM", "reviewer": "Reviewer_BBS6", "summary": "This paper investigates the emergence of “sparse attention” using a toy setup of training a small Transformer on a synthetic linear‐regression task. They show that the number of training steps to emergence scales with model size and sequence length, matching their theoretical derivation. They also find that repeating examples in the training data accelerates the emergence. Finally, they validate these findings on a more realistic associative‐recall task, confirming that sparse‐attention emergence governs performance there as well.", "questions": "Q: Do you observe any difference based on the position of the relevant token in the linear‐regression tasks? Would choosing later tokens help induce sparse attention more quickly?\n\nQ: It seems intuitive that repeating information in the context aids retrieval. However, that doesn’t necessarily require sparse attention. I’m confused about how repetition causally drives attention to become sparse—how are these two phenomena related?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. The paper provides a detailed theoretical analysis of sparse‐attention emergence in a toy linear‐regression setup and corresponding simplified Transformer model. It then extends this analysis to a more realistic associative‐recall task, showing that key observations generalize.\n\n2. The authors skillfully bridge mechanistic interpretability and theory, moving from gradient‐based proofs to empirical validation.\n\n3. This work offers a novel proof of concept for how mechanistic insights can inform realistic training‐data selection.\n\n4. The experimental setup is clear and reproducible: the authors thoroughly explore different model sizes, sequence lengths, and optimization methods.\n\nWeaknesses\n\n1. Although the paper shows that data repetition accelerates sparse‐attention emergence, it only briefly discusses implications for tasks requiring diverse data. A deeper analysis of which tasks benefit most from repetition (e.g., those relying heavily on in‐context information) would strengthen the discussion.\n\n2. Some observations—such as the proposed power‐law scaling—do not hold consistently across the linear‐regression and associative‐recall setups. Since both tasks remain somewhat artificial, it is unclear how these insights will generalize to real‐world applications.", "was_revised": true, "final_justification": "I remain my positive evaluation of this work.", "rebuttal": "We thank the reviewer for their positive feedback and thoughtful review. We address the weaknesses they raised and the questions they asked below:\n1. **Weakness: no discussion of which task could benefit from these effects.** We thank the reviewer for pointing out this blind spot in our argumentation. We believe **our findings to be general and not limited to tasks that rely on heavy in-context manipulations**. We believe that **some sparse context selection is needed** in order to see the emerging behavior we characterize, which might be relatively common given the sparsity of trained LLMs.\nWe expect **cross-sample repetition to be particularly helpful in tasks in which learning the feedforward mapping is hard** (high $d$ in our analysis, which this form of repetition effectively reduces). This includes learning facts – the findings of Zucchet et al. 2025 are one example of the benefits of cross-sample repetition – but may extend to some other tasks like reasoning-heavy tasks. \n**In-context repetition mostly when learning from very long sequences** (as they decrease the effective sequence length). It is a natural feature of language (it helps humans to process texts), and we expect that reducing it would actually considerably slow down the learning of language models.\nWe have included more detail about these considerations in the discussion and we hope that our work will motivate future studies to investigate these questions.\n2. **Weakness: theoretical power laws don’t exactly extend to other tasks.** Indeed, each task seems to have its own power law and its specifics are task-, architecture- and optimizer- dependent (our ablations of Appendix B.3 shows quite strong power law variations already on the single-location linear regression task only). We believe that the fact that there exists such a power law for more sophisticated tasks (see also Figure 2 right of Zucchet et al. 2025) is a good sign that **the core phenomenon, though not the exact coefficients, is robust and may generalize to real-world applications**.\n3. **Question: does the position of the selected token affect learning speed?** **We have no reasons to believe that it has a strong impact.** When looking at Figure 11, which shows learning dynamics for different seeds (and thus different locations), all of them have similar dynamics.\n4. **Question: role of in-context repetition and does repetition causally drive sparse attention?** The reviewer is right that the effect of in-context repetition is rather intuitive: if the same information is repeated multiple times in the context, it becomes more correlated with the token to predict and thus learning is faster. Our theory captures that analytically. **Repetition affects the learning of sparse attention** (as per the argument above), but **it does not causally drive sparse attention** in the sense that sparse attention would arise even without repetition (because it is useful to solve the task).\n\nWe hope this clarifies our results, and we remain available during the rebuttal period for additional clarifications."}, {"review_id": "XLRUxc7YZc", "reviewer": "Reviewer_AeaF", "summary": "This paper studies training time emergence (sudden jump in performance during training after an extended plateau) for a simplified in-context learning (ICL) task involving single-location linear regression. The authors simplify the problem to coupled dynamics of 2 scalar variables corresponding to learning the attention map and linear projection. They further study the effect of repetition (both in-context and cross-sample) to analyze how this affects emergence during training. Results in the toy model setup are verified qualitatively in Transformer training on an in-context recall task.", "questions": "1. I couldn't follow why “in-context repetition makes the attention pattern to be learned less sparse, thereby simplifying the task” (line 138) - could the authors clarify how making the pattern less sparse simplifies the task? (I understood how having in-context repetition \"allocates B times more weight to the relevant tokens\" in Line 143, but not how that simplifies the task itself)\n\n2. Just to clarify, is Figure 1 (right) and 2 numerical simulation for the toy model? If so, is the takeaway from this plot that linearizing early-time dynamics is OK for estimating plateau length?\n\n3. Line 117-118: “in these two equations, the first term is the relevant signal and the second term is the noise coming from non-relevant tokens.” - I couldn't follow this argument, could the authors clarify?\n\n4. Line 189-190: “Removing layer normalization, which made solving the task more challenging” - is there a reason / hypothesis why this is true for this specific setup? \n\n5. In-context associative recall task: Does induction head formulation map in any way to the toy model discussed in Eq (2)?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- Paper studies relevant problem of training-time emergence connecting it to attention map sparsity, and concretely shows how the two are related.\n- Toy setup is well-formulated and somewhat represents realistic Transformer setups.\n- Paper is overall well written and easy to follow.\n\nWeaknesses\n\n- Not a major weakness, but the theory setup of single-location in-context regression might be a bit too simplistic for many tasks (even the experiments considered in the paper). While it does seem to predict some emergent phenomena quite well, overall I am not sure if just a linear layer with a softmax-vector as Attention layer would be able to capture a lot of nuances in realistic setups.", "was_revised": true, "final_justification": "The authors have responded to my (clarifying) questions, and I've maintained my score of 5 for this submission.", "rebuttal": "We appreciate the reviewer’s positive feedback on our work. We agree with the weakness they raised (see our answer to reviewer oc9h for nuance on it), and answer their questions below:\n1. **Question: why does in-context repetition make the task easier?** With in-context repetition, the information relevant for the prediction is repeated multiple times within the context, and is thus **more correlated with the token to predict**. This provides a stronger statistical signal to the model, and thus speeds up learning. An alternative way of seeing that is that in-context repetition **reduces the effective sequence length** (it’s mathematically similar to having a sequence of length $T/B$ with only one relevant token). Given that learning time scales with sequence length, this effectively makes the task easier and speeds up learning.\n2. **Clarification: takeaway from Figure 1 right and 2.** Figure 1 right is the simulation of the simplified dynamics (Eq 3 and 4), which comes at the price of a few assumptions and on which we derive the theoretical exit time. Figure 2 consists of the learning dynamics of the model of Equation 2 (without the assumptions). The takeaway from these figures is indeed that linearizing the early dynamics of the simplified model captures the plateau length in the model of Eq. 2.\n3. **Clarification: signal and noise terms in equations 3 and 4.** At a high level, there are **two types of tokens** in this task, the relevant ones (**signal**) and the non-relevant ones (**noise**). The derivation of Equations 3 and 4, which can be found in Appendix A.2 and A.3, uses this insight to group the contributions to the loss of the tokens together, and Eq. 3 and 4 reflect this split.\n4. **Question: hypothesis for why layer normalization makes the task harder?** We have not thoroughly investigated this question as this is likely specific to the single-location regression task (we use layer norms in the in-context learning experiments). Our main hypothesis is that it is difficult to always output 0s at a specific location with layer norm, but this is possible without. As a consequence, the layers have to implement the selection mechanism together, a cooperation that may take longer to learn.\n5. **Question: how does the induction head map to equation 2?** We would first like to clarify that equation 2 is the equation of our toy model and that **it can not directly implement an induction head** (it is not a sequence to sequence layer, it cannot use relative position information which is needed for the copying mechanism, it has no semantic similarity needed for the moving mechanism). That said, the toy model of equation 2 is relevant in the sense that it can be used to **capture some of the dynamics underlying the learning of sparse attention and therefore of the induction head**. This is because the attention needed for the copying mechanism of the induction head is sparse (attention on the last token), and the one for the moving mechanism is sparse (attends to the same query token in the context). \n\nWe hope this answers the questions of the reviewer and we remain available for additional clarification."}, {"review_id": "PPQ66ZfCWs", "reviewer": "Reviewer_qdKB", "summary": "The paper considers the emergence of sparse attention. They adopt three settings: single-location linear regression, in-context repetition, and cross-sample repetition. In the simplified model, they discover a power law of the plateau length corresponds to the sequence length T and hidden dimension d. They build a simplified theoretical framework to explain the dynamics of attention weight a and value matrix W, matching the observation. They further use the theory to explain the benefits of in-context repetition and cross-sample repetition. They also extend the results to realistic transformers and learning problems.", "questions": "1. What is the motivation of removing layer norm in the 2-layer 4-heads Transformer? \n\n2. Just curious, have you observed the linear growth of the norm of $W$ (the value matrix in attention layer) along the training dynamics in 2-layer 4-heads TF if having layer norm and using Adam?", "limitations": "See my previous comment.", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Although the plateau is repeatedly discovered in in-context learning literature, the plateau length is rarely discussed. The authors make very interesting findings on the power law of the plateau length. They also make novel simplification of the training dynamics and provide theoretical insights on the observed power law. The insights from analyzing in-context repetition and cross-sample repetition are also interesting.\n\nFor the weakness, my major concern is the limited implication of the benefits of in-context repetition and cross-sample repetition. I understand that these are interesting findings. But they are restricted in the given in-context learning setup. \n\n1. The more interesting direction of in-context repetition (bustiness) is its relation to in-weight learning. It is less surprising that more in-context repetition leads to shorter plateau length. \n2. I would imagine that the cross-sample repetition is only useful given the special design of the in-context learning task. The benefits of data repetition in training, or the longer training (grokking), seem to come from reasoning mechanism (for example, see Nanda 2023), but barely relates to in-context learning. The coding example given in the last paragraph of the paper seems to be more related to reasoning rather than in-context learning. I also find it hard to relate the discoveries in the paper to active learning. In general, I feel the discussion in section 4 should be more carefully phrased.\n\nNanda, Neel, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2023. “Progress Measures for Grokking via Mechanistic Interpretability.” ArXiv.org. 2023. https://arxiv.org/abs/2301.05217.\n\n‌", "was_revised": true, "final_justification": "The paper is both theoretically solid and interesting in practice. I do not see any major issue. Therefore, I decide to give strong accept.", "rebuttal": "We thank the reviewer for their appraisal of our paper! \n\nWe agree with the reviewer that the role of in-context repetition is rather intuitive. \nPlease see our answer to BBS6 regarding the second weakness the reviewer mentions (specificity of our results to the in-context learning task). Given the generality of the mechanism we highlight, we expect cross-sample repetition to be useful beyond the in-context learning task we study; Zucchet et al. 2025 provide one such example. \n\nThe link with active learning is as follows: we show that varying diversity might be a powerful way to speed up emergence, but can be damaging for generalization. This insight provides a **simple active learning algorithm**: whenever an agent sees that it is **stuck on a task**, it **decreases data diversity** (which it can, as an active learner), and it increases it afterwards when learning finally kicks off.\n\nWe have made these points clearer in the current version of the manuscript.\n\nRegarding the questions asked by the reviewer:\n1. **Question: why remove the layer norm?** Note that we only remove it for the single-location linear regression experiments, and keep it for the in-context learning experiments. Our main hypothesis is that it is difficult to always output 0s at specific locations with layer norm, but this is possible without. As a consequence, the layers have to implement the selection mechanism together, a cooperation that may take longer to learn.\n2. **Question: linear growth of the value weights?** We haven’t experimented with this and do not have a good argument for why this would happen, as it would not help to improve the selection mechanism nor the required linear transformation of token embeddings. That said, we are happy to run that experiment if the reviewer has a hypothesis for why this would occur."}, {"review_id": "a33QmEwTSI", "reviewer": "Reviewer_oc9H", "summary": "The paper investigates the emergence of sparse attention patterns in Transformer models during training, focusing on how data distribution and repetition influence this process. The authors combine theoretical analysis with empirical experiments to uncover the mechanisms behind sparse attention emergence and its impact on model performance.", "questions": "Please see Weaknesses. My primary focus lies in points 1 through 5.", "limitations": "yes, some Limitations are discussed in Sec. Discussion.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "> **Strengths:**\n>\n> (1) Good visualizations.\n>\n> (2) They explore sparse attention as a lens to understand emergence during training.\n\n> **Weaknesses:**\n>\n> (1) Theoretical analysis primarily focuses on the single-location linear regression task, which is insufficient for analyzing the training dynamics of models.\n>\n> (2) On line 87, it states 'the relevant token location is always the same.' This will lead to limited diversity in training tasks, creating a gap with large language models. Additionally, it will also restrict the scope of applicability for theoretical analysis.\n>\n> (3) Line 104 \"The model’s parameters (a, W) are learned by minimizing the expected mean square error between predictions y and targets y*.\"  Lines 142-144 \"attention layer allocates B times more weight to the relevant tokens compared to the case without any repetition\".  So, in both scenarios, does the pair (w, a) end up learning the same thing in the end? Does \"B times\" imply that the value of 'a' can be different?\n>\n> (4) The authors need to more clearly present the corresponding evidence to support the claim made on Lines 185-187.\n>\n> (5) Lines 278-288, what are the differences and connections between \"grokking\" and \"emergence\"?", "was_revised": true, "final_justification": "The authors theoretically and experimentally elucidate the advantages of in-context repetition and cross-sample repetition, which provides guidance for practical applications of large language models, such as training data allocation. However, a limitation lies in the need for further examination of the gap between simplified models and real-world applications.", "rebuttal": "We thank the reviewer for their feedback and for clearly expressing their concerns about our paper. We provide some clarification below, which we believe address all the concerns raised by the reviewer:\n1. **Weakness: The single-location task is insufficient to analyze the training dynamics of real models.** The reviewer is right that our analysis (in particular the theoretical one) does not fully capture all aspects of the learning dynamics of large language models, trained on more sophisticated tasks. However, we would like to nuance this point. Our analysis was designed with minimality as a main goal: we wanted to find a **minimal setting** in which emergence occurs during training and in which we could see the benefits of repetition. This allows us to identify, for the first time, a simple mechanism (sparse attention) that is easier to understand intuitively, while being practically relevant as it is ubiquitous in trained Transformers, and has close links with induction heads in in-context learning. By design, we are not trying to quantitatively capture realistic learning dynamics, but rather get an **intuitive understanding of how different variables** (here sequence length, data dimension, or repetition) **affect learning**. We believe our analysis **provides insights about some non-trivial behaviors** that this simplification makes much easier to analyze and conceptually easier to grasp. And we hope that, in future work, these findings can be applied to wider settings to gain additional understanding.\n2. **Weakness: Fixing the location of the relevant token limits the applicability of the results.** The location is indeed fixed in all experiments provided in Section 2. The ablations in Figure 14 show that randomizing the token position (and providing a feature to the Transformer that allows it to select this position) has **some effect** on the exact scaling laws that the plateau length follows, **but the overall finding remains unchanged**.\n3. **Question: Values taken by $(w, a)$ depending on the B value?** $w$ always takes the same final value ($1$) whatever the B value. When there is in-context repetition, the attention is equally shared between all relevant tokens, so we have the identity $\\alpha B = 1$ at the end of training with $\\alpha$ being the attention given to one relevant token. This implies that the attention to each of the relevant tokens decreases as $B$ increases. In terms of $a$, the difference between the $a$ for relevant tokens and the $a$ for non-relevant tokens goes to infinity, whatever the $B$ value. To summarize, **increasing $B$ does not change the final function learned by the network, but only convergence speed.**\n4. **Question: evidence for the claims of l.185-187?** We thank the reviewer for pointing out this issue. These sentences **summarize the findings of the section**, and the rest of the section provides evidence for these claims. We have made that clearer in the current version of the manuscript.\n5. **Question: Differences and connections between grokking and emergence?** Emergence is a broad concept that goes beyond emergence during training (the type we study). **One can understand grokking as a specific form of emergence during training**, in which the generalization abilities of the model suddenly improve much later than the ability of the model to solve the training task. However, there are other forms of emergence during training, for example, there are cases in which emergence already occurs looking at the training loss only (we study one such example here).\n\nWe hope that this answers the concerns raised by the reviewer, and remain available for additional clarifications. Given these answers, we would appreciate it if the reviewer could reevaluate our work."}]

Author rebuttal:

Meta-review: After careful consideration of the paper and the reviewer comments, I recommend accepting this paper. The paper provides a novel, mechanistic account of how sparse attention patterns emerge during training, demonstrating through both a simplified theoretical model and empirical validation that this emergence follows power-law dynamics influenced by task structure, architecture, and optimizer choice, and is significantly accelerated by data repetition. While the theoretical setup (single-location linear regression) is indeed a simple model, it does the job of providing a minimal example of sparsification. The authors' thorough and effective responses, which satisfied or partially satisfied all reviewers, including the initially skeptical oc9H, will significantly strengthen the manuscript if applied, which I strongly recommend the authors to do.
Decision: Accept (oral)