Influence Dynamics and Stagewise Data Attribution
Authors: Jin Hwa Lee
Matthew J. A. Smith
Maxwell Adam
Jesse Hoogland
Venue: ICLR 2026 Poster (2026)
Keywords: Training data attribution
influence functions
singular learning theory
stagewise development
phase transitions
developmental interpretability
Bayesian influence functions
Primary area: interpretability and explainable AI
TL;DR: We demonstrate that neural network influence functions can change dramatically over training due to stagewise development, which challenges the static attribution paradigm and motivates a shift to stagewise data attribution.

Abstract: Current training data attribution (TDA) methods treat the influence one sample has on another as static, but neural networks learn in distinct stages that exhibit changing patterns of influence. In this work, we introduce a framework for stagewise data attribution grounded in singular learning theory. We predict that influence can change non-monotonically, including sign flips and sharp peaks at developmental transitions. We first validate these predictions analytically and empirically in a toy model, showing that dynamic shifts in influence directly map to the model's progressive learning of a semantic hierarchy. Finally, we demonstrate these phenomena at scale in language models, where token-level influence changes align with known developmental stages.

Published as a conference paper at ICLR 2026
INFLUENCE DYNAMICS AND STAGEWISE DATA ATTRI- BUTION
Jin Hwa Lee"
Matthew Smith Independent
Maxwell Adam University of Melbourne Timaeus max 1m eus. co
University College London jin. leelle.... 2222222lcc..... ac.
matt. ja smi th@gmm com
Jesse Hoogland Timaeus jesseêt imaeus.co
ABSTRACT
Current training data attribution (TDA) methods treat the influence one sample has another static, but neural networks learn in distinet stages that exhibit chang- ing patterns of influence. this work, we introduce a framework for stagewise data attribution grounded in singular learning theory. We predict that influence change non-monotonically, including sign flips and sharp peaks at developmental transitions. We first validate these predictions analytically and empirically in a toy model, showing that dynamic shifts in influence directly map to the model's 's progressive learning of semantic hierarchy. Finally, we demonstrate these phe- nomena at scale in language models, where token-level influence changes align with known developmental stages.
u
w
samples
P(w(D.)
MLE
Posterior Density
Regular Development (Bernstein von von-Mises)
Stagewise Development (Watanabe)
Figure Stagewise learning requires stagewise data attribution. (a) In regular models, de- velopment is uniform, monotonic process of posterior concentration around a single solution (Bernstein-wee von Mises). (b) singular models, development is stagewise process where the poste- rior undergoes phase transitions (Watanabe's s singular learning theory). This stagewise development means the influence one sample has on another can profoundly change over the course the learning process.
INTRODUCTION
Training data attribution (TDA) studies how training data shapes model behavior, a central problem in AI interpretability and safety (Cheng et al. 2025 Lehalleur al.] 2025). Understanding attribution requires accounting for the role of stagewise development and learning dynamics: when a model encounters given sample affects how the model learns from that sample. What helps the model learn "dog" in early training may actively harm its ability to distinguish "poodle" from "terrier" later.
Currently, however, most approaches to TDA still ignore the role of development. In particular, influence functions (IFs) assume that data ordering has no effect on influence, which implies that
Equal contribution.

---

Published as a conference paper at ICLR 2026
influence is static and global over the course of training (Cook) 1977 Cook & Weisberg 1980). This perspective, inherited from the amalysis of regular statistical models, breaks down catastrophically for deep neural networks (see Section 2.1.
The cause of this breakdown is degeneracy: neural networks have degenerate loss landscapes with non-isolated critical points and non-invertible Hessians. Singular learning theory (SLT) predicts that this degeneracy gives rise to stagewise development, where models undergo phase transitions marked by changes in degeneracy and Hessian rank (Watanabe) 2009] 2018). Taken together, this suggests that data attribution should take account of stagewise development, which motivates our work to connect influence functions and developmental phase transitions.
Contributions. We put forth three contributions:
1. A developmental framework for inffuence: We introduce theoretical framework, grounded in Singular Learning Theory (SLT), that connects influence functions to stagewise development. This predicts that influence is not static but can change non- non-monoton monotonically (including sign flips and sharp peaks) at the phase transitions that define stagewise learning. This motivates shift stagewise data attribution that studies the dynamics of influence over time.
2. Validation in controlled system: We analytically derive and empirically confirm predictions in a hierarchical feature-learning model (Saxe et al.) (019a). We provide a concrete -o-onctt-on showing that dynamic shifts in influence directly correspond to the model's 's sequential learning of the data's 's hierarchical structure.
3. Application at scale in language models: We demonstrate that these developmental phenomena are observable at scale in language models. In particular, we show that influence functions for tokens with key structural roles (e.g., delimiters, induction patterns) undergo non-monotone monoonnoton, sudden changes that align with known transitions.
Our findings challenge the foundational assumptions of the static influence paradigm, providing a new framework for understanding not just which data points matter, but when and why they matter during the learning process.
2
THEORY
This section develops our theoretical framework for stagewise data attribution. In Section 2.1 review the theory of stagewise development according to singular learning theory (SLT). We then, in Section 2.2 re-evaluate influence functions through this developmental lens, which motivates shift from the global, point-wise approach of classical influence functions to local, distributional variant. Finally, Section 2.3] we derive our central predictions: influence is not fixed property but changes non-monotonically, peaking at the phase transitions that define stagewise learning.
2.1 FROM UNIFORM TO STAGEWISE DEVELOPMENT
Developmental Interpretability: from SGD to Bayes and back again. While stochastic op- timizers (such as SGD and Adam) are the de facto approach to training deep learning systems, their complex dynamics make direct theoretical analysis difficult. To make progress. we follow the recipe of Developmental Interpretability (Lehalleur et al. 2025 Waag et al. 2025b): we model the optimizer's 's learning trajectory with an idealized Bayesian learning process, then apply singular learning theory (SLT: Watanabe 2009) to make predictions about stagewise development, and finally test those predictions empirically in real networks trained via stochastic optimization.
The regular learning process (Bernstein-von Mises), In regular statistical models (with a unique MLE and invertible Fisher information matrix, FIM), the Bernstein-von Mises (BvM) theorem predicts a smooth, monotonic learning process where the posterior narrows around a single solu- tion (Van der Vaart 2000). More precisely, as the number of samples increases, the Bayesian posterior converges to a Gaussian centered at the minimum w" of the population loss with covariance (n[("")) where I(w") is the FIM.
2

---

Published as a conference paper at ICLR 2026
The singular learning process (Watanabe). Neural networks violate the regularity assumptions required for the Bernstein- teasteeii-vonnnn von Mises theorem to hold. Not only do they have no unique minimum, but the loss landscape is degenerate: the Fisher information matrix is not everywhere invertible. SLT provides a framework for studying these singular models. [Watanabe] (2009) showed that degeneracy can give rise stagewise learning, where neural networks undergo succession of phase transitions between qualitatively distinct solutions, see Figure [
In this framework, development is driven by competition between data fit (or empirical loss, Ln(w) (w) E, ((ww) over dataset D of n. samples) and model complexity (as measured via measure of degeneracy known the local learning coefficient, X(w)). This evolving tradeoff can lead to first-order phase transitions, where the model abruptly shifts from concentrating in one region to another, and which can change the model's generalization behavior (see Appendix ( for formal treatment). As we will show in Section 2.3 these transitions are also responsible for the non-monotonic dynamics influence functions throughout the learning process.
2.2
FROM STATIC TO DEVELOPMENTAL INFLUENCE FUNCTIONS
The stagewise development of singular models requires corresponding shift in our tools for data attribution, moving from global, point-wise measures to local, distributional ones.
Classical influence functions: static view. Classical influence functions (IFs) are standard technique for training data attribution, quantifying how an infinitesimal upweighting of training point 2, affects an observable & evaluated at the final model parameters (Cook 1977). The
influence is given by:
0 IF(z;, Φ) 0B,
wlw w0wwwn (G))
= --vw@("" "T (w"""'''"
3=1
where H(w (w=) is the Hessian of the total loss evaluated the solution w".
Crucially, the classical IF relies on the same regularity assumptions required for the Bernstein-von Mises theorem: the existence of single, stable local minimum w², and invertible loss Hessian H (ww) at that minimum. As discussed, singular models like neural networks violate these conditions. Their loss landscapes are degenerate, featuring non-isolated minima and rank-deficient Hessians. This renders the classical IF theoretically ill-defined and practically unstable, which requires dampening factor (see Appendix C.4), especially intermediate checkpoints that are unconverged and away from minima.
Bayesian influence functions: a developmental tool. The Bayesian Influence Function (BIF) provides principled alternative that is well-suited to the dynamics of singular models (Giordano et al. 2017; Kreer et al.) 2025). Instead of measuring the change in point estimate of ( the BIF measures how the posterior expectation of observable E(o(w) w)) changes. This derivative is equivalent to the negative covariance between the observable and the sample's 's loss:
BIF Z, Φ)
E D) [d(w)]
-CV pt) (L((w,, d(w)).
as,
B=1
This formulation is ideal for studying development for three main reasons:
It is distributional. Its definition in terms expectations over posteriors makes a natural tool for the Bayesian learning framework of SLT.
It is inherently Hessian-free. By replacing the problematic Hessian inverse with covariance estimation, it remains well-defined even on degenerate loss landscapes.
It is well-defined at any point in the training trajectory, not just stable local minima. This is essential for studying influence as a dynamic quantity that evolves over time.
Moreover, when the regularity assumptions hold, the BIF asymptotically recovers the classical IF in the large-data limit (Kreer al. 2025); that is. the BIF is natural higher-order generalization of the classical influence function. For these reasons, we adopt the BIF as our primary tool for measuring influence.
3

---

Published as a conference paper at ICLR 2026
Estimating (local) Bayesian influence functions. To measure the BIF in practice, use
estimator based on stochastic-grann gradientt MCMC introduced in Kreer et al. (202)) This also introduces dampening term that enables localizing the BIF individual model checkpoints. For more details, see Appendix B, where we also discuss practical scaling advantages of the BIF compared to other popular IF methods, such EK-FAC (Grosse al. 2023)
2.3 STAGEWISE DATA ATTRIBUTION
Influence and susceptibility. In the language of statistical physics, the BIF is an example of a generalized susceptibility that measures a system's response (in this case, model's loss on sample j) to perturbation (in this case, change in importance of sample i). In physical systems, susceptibilities diverge at phase transitions, which makes them macroscopically the most legible signal that phase transition is taking place. This suggests using (Bayesian) influence functions to discover transitions during the learning process.
Setup: a bimodal posterior. A first-order phase transition is characterized by the posterior distri- bution p(wr 'D) having significant mass in two distinct neighborhoods, which we label /f and V. We can model this as mixture distribution:
p(w D) = Tup(w ()) + Typ(ww V)
where Ty and zy are the posterior probabilities of being in phase U or V respectively, with कy +2/+-10 -y 1. At the peak of a phase transition, Ty ~ Ro 0.5. Away from the transition, one of the weights is close to I and the other is close to O.
Decomposing influence with the law of total covariance. The BIF between samples i and is defined as BIF (z, f = -Cov p( D) (L,(w), Eg (w)). We can decompose this total covariance using the Law of Total Covariance, conditioning the phase (Z E {u, VH): + Cov(BB4 Z], E Z))
Cov(Li,
E[Cov e 2)
Average Within-Phase Influence
Between-Phase Phase Inffuence
Let's analyze each term:
1. Average Within-Phase Influence: This term the weighted average of the influence calculated strictly within each phase:
E[CovvL:, L 2) muCoove( Eg) + xyCovv(Le, This represents the "baseline" influence. there were no phase transition (e.g., Te = 1), this is the only term that would exist.
2. Between-Phase Influence: This term captures the covariance that arises because the expected losses themselves change the model switches phases. Let jaz = E[e, 24] be the expected loss of sample in phase U, and likewise for j and V. The term expands to:
Co E(e, Z]) = Tu Ty (pt. (1,v)) k5,y)
Predicting stagewise changes in influence. This decomposition predicts dynamic changing in- fluence patterns over learning. The departure from the classical view arises because influence is phase-depnndent: the baseline "within-phase influence may differ significantly across the transition (Covu * Covy), and the "between-phe phase" term introduces an additional effect during the transition. In particular, we derive two predictions that diverge from the classical view:
Influence Can Change Sign:
If the within-phase influences of the two phases have sig- nificantly different values or if the between-phase term is large enough to dominate the average baseline influence during the transition, then transitions can cause large change in magnitude or even change in sign.
Inter-Phase Inftuence Peaks at Transitions: The between-phase influence term is max- imized when the posterior mass is evenly split (Nu & my x 0.5), which under some conditions may cause a peak in total influence at the critical point of transition. The magnitude of the between- phase term is proportional to (μ4 (4,y), which means the influence spike is largest for the samples on which the two phases disagree the most: peaks in influence identify the specific samples that characterize given transition.
4

---

Published as a conference paper at ICLR 2026
Towards stagewise data attribution. These theoretical predictions call for shift from the classical static view of training data attribution to what we term stagewise data attribution: the analysis of influence dynamic trajectory over the entire learning process. The goal is to attribute learned behaviors not only to the data that influenced it, but also to the specific period of time in which that data had its effect. In the of this paper, turn to testing the basic predictions above in order to validate this developmental framework.
ToY MODEL OF DEVELOPMENT
The acquísition semantic knowledge involves the dynamic development of hierarchical structure in neural representations. Often, broader categorical distinctions are learned prior to finer-grained distinctions, and abrupt conceptual reorganization marks the non-static nature of knowledge acquisi- tion, studied in both psychology and deep learning literature (Keil 1979] Inhelder & Piaget 1958
Hinton 1986: Rumelhart & Todd 1993, McClelland 1995 Rogers & McClelland] 22044] For a
principled understanding of the dynamical aspect of influence training, study toy model of hierarchical feature learning introduced by Saxe et al. (2019)) for which ground truth and analytical tools accessible.
We confirm our theoretical predictions in Section 2.3 by finding that the dynamics of influence coincide with the stagewise development of hierarchical structure. Sign flips occur as the model shifts to learning progressively finer levels of hierarchical distinctions, and peaks occur when the model begins learning a new level of distinction.
Setup: a hierarchical semantic dataset. We train 2-layer deep linear network with MSE loss on hierarchical semantic dataset from Saxe et al. (2019a). The dataset consists of one-hot input vectors representing objects, and each input maps to an output vector representing a collection of features that the object possesses (Figure 2). Importantly, this toy model mathematically shows that deep neural network architecture develops neural representation that reflects hierarchical differentiation in progressive manner: learning animal plants first, then mammals vs. birds, and then dogs cats. See Appendix C for detailed description of the toy model and its analytical treatment.
a)
Y carreiation
pongun
foature
sрerrew
Loss
Startie
Index
Epochs
Figure 2: A toy model of hierarchical semantic knowledge acquisition. (a) toy dataset adopted from Saxe et al. (2019)) Each object maps to feature vector that describes the hierarchical structure of semantic knowledge, e.g.. 'penguin' and cat are all living animals, but 'penguin' is a bird and cat is a mammal. (b) The correlation matrix of the feature output shows hierarchical structure. (c) The hierarchical structure is acquired progressively during the training of the deep linear neural network.
Measuring influence dynamics. First, we probe the local BIF (Kreer et al. 2025) on the toy model over the entire learning trajectory. We use Appendix B to estimate posterior from each checkpoint w at training time t. See Appendix B for details of the local BIF implementation and Appendix C.2 for hyperparameter sweep. Furthermore, we derive the dynamics of the influence function analytically, leveraging the mathematical tractability of the toy model (see Appendix
SGLD sampler (described in
Leave-one-out (LOO) verification. To confirm our observation from the BIF and analytical treatment, conduct retraining experiments. Specifically, consider the Leave-One-Oui (L.OO)
5

---

Published as a conference paper at ICLR 2026
setting, where ablate one data point and measure the loss difference of other data points compared to the baseline loss without ablation. We measure the loss difference over training time t:
DC - -
measured at time where D is the full dataset, D i is the dataset with data index ablated, and is the index of the data point that we are querying. In Figure 3 we observe that the loss difference from LOO results in similar pattern what
in the BIF and the analytical derivation, and
validates their use. The additional results on perturbing other data points show a consistent trend in Appendix C.2 and are also visible in the dampened classical IF Appendix C.4
BIF BIF E2)
Analytic IF (dog, 4)
A A
query j
0.0002
pine
0.10
oak
0.0001
rose
0.05
0.00
lily
sparrow
0.0000
0.00
-0225
penguin
-0.50
-0.0001
-0.05
dog
-0,75
0
5k Epochs
10k
5k
10k
5k Epochs
10k
Epochs
Figure Influence over time on a hierarchical semantic dataset. We measure the influence of dog sample on other query samples with the following: (Left) BIF (3 1000, 1e-3, 5e + 3). (Center) Analytical IF (see full derivation in Appendix C.6p. (Right) Loss difference from Leave- One-Out (LOO) retraining experiment. All three measures agree that the influence one sample has on another can vary montmoootnnically over the course training as discussed in Section See Appendix for additional pairs of samples and experimental details.
With these results, we confirm the predictions from Section 2 (1) influence changes over time non- monotonically and can change sign, and (2) peaks influence are correlated with key developmental transitions in model behavior. In the rest of the section, we will establish how these observations are reflected the progressive learning of the hierarchical structure in our toy model.
"When" matters for measuring influence. We clearly observe that influence goes through non- monotonic change over the course of training, strengthening our argument that a static interpretation of influence is fallacious. In Appendix C.6 we analytically derive that influence is a function of singular mode strength of data input-output covariance learned by the network, which a time- dependent variable and thus justifies studying data attribution from dynamic perspective. Over the course of training, each data point induces a non non-static influence on other data points. The influence flip sign- the same data can be either helpful harmful, depending can also mark clear peaks at a specific time point on different query data points. Furthermore, the influence over time from one data point to another point is specific those points. The influence from dog to cat might be the same as dog to 'penguin' early on, but they are distinguished later in the learning process.
when it is presented. It
To confirm that these influence dynamics represent a causal effect on model performance, we perform an additional retraining experiment in Appendix C.3) Instead of removing sample for the full training run, we ablate specific data points only during short temporal windows. We find that ablating data precisely during the stage where the BIF assigns its peak influence induces the largest loss difference compared to ablation at other times. This demonstrates that our measure correctly identifies the critical window in which a specific sample drives the learning process.
Change of influence reflects stagewise learning. During the phase where one hierarchical distinc- tion (e.g., animal vs. plant) is being learned, upweighting data point in the same class (dog) is helpful for learning query data point (sparrow) covariance). In contrast, upweighting a data point that belongs to a different class harms learning that query data point (pine), reflected in positive influence. A data point dog is helpful (negative
indicated by negative influence (positive

---

Published as a conference paper at ICLR 2026
influence) to learning sparrow early on in the learning, while learning to distinguish an amial vs. plant, but it is harmful (positive) later on when learning to distinguish manmal vs. bi rd. Scaling (MDS, Torgerson 1952. Cox & Cox (008) of the hidden representation of each data over time as in Saxe et al. 201 2019a) where learning of each hierarchical level is reflected on the branching node (numbered). We observe that the time point of the branching node matches the peaks in influence. That is, at the transition, where the model learns to distinguish mammal vs bi rd within the animals and form new hierarchy level, the influence between mammal 1 and b rd is the highest.
In Figure b), we show
a)
BIF (dog, Ej)
b)
cat
dog
Epachs
query J
0.0002
hurts query j
pine
9000
oak
rose
0.0001
rose
MDS dim
liy
sparrow
6000
0.0000
sparrow
panguin
-05
penguin
fly
cat
-0.0001
dog
pine
helps query
oak
-0.5
0.5
Epochs
MDS dim
animals vs. plants
mammals vs, birds
dog vs.
Figure 4: BIF captures developmental influence. (a) The peak positive influence from dog to different data points is noted with D. 2, and 3. (b) MDS of the hidden representations in the network over the course of learning. The peaks in influence match the branching points of the MDS trajectory, where each hierarchical category develops (black points).
4 LANGUAGE MODELS
To investigate the dynamics of influence in a real-world setting, we study the acquisition of token-level syntactic knowledge using language models from the Pythia Scaling Suite (Biderman al. 2023)
We confirm our theoretical predictions in Section 2.3 by finding non-monotonic influence trajectories, with large changes in magnitude, sign flips, and peaks that correspond to known developmental changes like the formation of the induction circuit.
Per-token influence functions. One clear benefit of the BIF in the language-modeling setting is that computing influence at the level of individual tokens rather than sequences incurs no additional computational cost. Loss is computed on a per-token basis during MMSSprop-SGLD which is already necessary when considering the autoregressive losses that represent the standard for LLM pretraining:
((w))= =
Σ Elri, 01.0 ... .., 1,, W),
where Figy is the jth token in the ith text sequence. These per-token losses can then be stored individually and used to estimate the per-token BIF matrix for the relevant dataset.
Classifying tokens into syntactic classes. Following the experimental setup of Baker et al. (025) we classify individual tokens according to how they are used to give structure to text. These classes include strictly syntactic tokens (left delimiters, right delimiters, and formatting tokens-such as newlines), morphological roles (parts of words and word endings), and a broader structural class- tokens that have been used conjunction earlier the context, forming an inductive pattern. We note that this classification is not exhaustive, nor is it exclusive- not all tokens have class, and some may occupy multiple classes at once. A full classification is provided Appendix D.I.I
Calculating group influence. To estimate patterns of influence between tokens across classes, we make use of the following procedure:
7

---

Published as a conference paper at ICLR 2026
1. Using subset of The Pile (Gao et al. 2021). compute the normalized BIF (Kreer et al., 2025) between all pairs of tokens, sampled from the SGLD-estimated model posterior, as described in Appendix B
2. Each token is classified in accordance with the listed structural classes based on pattern- matching, following Baker et al. (0225).
3. For every possible pair of classes, we compute the average (or "group") influence between tokens in these classes, excluding influences between the same token in different classes.
These inter-class influences can then be used to provide insight into how much influence tokens from one class have on tokens from other classes.
Learning induction. In Figure 5 we plot the inter-class influence for each query class across all class pairings at several model checkpoints. From these plots, some clear dynamics emerge. Using the BIF, we see that the formation of model structure corresponding to induction relationships starts as early as 128 steps into learning, when there an inflection point in the average BIF between induction tokens. This continues to strengthen for the next 30k training steps before appearing to peak and fall. This echoes the results of Tigges et al. (2024), which finds that Pythia models begin to learn the induction circuit at this time, and exhibit an apex at 30k training steps before diminishing.
In Appendix D.3
we study a small language model trained from scratch and track the influence on targeted set of synthetic samples to automatically identify the induction bump. We find that the influence of induction-pattern tokens sharply rises as the induction circuit is being learned. We further validate the practical utility of these insights with stagewise intervention experiment in Appendix D.4 We find that upweighting induction patterns specifically when the induction circuit begins form accelerates the formation of induction heads significantly more than upweighting them before this window. This confirms the prediction derived from the influence dynamics: influence cannot be fully understood from static analysis at the end of training.
Inter-class BIF Pythia 14m (Logarithmic Checkpoints)
Induction
Left Delimite
Right Delimbeer
0.01
-0.02
-0.03
Averge Class BUF
-0.04
-0.05
Formatting
Word Part
Word End
-0.01
-0.03
-0.03
-0.5
10
10a
10a
10
10F
10
10
10F
10a
10""
Model Chockpa nt
Induction Left Dehmmter
Right Delimttr
Pormatting word Part Word and
Figure 5: Token-class relationships. BIF between structural classes through training. We observe structural relationships between tokens reflected in influence patterns between classes, including strong intra-class relationships, development of induction, and relationships between word elements and corresponding delimiter token classes. Dashed lines indicate major inflection points in the BIF Opacity captures notable influence patterns.
Learning where to end. Another notable pattern occurs in the influence traces of left delimiters and right delimiters. Both classes appear among constructive (negative) influence tokens for the
8

---

Published as a conference paper at ICLR 2026
other class very early in training, but this relationship quickly inverts, with tokens from the opposite delimiter class becoming increasingly positive as the model learns to distinguish between these classes. We also see increasing influence between left and right delimiters and related classes that may perform similar roles in structuring text. Left delimiters share a similar influence pattern overall with parts of words but not the ends of words. while right delimiters influence ends of words but not word parts. These relationships are also visible in the reverse direction (from word parts/word ends to left/right delimiters). These relationships develop over time, with a particularly notable spike in the relationship between word parts and left delimiters later in model training.
Dynamics of influence. Taken together, these results demonstrate that influence between different classes of tokens is not static property but a dynamic one that evolves throughout training. We observe changes in both the magnitude and sign of influence, with the timing of these shifts varying depending on the specifiç structural capability being learned. For instance, the influence related to induction patterns exhibits non-monotonic peak that aligns with the known developmental phase transition for this circuit. Similarly, the relationship between left and right delimiters, as well as the relationship between word parts/word ends and left/right delimiters undergoes sign flip, indicating qualitative shift in how the model processes scope and pairing. These findings confirm the predictions of our stagewise framework: different data becomes influential at different times, reflecting the model's 's progressive, stage-by stage-by-stage by-stage acquisition of syntactic and structural knowledge.
5 DISCUSSION & CONCLUSION
Stagewise data attribution and developmental interpretability. This work places data attribution within the context of Developmental Interpretability. We follow the established methodology of using singular learning theory (SLT) theoretical lens to study the dynamics of models trained with stochastic optimizers. Prior work has successfully used this pipeline to explain phenomena like phase transitions in toy models of superposition (Chen et al. (023). algorithm sclection in transformers (Carroll et al. (022), stagewise learning in toy transformers (Urdshals & Urdshals 2025), and the stagewise emergence of structure (e.g., n n-grams, induction, marenthesis-matchhee space-counting) in language models (Hoogland et al. [2024) Wang et al.) [2025b] Baker et al. 2225 Wang et al. 2025a)). Our contribution is to use the SLT account phase transitions to predict and subsequently verify that training sample's 's influence is not static but a dynamic quantity that evolves with the model's 's development.
Stagewise vs. sarolling-based attribution. Our work complements another important line of research that moves beyond static influence functions: trajectory-basee or "unrolling" methods like TracIn (Pruthi et al., 2020), HyDRA (Chen et al., 2021). and SOURCE (Bac et al. 2024). These techniques approximate the total influence of a sample by integrating its contributions (such as gradient updates or local influence scores) across numerous checkpoints along the full training path. This approach provides more faithful account of the path-dependent nature of SGD and can offer more accurate attribution scores than single-point estimates.
However, our findings regarding influence dynamics raise the possibility of cancellation effects in these unrolling-based measures. Since influence can flip signs during training, opposing contributions from different stages could potentially offset each other when integrated. While such score may still accurately reflect the expected cumulative impact, it risks obscuring the data point's 's role in driving specific developmental stages. Fundamentally, the goal of these methods differs from ours: unrolling techniques compute a single, cumulative summary of sample's total impact, treating the learning process as a black box. In contrast, our framework treats the learning trajectory as an object of study in itself, aiming to understand when data matters during development.
A mechanism for implicit curricula. The concept of curricluum-learinn from easier to harder data has been argued consistently for more efficient neural learning (Bengio et al. 2009 Wang 2024). However, the efficacy of this technique has been shown to be limited
ct al. 2 Lee et al.
2020; Mannelli et al. (0244). One prevalent explanation is that an implicit curriculum is adopted by learning through gradient descent in neural networks (Graves et al. 2017 Rahaman et al.) 2019 [axe et al.) 2019a) Valle-Peree et al. 2018). Our findings provide a new, more granular mechanism for understanding this phenomenon. The stagewise evolution of influence
in practice (Wu et al.
9

---

Published as a conference paper at ICLR 2026
demonstrates how different data points become "important" at different moments, effectively creating a dynamic, self-organizing curriculum.
Limitations and future work. Our framework points toward several avenues for future research. The primary theoretical gap remains the link between the Bayesian learning process of SLT and the non-equilibrium dynamies of SGD. On the empirical front, key direction is to move from a behavioral to a mechanistic account of influence. Ulimately, model generalization is grounded in the circuits and internal structure model acquires over training. A more complete science of interpretability therefore understands not just which samples are most influential (data attribution), but when they are most influential (stagewise attribution), and how they shape model internals such as features and circuits (mechanistic interpretability).
From pointwise to stagewise data attribution. Ultimately, this paper argues for shift in how approach training data attribution. The static perspective, which assigns a single, global influence score each data point, only the end of training, offers an incomplete and, times, misleading picture. By demonstrating that influence is inseparable from development, we advocate for moving from point-wise to stagewise data attribution.
This developmental lens is essential for tackling a key scientific challenge: understanding the correspondence between data structure, loss landscape geometry, learning dynamies, and model internals (Wang al.) 2025b). Stagewise data attribution, by tracking influence dynamics, provides concrete tool map these connections, opening new possibilities for interpreting, debugging, and ultimately steering how models learn.
ACKNOWLEDGMENTS
We would like to thank Simon Pepin Lehalleur for his detailed feedback on an earlier version of this manuscript. We are also grateful to Daniel Murfet, Andrew Saxe, Andrew Lampinen, Aaditya Singh, and Philipp Alexander Kreer for their valuable feedback and insightful discussions. We thank Rohan Hitchcock for his helpful input on hyperparameter calibration.
Jin Hwa Lee and Matthew Smith gratefully acknowledge the support of the Pivotal Mentorship Program, and thank Morgan Simpson and Euan McLean for their guidance, as well as Guillaume Corlouer and Avi Semler for their insights. Jin Hwa Lee thanks the Gatsby Charitable Foundation (GAT3755) and The Wellcome Trust (219627/7/19/Z), Jin Hwa Lee and Matthew Smith would also like to thank LISA for providing a productive working environment and support for this project. Finally, we thank Stan van Wingerden for his assistance with the compute infrastructure.
AUTHOR CONTRIBUTIONS
Jin Hwa Lee developed, implemented, and analyzed the toy model experiments. Matthew Smith designed and conducted the language model experiments. Maxwell Adam helped set up the language modeling experiments and contributed the induction-formatie investigation. Jesse Hoogland con- ceived the project, developed the theoretical framework, and led the research. All authors contributed to the writing of the manuscript.
REPRODUCIBILITY STATEMENT
We describe our primary method, the Bayesian Influence Function (BIF) estimated via RMSPropS- GLD, in Appendix B We first validate our predictions in controlled toy model (Section 3) providing analytical derivations, Seave--neeOne (LOO) verification, and all hyperparameters in Appendix C findings at scale using public Pythia language models The Pile dataset
We then demonstrate (Section 4.. with all experimental specifics, token classification schemes, and hyperparameters detailed in Appendix D
10

---

Published as a conference paper at ICLR 2026
LLM USAGE STATEMENT
We used large language models (LLMs) to assist in the preparation of this manuscript. Their role included proofreading, correcting grammatical errors, and rephrasing sentences to improve clarity and flow. Beyond surface-level edits, we utilized LLMs as a collaborative tool for brainstorming options for narrative structures and receiving feedback the clarity of our arguments. LLMs also served as coding assistants in implementing our experiments and generating visualizations. Additionally, we used LLMs aid in the research process, for instance, by suggesting relevant theoretical concepts such as the Law of Total Covariance for the decomposition in Section 2 In all instances, the final content, including all code, theoretical claims, and text, was reviewed, validated, and is the sole responsibility of the authors.
REFERENCES
Maxwell Adam, Zach Furman, Wilson Wu, Philipp Alexander Kreer, and Jesse Hoogland. The loss kernel: A geometric probe for deep learning interpretabilityy 2025.
Miki Aoyagi. Consideration on the learning efficiency of multiple-layered neural networks with linear units. Neural Networks, 172:106133 2024.
Juhan Bae, Nathan Ng, Alston Lo, Marzyeh Ghassemi, and Roger Grosse. If Influence Functions the Answer, Then What is the Question? 2022. CitationKey: pbrf-if.
Juhan Bae, Wu Lin, Jonathan Lorraine, and Roger B. Grosse. Training data attribution via approximate unrolling. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024.
Garrett Baker, George Wang, Jesse Hoogland, and Daniel Murfet. Structural Inference: Studying Small Language Models with Susceptibilities, April 2025. URL htt.tttt///aaxx...r//////
22504.18790 arXiv:2504. 18274 [cs].
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pp. 41-48, 2009.
Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O' Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar Van Der Wal. Pythia: a suite for analyzing large language models across training and scaling. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023.
Liam Carroll, Jesse Hoogland, Matthew Farrugia- and Daniel Murfet. Dynamics of Transient Structure in mn-Cottext Linear Regression Transformers, January 2025. URL http:://axxiv org/abs 2501. 17745 arXiv:2501.1774: [cs].
Yuanyuan Chen, Boyang Li, Han Yu, Pengeheng Wu, and Chunyan Miao. HyDRA: Hypergradient
Data Relevance Analysis for Interpreting Deep Neural Networks. Proceedings of the AAAI
Conference on Artificial Intelligence, 35(8):7081-700
May 2021. ISSN 2374-3468. doi: 10.
1609/aami 1351..16871 16871. URL https: /ojs js. aaai org/ index..dexxe//// https//pphtaaaa/aaal.ac.c
view/16871 Number: 8 CitationKey: HyDRA.
Zhongtian Chen, Edmund Lau, Jake Mendel, Susan Wei, and Daniel Murfet. Dynamical versus bayesian phase transitions in toy model of superposition. arXiv preprint arXiv:2310.0630/, 2023.
Deric Cheng, Juhan Bae, Justin Bullock, and David Kristofferson. Training Data Auribution (TDA): Examining Its Adoption & Use Cases, January 2025. URL http:/tttp::ti....eeea a2502 arXiv:2501. [cs].
12642
R. Dennis Cook. Detection of influential observation in linear regression. Technometrics :
journal of statisstics for the physical, chemical, and engineering sciences, February 1977. ISSN 0040-1 1706. URL lttps::/// www. www..tannde. tandfonling com/ doi/abs /10. 1080/004417006 1977. 10489493
tex.copyright: Copyright Taylor and Francis Group, L.C.C
11

---

Published as a conference paper at ICLR 2026
R Dennis Cook and Sanford Weisberg. Characterizations of empirical influence function for detecting influential cases in regression. Technometrics, 22(():495 508, 1980.
R. Dennis Cook and Sanford Weisberg. Residualy and influence in regression. Monographs on statistics and applied probability. Chapman and Hall, New York, 1982. ISBN 0-412-24288-0, URL
https: hdl handle net meeeeee19//3777777
Michael AA Cox and Trevor Cox. Multidimensional scaling. In Handbook of data visualization, pp. 315-347. Springer, 2008
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The Pile: An 800gb dataset of diverse text for language modeling. CoRR, abs/2101.00027, 2021. URL
https: wrg. 2101. 001111000000 00027
Ryan Giordano, Tamara Broderick, and Michael I. Jordan, Covariances, robustness, and variational
Bayes. Journal of Machine Learning Research, 19:51:1-51:49, 2017. URL https://app
Semanticschoo lar. ottp://eess.tttlooooooooooooooaa 53238793)
Alex Graves, Mare G Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu. Automated curriculum learning for neural networks. In international conference on machine learning, pp. 1311-1320. Pmlr, 2017.
Roger Grosse, Juhan Bac, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamilé Lukošiūté, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R. Bowman. Studying Large Language Model Generalization with Influence Functions, August 2023. URL http://aarxi org/ hbs:/ab/// 2308 03296 arXiv:2308.0322 [cs].
Geoffrey E Hinton. Learning distributed representations of concepts, In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 8, 1986.
Rohan Hitchcock, Gary W Delaney, Jonathan H Manton, Richard Scalzo, and Jingge Zhu. Emergence of computational structure in neural network physics simulator. arXiv preprint arXiv:2504.11830, 2025.
Jesse Hoogland, George Wang, Matthew Farrugia-Robess Liam Carroll, Susan Wei, and Daniel Murfet. The developmental landscape of in-context learning. arXiv preprint 2tt::220020023664 2024.
Jesse Hoogland, George Wang, Matthew Farrugia-Robert Liam Carroll, Susan Wei, and Daniel Murfet. Loss landscape degeneracy and stagewise development in transformers. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. httttttss///oeeeee
forum?31d-45aayBBG30
Bärbel Inhelder and Jean Piaget. The growth of logical thinking from childhood to adolescence: An essay on the construction of formal operational structures. Routledge, 1958.
Frank C Keil. Semantic and conceptual development: An ontological perspective. Harvard University Press, 1979.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions, arXiv:1703.04730 [stat]
December 2020. httptp////aail..oo////////////// wttp:/v..rr//////110000 CitationKey: deep-inffuenne--unncioon
Philipp Alexander Kreer, Wilson Wu, Maxwell Adam, Zach Furman, and Jesse Hoogland. Bayesian influence functions for hessian-free data attribution, 2025.
Edmund Lau, Zach Furman, George Wang, Daniel Murfet, and Susan Wei. The local learning coefficient: singularity complexity measure. In The 28th international conference artificial intelligence and statistics, 2025. URL httpp:://ppezeeeiiiie net. wtt.:/wwwwweee/oo.........
1av5 Il.u..
12

---

Published as a conference paper at ICLR 2026
Jin Hwa Lee, Stefano Sarao Mannelli, and Andrew Saxe. Why do animals need shaping? theory of task composition and curriculum learning. arXiv preprint arXiv:2402.18361, 2024.
Simon Pepin Lehalleur, Jesse Hoogland. Matthew Farrrugi-Rooeerss, Susan Wei, Alexander Gietelink Oldenziel, George Wang. Liam Carroll, and Daniel Murfet. You Are What You Eat- Al Alignment Requires Understanding How Data Shapes Structure and Generalisation, February 2025. URL
bttt:://rxiie www.o//gg//o... abw.////22222 05475 arXiv:2502.05475 [es].
Chunyuan Li, Changyou Chen, David Carlson, and Lawrence Carin. Preconditioned stochastic gradient langevin dynamics for deep neural networks. In Proceedings of the AAAI conference artificial intelligence, volume 30, 2016.
Stefano Sarao Mannelli, Yaraslau Ivashynka, Andrew Saxe, and Luca Saglietti. Tilting the odds at the lottery: the interplay of overparameterisation and curricula in neural networks. Journal of Statistical Mechanics: Theory and Experiment, (024(11):11 14001, 2024.
James Martens and Roger Grosse. Optimizing neural networks with Cronecker-factoree approximate curvature. Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 2408 2417, Lille, France, 07-09 Jul 2015, PMLR. URL
martens15. h
JL McClelland. A connectionist perspective knowledge and development. In Developing cognitive competence, pp. 157-204. Psychology Press, 1995.
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. In-context learning and induction heads. Transformer Circuits Thread, 2022. heads//ndex..tmm
http::/ttaasfommrrccccutsss pubb:202/.pu httpssssspppppoioooooooeeeeeeeie
Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimat- ing Training Data Influence by Tracing Gradient Descent. In Advances in Neu- ral Information Processing Systems, volume 33, pp. 19920-19930. Curran Asso- ciates, Inc., 2020. URL https::/pproceeeiingt neurips cccccc///aa. http//aapr/222//////// / Absteecccbbrrtaccc html CitationKey: TracIn.
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias neural networks. In International conference machine learning, pp. 5301-5310. PMLR, 2019.
Timothy T Rogers and James L McClelland. Semantic cognition: A parallel distributed processing approach. MIT press, 2004.
David E Rumelhart and Peter M Todd. 1 learning and connectionist. Attention and performance X/V: Synergies experimental psychology, artificial intelligence, and cognitive neuroscience, 14: 3, 1993.
Andrew M Saxe, James L McClelland, and Surya Ganguli. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv: 2013.
Andrew M. Saxe, James L.. McClelland, and Surya Ganguli. A mathematical theory of semantic development in deep neural networks. Proceedings of the National Academy Sciences, 116(23): 11537-11546, 2019a doi: 10. 10.10/3ppss. htt.://paa...2221111111 abs/10.1073/ http:///ppaaa.122211111111
1820226116. URL https:tp///. www tttp//wwwwpppwppp////////tt/..sp pnas org//oi//
Andrew M. Saxe, James I.. McClelland, and Surya Ganguli. A mathematical theory of semantic development in deep neural networks. Proceedings of the National Academy of Sciences, 116(23):
11537-11546, June 2019b. ISSN 0027-8424, 1091-6490, doi: htt.3//aa..1802266666 URL https://pnase org/ doi / www./llll///// http:///.11111//aaaaa.aa 1820226116
Gilbert W Stewart and Ji-guang Sun. Matrix perturbation theory. (No Title), 1990.
13

---

Published as a conference paper at ICLR 2026
Curt Tigges, Michael Hanna, Qinan Yu, and Stella Biderman. LLM Circuit Analyses Are Consistent Across Training and Scale. In The Thirty-eighth Anmual Conference on Neural Information Processing Systems, November 2024.
Warren S Torgerson. Multidimensional scaling: I. theory and method. Psychometrika, 17(4):401-41, 1952.
Einar Urdshals and Jasmina Urdshals. Structure development in list-sorting transformers. arXiv preprint arXiv:2501. 2025.
Guillermo Valle-Perez, Chico Q Camargo, and Ard A Louis. Deep learning generalizes because the paramiter-function map is biased towards simple functions. arXiv preprint 2018.
Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
George Wang, Garrett Baker, Andrew Gordon, and Daniel Murfet. Embryology of a Language Model, August 2025a.
00331 arXiv:2508.003: [es].
George Wang, Jesse Hoogland, Stan van Wingerden, Zach Furman, and Daniel Murfet. Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficient. In Proceedings of The 13th International Conference on Learning Representations, 2025b. URL https:// openreview net http:://w.......o...eo........e.
Xin Wang, Yudong Chen, and Wenwu Zhu. survey on curriculum learning. IEEE transactions on paltern analysis and machine intelligence, 4409):555-4577, 2021.
Sumio Watanabe. Algebraic Geometry and Statistical Learning Theory. Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press, 2009.
Sumio Watanabe. Mathematical theory of Bayesian statistics. Chapman and Hall, 2018.
Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th International Conference on International Conference Machine Learning, ICML' 11, pp. 681-6666 1-88,8 Madison, WI, USA, 2011. Omnipress. ISBN 978 9781450306195
Xiaoxia WU, Ethan Dyer, and Behnam Neyshabur. When do curricula work? arXiv:2012.03107, 2020.
arXiv preprint
14

---

Published as a conference paper at ICLR 2026
Appendix
1. Appendix A Phase Transitions. This section provides a formal outline of how phase transitions arise in singular models and how they impact influence functions.
2. Appendix B. Bayesian Influence Functions. This section provides additional experimental details on estimating the local BIF.
3. Appendix C Toy Model Details and Additional Results. This subsection describes the hierarchical dataset, model architecture, and Leave-One-Out (L.OO) training procedure. It also contains additional LOO results and the full analytical derivation of the influence dynamics in the deep linear network model.
4. Appendix D: Language Model Details and Additional Results. This subsection describes the structural token classification scheme, provides RMSPropSGLD hyperparameters for the Pythia experiments, and details additional validation experiments.
A PHASE TRANSITIONS
In this appendix, we provide theoretical justification for the claims made Section regarding phase transitions and their effect model predictions and influence, grounded in statistical physics and the theory of Bayesian statistics. This theoretical treatment motivates our empirical investigation into the interaction between stagewise development and influence functions for models trained via standard stochastic optimizers.
A.1 BIF AS SUSCEPTIBILITY
Setup. We begin with the model-truth-prion triplet:
1. The model píz w) assigns probability sample Z for given choice of weights E WW C Rd.
2. The truth or data lata-gennrating distribution ((z) generates dataset of i.i.d. samples = H
3. The prior p(w) assigns an initial probability distribution to each choice of weights.
From this triplet, we obtain a posterior through (the repeated application of) Bayes' rules:
p(D) w)
p(w D)
p(D)
where p(D | w) p(z w) is the likelihood, and (DD) = Jw P(D w) is the marginal likelihood. Often, we're interested in tempered Bayesian posterior:
(D u) ())(())
e (-3-(())p((u)
Pa(w D)
(5)
(
P (D)
where B is a vector of per-sample importance weights, and {(w) is the vector of per-sample losses for given weight w.
The (global) free energy formula. The (global) free energy is the negative log marginal likelihood, F = - log Pog)) asymptotic (in the limit of infinite data) expression for this quantity. For critical point w" in neighborhood W, the free energy asymptotically expands as
central result of singular learning theory (SLT; Watanabe 2009) is an
F. = P8
= nLL((W)) A(w") logn + Op l)g
(6)
where A(w*) ) is the learning coefficient, degeneracy-awee aware measure of complexity (that coincides with half the effective parameter count for minimally singular models).
15

---

For example, in the learning theory setting, the expected (per-sample) loss can expressed as Published as a conference paper at ICLR 2026
Free energy moment-generating function. In statistical physics and thermodynamics. the free energy is the central object of study. This is because, if one manages to obtain a closed-form expression for the free energy, it is possible to calculate arbitrary expectation values simply by differentiating the free energy. Expectation values are of primary interest because they correspond to the things measure in experimental setting.
OFF
[[([ = 3 8-1
Combining Equation 7 with Equation 2 lets us express the BIF as a 2nd-order derivative of the free energy, assuming both samples are in the training dataset:
dEgle (
F F
BIF 22)
(8)
08,
03,03, 3-1
Susceptibilities are defined as second-order derivatives of the free energy, which makes the BIF an example of generalized susceptibility.
A.2 INTERNAL MODEL SELECTION
The (local) free energy formula. The local free energy formula is defined analogously to the global free energy. but with the domain of integration restricted to a particular region of parameter space or "phase" W W:
log / Jw
Fd F (W)
(D w) (w))(w))..
(9)
This admits analogous asymptotic form to Equation 66:
F. (w(W) = F. (W)
= nnnnww" + >(w")(w)))))))) log + (loggogg))
(10)
3=1
but where now wh is a local minimum within W, and À is the local learning coefficient associated with that local minimum (Lau et al. 2025).
Coarse-graining. Given partitioning of parameter space into multiple disjoint "phases" W U,W, the global free energy can be computed from the local per-phase free energies as follows:
Fn - log / e (DW,, ((w)dd
(11)
log Σ Wi
(_lLaa()w) p(w)d
(12)
log
F.(WW)
(13)
2 min F2 (W.).
(14)
The last line follows from the well-known of log-sum exponential as smooth approximation for the max function. This is to say that globally, the free energy is determined primarily by the phase with the lowest free energy, with exponentially suppressed contributions from all other regions of phase space.
Competition between phases. Assume the posterior distribution concentrated in just two distinct neighborhoods, which we label & and V. and is vanishing everywhere else. That is, these two phases
16

---

Published as a conference paper at ICLR 2026
constitute degenerate set of minimizers of the free energy. We can then write the posterior mixture distribution:
p(w D) p(w (2) D) + p(w V)p(V/ D)
where p(w (4) the posterior distribution conditioned on the model being in phase U, and p(4 D) is the total posterior probability of this phase, with Tu + Ty 2 1.
First-order phase transitions. This free energy formula predicts the existence of first-order "Bayesian" phase transitions. When two solutions compete, c.g., with neighborhood U and with neighborhood V. then the posterior log-odds evolve as log (x(())
Pr, (V) nAL, + log(n) AA Opllo log nt), where dn = (v) - Ln) and AA = A(v) - -(u). If has higher loss but lower complexity (A O, Aλ 0), the posterior initially prefers the simple solution U but switches to
the complex solution V when log(n)/n < -LL. AX. This is a "Type-A" transition (Carroll et al.
2025 Hoogland et al., 2025), see Figure
Alternatively, if the two phases agree on the linear term (they have the same minimum loss), then the trade-off will pushed down into lower-order terms, and there can be "Type-B" transition, in which complexity (as measured by the LLCC decreases, in exchange for an increase in lower-order terms.
B BAYESIAN INFLUENCE FUNCTIONS
Estimating the BIF with SGMCMC. To estimate the BIF in practice, we use an RMSProp- preconditioned SGLD sampler (Weling & Teh 2011 Li al. 2016) to estimate posterior from each checkpoint w* This approximates Langevin dynamics with loss gradients and locality regularization from initial w" modulated by localization strength "
V + yww, w²') + +0,
w x+1 W,
(15)
iE
The update rule takes ad advantage of an adaptive learning rate é, compared to vanilla SGLD and is more robust varying step size (Hitchcock et al. 2025). The full algorithm described in Algorithm T
Practical considerations with the BIF. As mentioned in the main body and following the discus- sion length in Kreer al. (025), we consider several practical modifications the local BIF, First, normalized BIF for the language- modeling experiments, which involves computing the Pearson correlation instead of the covariance over losses. Empirically, we find that this behaves more stably than the raw covariance and thus is easier to track over time. Second, consider per-token BIF, which can be trivially obtained by avoiding the loss accumulation over token indices and saving all per-token losses at each SGLD draw. Finally, avoid potentially spuriously high covariances, we drop same-token influence scores when computing aggregate group influence scores, as in Adam al. (025)..
Addressing influence from unseen training samples. A potential objection to our developmental analysis is that at early checkpoints, the model's 's optimizer has not yet encountered every training sample. It might therefore seem paradoxical to measure the "influence" of sample the model has not yet "seen." "
This concern can be straightforwardly resolved. The BIF is defined as the sensitivity of observable to a sample's is contribution weight, 3, to the total loss. For a sample z; that has already been processed, measure influence by considering perturbations around its bascline weight of B 1. For a sample the optimizer has not yet encountered, we can simply treat its current weight as B, = 0 and evaluate the same derivative at that point.
The resulting quantity remains well-defined, with a slightly different interpretation: it measures the model's sensitivity to the initial introduction of new sample, rather than the re-weighting of an existing one. Our practical implementation naturally accommodates this, as we use independent data sources for the SGLD gradient updates and the forward passes used compute the loss covariance.
17

---

Published as a conference paper at ICLR 2026
Algorithm 1 RMSPPopSGLD for Bayesian influence
Input: Initial model parameters w" E w, training dataset
(= 1. loss functions ((z;-) w + IR for each E [n], observables 0y: w + R for each j E [n], RMSPropSGLD hyperparameters (step size), /3 (inverse temperature), 7 (localization), m (sampling batch size), C (number of chains), T (chain length), (decay rate), a (stability constant).
Output: B = (BIF(z;, ()) si) R
L - CT 4 +- 0xxCT
for - c C do
w"
for 1 t do
for 1 Si do
(e t (ww)
D Compute train losses
end for for 1 S - m do
1)-++ - (5(w)
D. Compute observables
end for Sample full batch B of size m = n for small toy dataset.
V, + iV, (1 b) V..
V.
1 V
I-
Vi+a
f oj No),1)
8m m
ELEB, Vill(we) + +-w w*)) + Vene
SGLD update
end for
end for
1 B CT-1 L ICT colertor) T
Covariance between and +
Return B
Scalability of BIF. Besides the theoretical advantages discussed in Section 2.2 the BIF also has several practical advantages over classical IF approximations, as discussed in Kreer et al. (0225).
First, classical approximations like EK-FAC Grrsse al. 2023) incur a substantial up-from compu- tational cost to "fit" the inverse Hessian estimate. The BIF, in contrast, has no upfront fitting cost, This comes at the cost of having a higher compute cost per individual query. This means that the BIF is more computationally efficient for smaller datasets, while the classical IF is more efficient for larger datasets, where it can amortize the upfront costs over many queries. For the settings we considered, where the focus is studying development, the priority coverage than coverage over the entire dataset. These tradeoffs favored the BIF as the more tractable choice.
many checkpoints rather
Furthermore, our focus on fine- fineigrained structural attribution places us in the specific regime where BIF most efficient: computing dense, per-token influence matrices for targeted subsets data. Unlike Hessian-bait based methods, which typically require sequential scoring passes isolate each individual token contribution, the BIF estimator computes the full set token-by-toke influences in single batched process without additional memory overhead for backpropagating individual per-token gradients.
C TOY MODEL OF DEVELOPMENT
In this section, we provide details the toy model experimental details and analytical investigation of deep linear neural network learning dynamics under the data perturbation introduced in Section B It includes training setup of toy data with 2-layer deep linear neural network and BIF hyperparameter sweeps, and extended results of the Leave-One-Our (1.00) experiments and BIF measurements.
18

---

Published as a conference paper at ICLR 2026
C.I TOY DATASET
We use the semantic hierarchical dataset introduced in Saxe et al. (2019a). We set the number of data points N with hierarchy level H 3, that is, the data can be organized by 4 levels of hierarchy, e.g., dog belongs to the lowest hierarchy level of living organism, and level of animals (vs. plants), and level 3 of mammals (vs. birds). The input is data index given by an identity matrix of size N x N. The model needs to learn to associate with the feature of each data point, which forms the above hierarchical structure, the output size of N x O with O
We train 2-layer bias-free linear neural network with 50 hidden neurons, which is overparameterized for our rank 8 dataset. We initialize the network with small weights by sampling from a Gaussian distribution N (0, o²), le-3, We train the network on mean squared enror (MSE) loss with small learning rate 5e-S and train for 10K epochs with full batch unless mentioned otherwise.
C.2 BIF ADDITIONAL RESULTS AND HYPERPARAMETER SWEEP
For principled decision of hyperparameters for the SGLD sampler, we perform preliminary hyperparameter sweep on the SGLD sampler using the same deep linear neural network architecture and the data to estimate LLC. Since ground truth LLC is known for deep linear neural networks (Aoyagi 2024), we can select a range of hyperparameters by choosing ones that have a well- matched LLC estimate to the ground truth. Based on this procedure, RMSPPOOSGLLD to vanilla SGLD and also the insensitivity of the sampler quality to decay rate and stability constant in certain range and fixed the value to reduce the grid search dimension. We also narrowed down the localization strength range.
validate the superiority of
The range of the conducted hyperparameter sweeps for BIF measurement is summarized in Table The values given in the range were sampled on a logarithmic scale. RMSPropSGLD sampling procedure is shown in B
Table 1: Summary of hyperparameter sweep range for BIF experiments toy model.
Hyperparameter
Range
(step size)
[le-7 1e-2]
8 (inverse temperature)
[le+1 le+4]
7 (localization strength)
[le-2 1e+6]
m (batch size)
8 (full batch)
C' (number of chains)
[2,4.8]
T (chain length)
[200, 400, 800, 1000] [0.8, 0.9, 0.95, 0.99]
(decay rate)
(stability constant)
[le-4 1.0]
We present the Pearson correlation coefficient between Leave-One-Out (LOO) loss difference trajec- tory and BIF trajectory over learning over varying hyperparameter choices of inverse temperature 3, localization strength 7 and step size in Figure 6 In Figure 6 we see that the correlation co- efficient changes smoothly as we vary the hyperparameters. In the main Section 3 Figure 3 we showed the BIF measured with the hyperparameters that have the highest correlation with LOO
(3 1000, e = 1e-3, 7 5e 3).
C.3 LEAVE-ONE-OUT (LOO) EXPERIMENT
For the Leave-One-Out (LOO) experiment, which is also referred to retraining, we mask the corresponding data index to 0 for both input and output. We use the same hyperparameter as above
and we report the loss difference of data after ablating data i at each time, A6j, Eg f,
Di:
C.4 CLASSICAL INFLUENCE FUNCTION
We measure classical influence over the training trajectory of the toy model
IF, (2;, Φ) -V -w, ((n,)) Н H- V... w. (((w:))
(16)
19

---

Published as a conference paper at ICLR 2026
1M
0.01
100k
Gamma
Epoilon
100p
100
19
100
10
100
1000
10
100
1000
10k
Epsilon
Beta
Peason Corr.
0.2
0.4
0.6
Figure 6: BIF-LOO trace correlation with varying inverse temperature 39 localization strength and step size On single grid point between two hyperparameters (e.g. in the leftmost), multiple points refer to different values of the remaining hyperparameter (8 in the leftmost). In general, high and low gave the highest correlation with the gold standard LOO experiment, and was less significant in the toy model.
BIF
pine
oak
rose
lily
0.0002
0.0001
query ]
pine
0.0000
oak
-0.0001
rose
lily
0
10k 0
10k 0
5k Epoche cat
10k
5k
10k
sparrow
Epoche sparrow
Epoche penguin
Epoche
penguin
dog
0.0002
cat
dog
0.0001
0.0000
-0.0001
10k
10k
Epochs
Epochs
Epochs
Epocha
Figure BIF for perturbing each data point. The subtitle indicates the perturbed data point.
at time 6. Note that we do not the global minimum w- but the minimum at that specific measurement time t. Since the Hessian H the overparameterized neural network is non-invertible due to multiple zero near-zero eigenvalues, take the approximation methods as in Koh & Liang (2020): Grosse et al. 2003). First to add a constant dampening term y to the Hessian,
-
(17)
Suitable r enforces H have positive eigenvalues. Grosse et al. 2023): Martens Grosse 2015
Bae et al.) (2022) used damped Seeis-Nettonn--esee (GNH), approximation to H which lincarizes the network's parameter-ougpe mapping around the current parameters,
G = E HJ
(18)
H G
(19)
Using approximated damped-Hessee Hessian (H) and damped GNH (H), we measure classical influence function over time (Eq. 16) with varying dampening constant n
20

---

Published as a conference paper at ICLR 2026
Leave-One-Our Loss Difference
pine
oak
rose
lily
0.25
query J
pine
-0.25
oak
-0.50
rose
-0.75
lily
5k Epochs penguin
10k
5k Epochs cat
10k
5k
10k
sparrow
Epochs sparrow
Epochs
penguin
dog
cat
0.25
dog
0.00
-0.25
-0.50
-0.75
5k Epochs
10k
0
5k
10k
0
5k
10k
0
Sk
10k
Figure Leave-One-Ol (LO0) loss difference for perturbing each data point. The title indicates the ablated data point.
The dampening constant >" is scaled with the maximum absolute value of all eigenvalues of the H
ya,
(20)
and we vary y in range of [1e-3, 10] logarithmic scale.
Damped Hessian
A:10
A:1
A:0.1
0.0005
0.00
0.0000
-10
-20
-0.0010
-0.04
-30
2k
A.0.01
A:1.001
A:0.0001
De
2k
8k Epochs
7k
10k
2k
6k Epochs
7x
10k
2k
5k
7k
10k
Epochs
Figure 9: Influence with damped Hessian approximation. We measure the classical influence of the dog sample with damped Hessian approximation H with varying dampening constant y.
C.5 TIME-SPECIFIC ABLATION EXPERIMENT
Our observation that influence is a dependent function proposes another perspective perturbing the same data point but at different training times would lead to non-identical influence. That is, the exact moment the network interacts with each data point matters. Here we probe this hypothesis with simple retraining experiment in the toy model. We hypothesize that the measured influence
21

---

Published as a conference paper at ICLR 2026
Damped Gauss- Hause-Newton-Hessian
A:10
A-1
A/0.1
0.001
0.075
0.005
0.000
-0.00
0.005
0.025
-0.010
0.000
5k
A.0.001
A:0.0001
0.9
80
0.4
10
0.0
9K
8k
Sk
Epochts
Epoctos
Ecochos
Figure 10: Influence with damped Gauss-Newton-eesian approximation. We measure the classical influence of the dog sample with damped Gauss-Newter -eessan approximation H with varying dampening constant ?..
trajectory tells us which time point would be most critical for a model to learning each sample. For example, the influence of dog on cat makes peak around 6000, which implies ablating dog around that time would be most helpful in learning cat. We probe this hypothesis with a retraining experiment with brief ablation at different stages of learning.
In Figure I1 we ablate a data point dog during the duration of epochs from t and show the integrated loss difference compared to the baseline (no ablation) during D. As expected, the loss difference is the biggest around = 6000, when the highest influence occurs, indicating ablating dog helpful the most this stage. Similarly, difference around & = 3000, which also matches the negative minimum in the influence measurement, implying that learning of the cat was the most detrimental this time point when dog ablated. We show that the most important stage changes with the query data that belongs to a different hierarchy. e.g.. sparrow. Collectively, we strengthen our claim that influence over time shows us how sample interacts with "what" data "when", and that it is often correlated with the structure of the data, such as hierarchy.
observe the negative peak of integrated loss
C.6 ANALYTICAL INVESTIGATION OF TOY MODEL
In this section, analytically derive the dynamics of the influence function in deep linear neural network. First, we introduce deep linear neural network learning dynamics studied in [aax et al. (2013) 2019b) and then we show the time dependency of the influence function and its relation to progressive learning of the hierarchical structure.
DEEP LINEAR NEURAL NETWORK LEARNING DYNAMICS
Following the singular modes learning formulation Saxe al. (201 2013), we can describe each data point i's loss into the singular modes basis. We assume whitened input X E NxD XTX and output Y E xo The input-output correlation and its singular value decomposition (SVD) become
C Eyz YT X SV D(E USVT
(21)
(22)
With squared error loss, individual data point at data i is l 2 1.1. We project data to V space (object analyzer) and its component becomes 6/k U/ x;. where un is kth column of V. Similarly, project 7/kk ur y: and with rank rank
22

---

Published as a conference paper at ICLR 2026
0.05
0.06
cat
sparrow
Integrated A
0.03
0.02
0.02
0.01
-0.01
-0.04
-0.02
2k
4k
6k 10k 0
2k 6k
8k 10k
Ablation
Ablation t
Figure 11: Brief ablation of dog. For each experiment, we ablate dog for duration
100 epochs
starting from timepoint f. Starting time t is sampled uniformly with an interval of 200 epochs. For each experiment, we integrate the loss difference to the baseline during D. We show the integrated loss of query (a) cat and (b) sparrow over different ablation windows. The most critical period of influence from the dog on the two samples different. The higher level hierarchy (animals) sharing sample sparrow has earlier critical point than the lower level hierarchy (mammals) sharing sample eat.
Thi - gkEck) 4 2
(23)
Ok is a singular mode strength of mode is of the network obtained from
(tt) (t) = UG
(24)
assuming the network and the data share the aligned right and left singular vectors U, V. This assumption holds with a small initialization and small learning rate.
Now, we can compute the loss gradient with respect to the parameter space and decoupled mode space. In parameter space,
VLw, = WT (Eyz - W2W V Vw, (Eynr W2T We can substitute the terms in singular decomposed form,
(25)
(26)
VLw, USS G)A,
(27)
VLw, -
(28)
W RAVT W. = UARR A G.
(29)
The effective singular values from each layer W122 are A1 and R which eventually aligns the effective weights' singular vectors that of the data correlation Eyz. In general, we cannot assume @1 02k throughout learning, but the equality holds with the strictly balanced initialization. With the small initialization assumption, the first-order approximation of equality holds, which eventually allows us to write down the effective singular value trajectory throughout learning as in Saxe al. (0019a).
Importantly, G(() is time-narying variable, evolving throughout the training that could be written in closed form with 85 being the corresponding diagonal singular value of from data covariance,
20xt/
GEL(!)) = 20sl//T + sk/Gu,
Gb
G = 0),
(30)
23

---

Published as a conference paper at ICLR 2026
and the model that learned the training data recovers
W UG.
(31)
For convenience, we will drop the notation of time dependence in the rest of this section. For an end-to-end linear map W W2 Rx define its representation in the data singular basis and the singular mode strength gk.
== U WV,
Ok = GRee = ug Wok-
(32)
For data point p (Tps Up), write the coordinates in the singular basis
Tpp V Tp ER"
1/p U Up ER'
(33)
INFLUENCE FUNCTION IN Toy MODEL USING SINGLE DATA POINT PERTURBATION
celtueeneefuncton,,, function, that is, the change model prediction with targeted perturbation of training data. In the following, we will derive the response of original U, V and S as a function of a single data point perturbation to derive the influence function singular basis based on the derivation and results from Stewart Sun (1990) on the first-order SVD perturbation. With those in hand, we can use the closed-fore dynamics on Eq. to to get time-dependent dynamics.
Following Cook & Weisberg 1982) Koh & Liang 2020), we will derive influence
First, we consider perturbation by upweighting single data point p with factor e. The perturbed cross-covariance becomes
Cp(e) (p(()))())V(())) = + = C C
(34)
Pzf
Let C' be the first-order derivative of the data covariance with e-upweighting of a single data point p.
@Cp(e)
C₁ =
(35)
Oe
With the perturbed dataset, the learned 2-layer linear network mapping time becomes
Wp(e,t) t) (,,,)))=((()),,(,()))) (E), GS, (i)
(36)
(37)
where s a singular value from diagonals of S and gr (.) follows a closed form dynamics as given in Eq. 30 We assume that the network will have aligned singular basis with the perturbed data
covariance, given in (E) and (E). For convenience, we will move the parameter to a subscript and drop the perturbed data index p.
We apply product rule to C²
= US + +USV + +US...
(38)
We project the above perturbation to the singular mode coordinates,
Qc --=VVV-- T (U.SVT
+USV + +USV
(39)
Since U, V are orthonormal matrices,
U V
(40)
and differentiating these orthonormality constraints with respect to the perturbation e
U
(UT))
c U U.UUUUT 0.
(41)
0=
(rT))
v
- V.' + TV 0.
(42)
ds
Due to orthonormality, U,, V lie in the tangent space of U, VT. We can define
A:UTU. B V VV.
(43)
24

---

Published as a conference paper at ICLR 2026
where the orthonormal constraints give
A A 0, B B² = =0,
(44)
and A, become skew-symmetric, A - B = B= Intuitively, A, is equivalent to the in-subspace angular velocity (rotation rate) of the columns of U and V due to the perturbation, respectively.
With this, rewrite Eq.
39
Q Q= = S SB,
(45)
where diag(A) A) = diag B) since A, B are both skew symmetric.
Now, solve Eq. 45 for A, B and S.. First, consider
where the singular values diag(S) are non-degenerate. Due to the diagonal constraint of singular values, the response from the perturbation should also be diagonal for the singular values
-2 Q Ei U, C.
(46)
The response of the singular basis is obtained from Eq.
43
= V BVT,
(47)
where
8j Qij
8 Q
i
Aij =
(48)
0
i i=
8 Qij
8
Q
i i
Bi
5
(49)
0
i i=
When S has degenerate singular values or a near-zero gap, above ill-defined as the denominator o² - - = O. In this case, we apply block perturbation, treating the whole invariant subspace (sharing the degenerate singular values) all at once.
We take the sets of the degenerate singular values {sa sy Vsy E diag(S)), define the block coupling by projecting the perturbation C' on this block.
Q: :
(50)
where U and V T are sets of the degenerate singular values corresponding to sets of columns of U and V. can be further decomposed into symmetric and symmetric-skeww skewed part.
M. = ] Q. + Q.).. K. = - (Q: - QT), M. + K. QQ. 2
(51)
Due to diagonality constraints of singular values and zero-diagonals in the skew-symmetric part Kg, the first-order singular value consistency reduces to M, S To have diagonal singular value matrix St, we choose R such that
M S R AR.
(52)
where (A) becomes the first-order splits within the degenerate block with preferred direction defined by R. Skewed part K2 the in-block angular velocity,
1
A= =
K
(53)
28b
Then,
S = Aa, Ü-ÜÁ V = VB.
(54)
25

---

Published as a conference paper at ICLR 2026
Now equipped with U. V S/ described by original we define influence (t, from the perturbation on data point at fixed time by applying derivative on Eq. 36
Ow
""t = 0E = U' + LUG(S + UG(S,
(55)
UAG(S, (ddiag(g((s)) diag(S)))) ())) +UG(S
(56)
We can further derive influence on measurable 2, squared loss 2|10:11 with residual of measured data point Φs using the chain rule,
(2z,, 21) = O aw Ōs
@w xx) = o, (AG, G. G,B)V Jin ob ( zi)
(57)
where 2p is the perturbed data point. This perturbation is expected to hold with a small perturbation Ro O. We nown-weighting -0.1 to approximate the ablation effect in Figure
Analytic IF
pine
oak
rose
lily
0.10
query j
0.05
pine
oak
rose
lily
0
5k Epochs sparrow
10k
5k Epochs
10k
5k Epochs
10k
5k
10k
sparrow
Epochs
penguin
penguin
dog
cat
dog
0.10
0.05
0.00
-0.05
10k
10k
10k
Epochs
Epochh
Epochs
Epochs
Figure 12: Analytical loss difference from perturbing each data point. The title indicates the perturbed data point -0.1).
D LANGUAGE MODELS
This section discusses additional experimental details for the language model experiments.
D. STRUCTURAL TOKEN CLASSIFICATION
Here, we present additional details and further experiments conducted investigate the development of influence with respect to how tokens structure text. These experiments were conducted using the 14 million parameter Pythia model. Tokens are generated using the same tokenizer as the Pythia models use in order to be able to use these tokens with the model suite.
D.1.1 TOKEN CLASSES
Tokens classified based their role in structuring text. The classes used are based on the classes used by Baker et al. (2025). Figure [3] demonstraasss these classes graphically, with tokens outlined in bold indicating class membership, and distinct colors representing distinct classes.
26

---

Published as a conference paper at ICLR 2026
Word Part
Word End
Formatting
Wa
vel
ength
vel ength
####
differe
nces
differe
nces
***
n
Induction Pattern
Delimiters
B
A
B
(
Figure 13: Examples structural classification of tokens. Bold outline indicates class membership.
The classes as follows:
Word Part Tokens that constitute part of word that is not the end of the word. This class excludes tokens that represent entire words or entire words with the addition of additional characters (e.g. space and entire word).
Word End Tokens that constitute the ending that represent entire words and entire words with the addition of additional characters.
multi-token word. This class also excludes tokens
Formatting Tokens that generally used to format text including newlines and repeated characters.
Induction Pattern Tokens that appear context in which the token preceded by another particular token, and the same token has been preceded by the same other token earlier in the context.
Left Delimiters Tokens that are generally followed by an equivalent right delimiter with interceding text considered occupy distinct scope.
Right Delimiters Tokens that close the scope opened by paired left delimiter.
D.1.2 RMSPROPSGLD HYPERPARAMETERS
Table 2: Summary of hyperparameter grid search for BIF experiments on Pythia 14M.
Hyperparameter
Range
n (number of sequences)
600
J (sequence length)
55
(step size)
le 6
n3 (inverse temperature)
{256, 1024]
? (localization strength)
(500, 1000]
(batch size)
64
C' (number chains)
4
T (chain length)
200
burn in steps
0
k (nearest neighbors)
30
We found that the overall dynamics expressed by the BIF did not vary substantially among hyper- parameters chosen in this setting. Plots the main paper were selected based on the parameters that optimized the average class recall the KNN experiment which is discussed in section Ap- pendix D.1.3
D.1.3 SAME-CLASS PREDICTION WITH BIF K-NEAREST NEIGHBORS
One way investigate the ability of the BIF to capture what sorts of text structure model has learned is to use to predict which structural class a token belongs to based on the classes that token shares high influence with.
27

---

Published as a conference paper at ICLR 2026
We query the predictive capacity of token-level influences using simple nearest-neighhors approach. The predictive model as follows:
yi(mi) = argmax,
Σ In(z))
Ta (Tx)
symmy
where 1, is the indicator function for elements of class a, and ra is the token-dependent class frequency in the dataset:
Ta (z) Exd (x))
(x)) (1)e(((")))
with { denoting the set of all tokens that are not Tokens are thus predicted to have the majority label of their corresponding top-influence set m, adjusted for class rates overall.
BIF KNN Same-class Recall Pythia 14m
Induction
0.6
Left Delimiter
Right Delimber
Rocalt- Rate
Formatiing
0.4
Word End
0.2
10k
30k
40k
Checkpoint
Figure 14: KNN BIF predictions change over time. Using the BIF in order select the top 30 highest influence tokens and taking the majority class from among them (adjusted for class rates), we plot the recall minus the class rate for each class. This means that represents the random baseline. We see that the BIF selects for in-class tokens above random for all categories by the end of training, indicating that sensitive the class structure learned by the model.
Figure 14 shows the results of this experiment. We observe that the classes of high-influence other tokens act as a better than random predictor for all classes by the end of training, with recall generally improving as the model progresses through training.
D.2 PER-TOKEN INFLUENCE DYNAMICS
We presented a sparse-grainet amalysis of aggregated influence between different classes of syntactic tokens in F Here, additionally measured intra-class BIFs training phases capture more fine-grained influences between token classes. We focus on right and left delimiters classes, where we can make finer categorization of tokens into parentheses, curly brackets, square brackets, and angle brackets. In other words, we make finer categories of delimiters. including types, not only left and right directionality. In total, there are classes (2 directions times 4 types). We exclude tokens that have more than one different token types (e.g.. } ). We measure influence dynamics on Pythia- 1 14m model throughout the training for each reference token type. The hyperparameters for BIF measurement are kept the same as in Figure 5
we observe that different subgroups of tokens impose different magnitudes and signs of influence, which again can change over time. We see few interpretable features. For example, we see that the same subtypes of brackets can assist learning as training goes by, even if the directionality different (observed for curly brackets, square brackets, and angle brackets but not for parentheses), which might be due to the paired nature of the bracket signs.
In Figure L5
D.3 INFLUENCE DYNAMICS OF INDUCTION HEAD FORMATION
Our initial experiments with the public Pythia checkpoints (Bderman et al. 2023) suggest a signal for the learning of induction patterns. However, the checkpoint frequency too coarse in the vicinity
28

---

Published as a conference paper at ICLR 2026
1gtt panatibnt
un parrtion
0.22
0.50
4000
-0.25
Crecipaatt
Checkpore
19t oaly tecchet
cutly
-015
0.3
-0.15
-0.00
-0.00
Chadipolt
Cherkpooe
01
03
-24
-24
-03
-05
-24
Cheapsed
Checkpoie
0.05
-235
11'
10
10'
Chickgoet
right parenthesis left parenthesis
right curly bracket curly bracket
square bracket left bracket
rilht point angle bracket bracket
Figure 15: Intra-classes (delimiters subtype) BIFs in PYthia-14M. We plot BIFs between more fine-grained syntactic token classes of delimiters.
of checkpoints 1K to 10k to observe the fine-grained dynamics this developmental transition. To investigate this critical phase in more detail, we trained Pythia-14M model from scratch, saving checkpoints every 100 steps for the first 20,000 training steps. Though we train on the same Pile data (Gao et al. 2021). it is in a different order and on different hardware, which means this training run is not the same as the original Pythia-144 14M training run.
We constructed synthetic dataset with sequences where the second half repeats the first, e.g., Sequence 1: A B... and Sequence 2: C D... D. Sequences were constructed to not share tokens. The first half of each sequence is made up of random tokens, and so the only structure that be used for prediction is the repetition of the second half of the context.
29

---

Published as a conference paper at ICLR 2026
During SGLD collect losses this dataset, but sample using the Pile, meaning that loss these synthetic samples does not affect our sampling trajectory. We compute the BIF matrices at each checkpoint of our homemade Pythia- 14M and then look at the mean correlation between the different parts of our sequence (Namely repeated tokens with repeated tokens from other sequences, non-repeated tokens with non-repeated tokens from other sequences, and non-repeated tokens with repeated tokens from other sequences).
The results shown Figure 16 The influence between tokens in the repeated segments of each sequence (top panel, blue line) undergoes sudden, large increase, peaking and then stabilizing. Meanwhile, the BIF between non-repeated segments across non-repeated and repeated segments shows no such change.
Simultancously, measure the "induction score" of the model's attention heads-a standard metric from mechanistic interpretability that quantifies how strongly a head implements the induction algorithm (Olsson al., 2022). As shown in the bottom panel, the induction score for the heads that become the "induction heads" begins to rise right before the BIF increases between the repeated groups
MEAN CORRELATION BETNEEN TOKEN TYPES
Repeated :1 Repeated
ABC... ABC..
Randon 11 Repeated
Random :1 Random
3... 123
Posterior Correlation
A B
ABC... ...
2
Sequence
Sequence
Repee
Repeatted
Repeeteed th
Repestee
Sequence A
Taken Token B7F
1.0
LAYER ATTENTION HEADS
Fully Forned
Induction Score
Induction Heads Other Heads
10'
10
Training Step
Figure 16: Influence dynamics during induction head formation. We trained small transformer from scratch with feegufrry checkpointing to capture the formation of induction heads. (Top) The normalized BIF between corresponding tokens repeated sequences (blue) shows sharp increase and peak, while correlations between non-repeated samples (red) or between non-repeated and repeated segments (green) remain low. (Bottom) The induction for the attention heads that become induction heads rises together with the posterior correlation, while other heads remain inactive.
D.4 INTERVENING INDUCTION HEAD FORMATION
To validate the causal implications of stagewise influence, conducted intervention experiment targeting the acquisition of induction OOlsson et al. 2022). Following the results from the previous section that show that the influence of induction patterns predict that upweighting the same subset data should have different effects on model development depending on when that upweighting occurs. In particular, expect upweighting induction samples
vary significantly over training,
30

---

Published as a conference paper at ICLR 2026
will lead to faster stronger induction head formation if these samples upweighted during periods when the average influence between tokens representing an induction pattern is more strongly negative, indicating that losses across induction tokens are correlated and induction is being learned
Setup. We trained a 3-million parameter transformer on a dataset consisting of 13 million sequences from The Pile for a total of 15,000 steps. To isolate the effect of timing, we created five experimental runs where we upweighted the loss for tokens belonging to an induction pattern (as defined in Appendix D.I.T) by a factor of during specific 500-step windows: 0-500, 1k-1.5k, 2k-2.5k, 3k- 3.5k, 4k-4.5k, and 5k-5.5k. We compared these against control model trained with no upweighted loss. All models shared identical initialization and data ordering; thus, deviations in dynamies are strictly attributable to the timing of the intervention.
We track the formation of the induction circuit using two methods: the induction score metric described in Appendix [.3] and the average BIF between induction tokens (excluding same-token pairs) on the unweighted baseline model for the first 6k training steps. We compare the induction scores for each model's highest-scoring head in Figure 17 and evaluate how induction learning dynamics are sensitive to the timing of BIF learning.
Results. We find that the BIF between induction tokens is highly informative of how the timing of the upweighting window alters the developmental trajectory of the induction circuit. This confirms that the influence function can be used measure stagewise data sensitivity during model training. In the control setting, the induction circuit emerges gradually, with the induction score peaking relatively late in the training process at timestep 9k. Correspondingly, we observe that the BIF between induction tokens (on the synthetic dataset from the preceding section) is generally decreasing during this regime, indicating increasingly correlated learning on induction tokens. The BIF decreases relatively gradually for the first 2k tokens after which we observe more rapid decline. Upweighting induction tokens during early training stages while the BIF is more positive (indicating interference) leads to delayed learning or completely disrupts learning altogether. There is intermediate point where upweighting in the 2k-2.5k window leads to a similar learning dynamic to the baseline, though the induction score remains higher for longer after peaking. Intervening during stages beyond this point, when the BIF is decreasing rapidly, leads to accelerated learning, with induction scores rapidly spiking during the intervention window for the 4k-4.5k and 5k-5.5k interventions.
These results provide causal evidence for our stagewise framework. Upweighting a data pattern outside the phase where the model receptive to it can prevent learning or yield developmental delays, whereas upweighting it inside the critical window can accelerate structure formation. We find that the BIF can be used to measure these dynamics, with a more positive BIF indicating that learning will be delayed or disrupted while more negative BIF indicates faster and stronger effects of token weighting.
31

---

Published as a conference paper at ICLR 2026
Layer 1 Max Induction Score
0.15
0.1
0.05
2k
4k
6k
8k
10k
12k
14k
Step
BIF (No Reweightingl
-0.004
Upweighting window
No reweighting
2000-2500
5000-5500
-0.005
0.500
3000-3500
-0.006
1000-1500
4000-4500
Induction BIF
-0.007-
Average BIF between induction tokens
Windowed linear
2000
4000
6000
Step
Figure 17: Stagewise intervention accelerates induction head formation. A comparison of the maximum induction score across Layer 1 heads for each intervention window (top). The baseline model (top, green) naturally forms induction heads at step 7k Upweighting induction data during the critical developmental windows, indicated by red lines (top) and windows (top & bottom), significantly accelerates this capability, causing heads to form as early as step 5k. In contrast, interventions applied too early fail to produce stable induction heads or delay induction learning, see blue lines (top) and windows (top & bottom). There an intermediate window where there is little effect of upweighting on learning dynamics (gray). Comparing the timing of these windows to the BIF of the baseline model (green, bottom), we observe that decreasing BIF indicates faster learning, as predicted. The shaded region is a 90% CI of the 100-sample bootttoott from the approximate posterior estimated via Algorithm
32


Reviews:
[{"review_id": "qbJZhDQgrY", "reviewer": "Reviewer_Z984", "summary": "This paper challenges the static view of data attribution, arguing that deep networks' stagewise learning makes fixed influence scores capturing only part of the picture. The authors propose a dynamic \"stagewise\" framework using the Bayesian Influence Function, which predicts that a data point's influence can change over the training process. This theory is validated in both toy linear networks and at-scale Pythia language models, where influence dynamics directly correlate with known developmental milestones, such as the formation of induction heads. The core conclusion is that the timing of measurement (during training) impacts the measured influence.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "* Novel and Significant Conceptual Contribution: The paper's primary strength is the novel connection it forges between Singular Learning Theory and Training Data Attribution. This challenges a foundational assumption in TDA and provides a compelling, theory-backed explanation for why static influence measures are insufficient for deep learning.\n* Strong Theoretical Grounding: The paper does an excellent job motivating its theoretical choices. It clearly explains the failure of classical IFs (reliance on an invertible Hessian) and the suitability of the BIF (Hessian-free, well-defined on degenerate landscapes). The derivation in Section 2.3, which uses the Law of Total Covariance to predict influence peaks at transitions, is elegant and provides a clear, falsifiable prediction.\n* Attempt to Demonstration at Scale: The authors successfully bridge this gap by attempting to demonstrating their predicted phenomena in Pythia LMs, beyond the toy setting. While these results are correlational (linking influence dynamics to known developmental milestones like induction head formation), they are a crucial step in showing this framework is relevant for models we care about.", "weaknesses": "* Discrepancy in Observed Dynamics between Toy and Large-Scale Models: A significant discrepancy arises between the compelling theoretical predictions validated in the toy model and the empirical results from the large-scale language model experiments. The core theoretical argument hinges on influence being a highly dynamic quantity, subject to non-monotonic changes and sign-flips (as clearly demonstrated in Fig. 3). However, the influence dynamics observed in the Pythia models (Fig. 5), while temporally staged, appear to be largely monotonic once they become non-zero.\n\nThis observation materially weakens the paper's central critique of static, endpoint-based attribution or simpler trajectory-aggregation methods (e.g., TracIn). If the influence of key data groups simply increases monotonically after a specific developmental stage, then traditional attribution methods applied at or aggregated near the end of training would likely still provide a reasonable approximation of data importance. The practical necessity of the authors' far more complex, dynamically-sampled framework is therefore less evident in the very setting (LLMs) it purports to be essential for.\n\n* Lack of Demonstrated Practical Utility or Actionable Insights: The paper compellingly argues that when a data point exerts its influence is a critical, and previously overlooked, dimension of attribution. However, the analysis remains at an observational level, and the practical utility of this framework is not demonstrated. It is unclear how these findings could inform model development, for instance, to train more capable or efficient models.\n\nA powerful validation of the framework's utility would be to move from this observational analysis to an interventional one. For example, could the authors leverage their findings to design a dynamic data curriculum? If, as the results suggest, the influence of \"induction pattern\" tokens becomes active only during a specific training phase, could dynamically up-weighting these examples during (or just before) that critical phase lead to demonstrably better model performance on induction-related tasks or faster convergence? Without such a demonstration connecting stagewise influence to actionable training strategies, the proposed method, while theoretically interesting, risks being perceived as a diagnostic tool that lacks a clear feedback loop into practical model improvement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AieO46ly7J", "reviewer": "Reviewer_LhWa", "summary": "This paper argues for a position that training data attribution should be done dynamically, rather than statically at the end of the training. Inspired by the singular learning theory, the paper predicts a non-monotonic influence trajectories with sign flips and sharp peaks. The paper then simulates several scenarios to demonstrate their prediction.", "questions": "1. As the authors also brought up in the discussion section, there is a class of unrolling-based attribution methods. Can the authors propose some further discussions about these methods? Does the sign flip somehow imply that these methods are inaccurate, since the middle stage influences are canceling each other?\n2. The current motivation/theoretical analysis is conducted solely based on BIF, which is a relatively new paper on this field. I wonder can the authors provide motivation in a more multifaceted manner?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The stage-wise attribution problem raised by the authors is interesting and well-presented. I am convinced that the same data point might be influential to other data in different ways at different stages of the training.", "weaknesses": "While the stage-wise framing is compelling, simply listing the most influential samples is of limited use. To establish real value, attribution should enable actionable interventions that beat a naive one-shot final stage attribution baseline. I strongly suggest that the authors please dedicate more space to what stage-wise attribution concretely enables. As of the current draft, the support is largely heuristic; adding such experiments would substantially strengthen the practical case.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cy1R8kyctr", "reviewer": "Reviewer_CzFc", "summary": "The paper introduces a stage-wise influence framework to track how data influence evolves throughout training. It predicts non-monotonic behavior and validates these in both a toy model and large language models (LLMs).", "questions": "1. Figure 3: BIF and analytical IF resemble LOO but peak at different times. Why don’t the maxima align?\n2. Figure 7: The influence curve between “Dog” and all plant classes looks identical; the influence curve of “lily” and all animal classes mirrors that shape. Is this expected symmetry or some artifacts? \n3. Pointer: The text says the RMSProp-preconditioned SGLD sampler is “introduced in Section 2.2,” but the *details* are in Appendix B.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It introduces a novel framework of *stage-wise influence*, enriching the understanding of data attribution beyond static snapshots.\n2. Makes clear predictions and uses Bayesian Influence Functions to measure influence over time, confirming them in a toy model and in LLMs.", "weaknesses": "1. Empirical verification depends on BIF and SGLD:\nValidation relies on BIF, which requires sampling a local posterior via RMSProp-SGLD around each checkpoint, while most models are trained with Adam-family optimizers, raising questions about stage-wise behavior under Adam. \n    \n2. Computational overhead:\nBIF uses Monte Carlo sampling, which is costly; complexity/scalability is not thoroughly analyzed, so practicality at scale remains uncertain.\n    \n3. Limited experimental scope:\nExperiments emphasize inter-class patterns (to align with Sec. 2.3), but broader claims about *when* data are influential and *how* they shape internals would benefit from intra-class analyses and more mechanistic evidence", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)