Small Transformers Don’t Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and Implications for Mechanistic Interpretability
Authors: Luca Baroni
Galvin Khara
Joachim Schaeffer
Marat Subkhankulov
Stefan Heimersheim
Venue: ICLR 2026 Poster (2026)
Keywords: mechanistic interpretability
language models
Primary area: interpretability and explainable AI
TL;DR: We remove LayerNorm from all GPT-2 models and test whether this enables easier interpretability.

Abstract: Layer-wise normalization (LN) is an essential component of virtually all transformer-based large language models. While its effects on training stability are well documented, its role at inference time is poorly understood. Additionally, LN layers hinder mechanistic interpretability by introducing additional nonlinearities and increasing the interconnectedness of individual model components. Here, we show that all LN layers can be removed via fine-tuning from every GPT-2 model with only a small increase in validation loss (e.g. +0.03 cross-entropy loss for GPT-2 XL). Thus, LN is not essential at inference to maintain comparable performance in these models. We find that the amount of fine-tuning data needed for LN removal grows sublinearly with model parameters, suggesting scaling to larger models is feasible. We release a suite of LN-free GPT-2 models on Hugging Face. Furthermore, we test interpretability techniques on LN-free models. Direct logit attribution now gives the exact direct effect of individual components, while the accuracy of attribution patching does not significantly improve. We also confirm that GPT-2's  "confidence neurons" are inactive in the LN-free models. Our work clarifies the role of LN layers in language modeling, showing that GPT-2-class models can function without LN layers. We hope that our LN-free analogs of the GPT-2 family of models will enable more precise interpretability research and improve our understanding of language models.

Published as a conference paper at ICLR 2026
SMALL TRANSFORMERS DON'T NEED LAYERNORM AT INFERENCE TIME: SCALING LAYERNORM REMOVAL TO GPT-2 XL AND IMPLICATIONS FOR MECHANISTIC INTERPRETABILITY
Luca Baroni
Galvin Khara" Imperial College London
Joachim Schaeffer" TU Darmstadt MIT joachim. Ber
Charles University
baroni8kevi. mLT. ouni
gk2513eic.ac.uk
Marat Subkhankuloy* Independent
Stefan Heimersheim
Apollo Research
ataffn@@pollloreseocc..oo
ABSTRACT
Layer-wise normalization (LN) is an essential component of virtually all transformer-bused large language models. While its effects training stability are well documented, its role at inference time is poorly understood. Additionally, LN layers hinder mechanistic interpretability by introducing additional nonlinearities and increasing the interconnectedness of individual model components. Here, we show that all LN layers be removed via fine-tuning from every GPT-2 model with only a small increase in validation loss (e.g. +0.03 cross-entropy loss for essential inference maintain comparable perfor-
GPT-2 XL). Thus, LN mance in these models. We find that the amount of fine-tuning data needed for LN removal grows sublinearly with model parameters, suggesting scaling to larger models is feasible. We release suite of LN-free GPT-2 models on Hugging Face. Furthermore, we test interpretability techniques on LN-free models. Direct logit attribution now gives the exact direct effect of individual components, while the accuracy of attribution patching does not significantly improve. We also confirm that GPT-2's "confidence neurons" are inactive in the LN-free models. Our work clarifies the role of LN layers in language modeling, showing that GPT-2-class models function without LN layers. We hope that LN-frree analogs of the GPT-2 family of models will enable more precise interpretability research and improve our understanding of language models [
INTRODUCTION
Large language models (LLMs) have seen widespread adoption in recent years (Touvron et al. 2023 OpenAi et al. 2024, Gemini Team al. (2024), most of which are based on the Transformer architecture (Vaswani et al. 2017). A key component of virtually all such LLMs are layer-wise normalization (LNN layers, often implemented via LayerNorm (Ba et al. 2016)
LN(x) = 1 -
The
1 H
Σ
(In 1)2 +
(I)
where ④ denotes the Hadamard (element-wisee product. common alternative RMSNorm (Zhang Sennrich 2019), which performs an equivalent computation, without removing the component in
'These authors contributed equally
A precursor of this work has been presented the Interpretable Al: Past, Present and Future workshop NeurTPS 2024, under the title "You remove GPT2's LayerNorm by fine-tuning" (Heimersheim 2024

---

Published as a conference paper at ICLR 2026
4.38
GPT-2 original OpenWab Text eval
Small
removal startlend
3.53
3.10
3.39
Medium
3.18
2.98
2.81
3.37
Large
3.13
2.89
2.68
2.77
2.67
2.56
2.46
700
Step
Figure Main training loss curves for all GPT-2 variants during LN removal. Original GPT-2 Open WebText eval losses are shown for reference. Curves terminate at model suite checkpoints. LN removal period shown as vertical lines.
the 11, 1) direction (see Appendix D for more in depth comparison). These layers have been introduced stabilize the training process al. 2016), similar batch normalization (Ioffe & Szegedy 2015) in other network architectures.
Unlike batch normalization however, LN layers cannot replaced with a linear transformation at inference time. While the mean centering (μ)) weight (y), and bias (3) parameters can be folded into neighboring layers (e.g. fold_ Nanda & Bloom 2022). the non-linear division by the norm or standard deviation of the residual stream must be executed at inference time. This raises the question of what role LNs play the model and whether is necessary for the model function. Prior work has shown that LN functions implement complex non-linear functions in toy models (Winsor) 2022), and proposed that LNs might play a role in confidence regulation in LLMs (Stolfo et al. 2024).
Additionally, LN layers complicate mechanistic interpretability. Mechanistic interpretability typically aims to decompose the model into smaller components and to understand their individual effects and interactions. Both of these complicated by the non-linearity of LN layers. Individual components cannot be easily attributed as their effect on LN depends on the residual stream activations (nostalgebraist) 2020 Elhage al.) 2021 Wang al.) 2022b Nanda 2023b a). Interactions between components are also complicated by LN because it causes each component to affect almost every downstream component
the model (via the LN scale). This makes analyzing the interactions complex (e.g. [Bushnaq et al. 2024 Farnik et al. 2025). In practice researchers approximate the LN layers as linear transformations (referred to as "freezing LayerNorm", [Bricken et 2023
McDougal et al. 2023 Kissane et al. 2024), train models without LN layers (Elhage et al. 2021
Nabeshima 2024).
In this work we show that LN layers can be removed from transformer models at the end of training. We replace the LN layers with a linear transformation that is initialized to be close to the original LN transformation, and fine-tune the model on small fraction of its training data. We do this for one LN layer at a time, essentially slowly weaning the model off LN. This (a) shows that LLMs function without LN layers, and (b) provides LN-free versions of the GPT-2 family of models. These models can be studied on their own, simply to understand any large language model, or as a proxy for their corresponding original GPT-2 models. The latter is possible have similar internals, but should be used with caution as similarity is not exact.
fine-tuned models
Our contribution is threefold:
2

---

Published as a conference paper at ICLR 2026
We show that LLMs can function without LN layers, achieving a cross-entropy loss compa- rable to the original models.
We provide an optimized protocol for removing LN layers from LLMs at the end of training during fine-tuning, and provide suite of L--ffree GPT-2 models Hugging Face.
We validate that the interpretability of LN-free models is improved, finding that the direct logit attribution (DLAA) error reduced from 50% to 0%, and that attribution patching- contrary to expectations in the literature-does not improve with LN-free models.
RELATED WORK
Interpretability aims to understand the internals of neural networks and the algorithms they implement. Most mechanistic interpretability methods attempt to decompose a model into smaller components and aim to understand the interactions between those components. The most popular methods are based on sparse dictionary learning, such sparse autoencoders (Bricken et al. 2023) or cross-layer transcoders (Ameisen et al. 2025). In both cases, researchers attempt find -inteell-nnttaacting set of components that explain the model's behavior (Marks) it al. 2024 Lindsey et al. 2025). The most common approach to deal with LN is to approximate the
Mechanistic interpretability:
layer norm scale as constant (e.g. Bricken et 2023 McDougall et al. 2023 Kissane et al. 2024).
Other methods introduce special cases for LN layers (e.g. Bushnaq et al. 2024).
LN alternatives:
The main alternative to layer normalization is batch normalization (BN), However, BN performs worse than LN in language model transformers due to changes between the training and inference distributions (e.g.
Wang et al., 2022a).
Concurrent work (Zhu al. 2025) proposed Dynamic Tanh (DyT) as an alternative to normalization. Instead of an LN layer, they apply an element- wise tanh(az) function to the residual stream. This work confirms our results, finding that language models can work without LN. While DyT is preferable over I.N, in some use cases, DyT' is still non-linear function whose role we don't 't understand, and that affects interpretability. Our work goes further, replacing LN with a purely linear transformation.
Transformers trained without normalization: Finally, Nabeshimal (2024) trains toy language models from scratch, without normalization. However, we expect this method to work only for small language models, state-ofthhe-e language models continue being trained with normalization. Thus we focus removing LN from already-trained model.
LN REMOVAL STRATEGY AND METHODS
We remove the nonlinearity of LN by replacing the standard deviation in L by scalar, corresponding to an estimate of the average standard deviation, of aug. while fine-tuning on Open WebText. We define a FakeLN block as
x- + Jayg
1 H
FakeLN(xx)
Tb.s
Lahaah 1bb,s)2 + e, Cnvg
Tb,s
BS
where ob.s is the standard deviation across the model dimension for batch index b and sequence position s. and davg is the average across all tokens in a batch. Tag the fixed scalar value used when replacing LN with FakeLN. Because removing all LN blocks simultancously irreparably breaks the model's performance, we adopt a sequential removal process during fine-tuning: we remove one LN block, fine-tune for fixed number of steps to stabilize the loss (which typically spikes after each removal), and then proceed to the next LN block. Furthermore, Oavg can drift during fine-tuning. Therefore, to minimize the disruption introduced by LN removal and stabilize the fine-tuning process, we recompute Gayg for each batch and freeze the scale factor in FakeLN at the moment of removal to Jang aug. For the small and medium models, the batch size is significantly large enough to produce reliable estimates of awg. For GPT-2 Large and GPT-2 XL, we use an exponential moving average filter update aug for new batches. After LN removal, davg is not updated anymore.
3

---

find this Published as a conference paper at ICLR 2026
We categorize LN blocks into LN gk. LN,, LN and LN', where I indicates the layer number. Respectively, these LN blocks normalize imputs to the query/ey path, the value path, the MLP, and the final unembedding. While splitting LN for attention heads paths is uncommon, more fine- greiggr removal of LN improves stability during fine-tuning. Our sequential removal process begins after initial standard fine-tuning phase with the removal of LN MLP- followed by Gmip fine-tuning steps. We then remove LN MLP. fine-tune again for Gmlp steps and continue this pattern layer by layer until IN MLP is removed where 7. = Mayers- We then apply the same pattern to remove LN and LN; blocks, each separated by 9qk and g. fine-tuning steps, respectively. Finally, we remove LN The gaps between removal events are hyperparameters that have to be chosen carefully. Too small gaps can result in instabilities, while choosing very large gaps results in unnecessarily high computational costs. We provide a table with LN removal schedule and more details in Appendix B
We empirically found that beginning with L. rather than L. led more stable fine-tuning, likely because residual norm variance at beginning-of-sequeenee nism more strongly when its normalization is removed first. Despite removing LN blocks sequentially, instabilities can still occur during LN removal. To further stabilize LN-removal by fine-tuning, we used an additional auxiliary loss.
tokens affects the attention mecha-
Auxiliary Loss In models with LN, residual stream vectors scaled by their standard deviation When LN is removed, large norm disparities across positions can lead to gradient spikes and destabi- lize fine-tuning, To encourage stable activations during this process, we introduce an auxiliary loss that promotes consistent standard deviations across token positions:
1
Laux A EB,,, [[ob,s - ])
Σ
Øb.5t
M
(b,s)EM
where À is a scalar hyperparameter. While the loss itself is computed across all positions in the batch, the target & is the average standard deviation across the subset of token positions M, excluding the first token (position 0) and any positions containing the end-of-text token (ID 50256). These exclusions from the target target calculation are motivated by the observation that such positions consistently exhibit higher variance in GPT-2 models. We apply the auxiliary loss only at LN since all residual streams propagate through this final normalization layer, making it natural global target for norm regularization.
4 LN REMOVAL RESULTS
We successfully remove LN during fine-tuning on Open from GPT-2 Small, Medium, Large, and XI. (Tab. demonstrating that our sequential LN removal strategy with auxiliary loss scales from a 124 million parameter model to a 1.5 billion parameter model. Figure shows the main loss during fine-tuning for LN-removal. Notably, the amount of fine-tuning data required grows sublinearly with model size. For details of the sequential LN-removal schedule, hyperparameters. and fine-tuning data requirements, during the removal of LNMIP blocks, which is the first LN block that is removed. The LN and LN, block removals result only in small main loss spikes. Before introducing the auxiliary loss, the LN-removal fine-tuning loss curves were more spiky, suggesting that the auxiliary loss effectively absorbs some of the effects of LN removal. Furthermore, the auxiliary loss decreases quickly the beginning of tine-tuning, indicating that the model successfully learns to maintain consistent standard deviations across token positions.
Appendix
[ We find that the largest main loss spikes appear
As control, we compare the LN-free GPT-2 model suite to the original GPT-2 models and vanilla fine-tuned models. The vanilla fine-tuned models were fine-tuned for the same number of steps and with the same learning rate schedule as the LN-free models, but without auxiliary loss and without removing LN. This control allows us to disentangle the effects of LN from the effects of fine-tuning.
We evaluate performance using mean cross-entropy loss on validation set of WebTeet The Pile, and The Pile-filtered (Tab [] The Pile-filtered consists of sequences from The Pile dataset
filtered by removing sequences containing tokens that appear in The
After mcan-centering, i.c., removing the component in the (1, 1,..., 1) direction Gupta et al. 2025
4

---

Published as a conference paper at ICLR 2026
Pile but not in OpenWebText such control characters which arise from formatting discrepancies between the two datasets (see Appendix [ for more details).
We find that LN-free models perform comparably to their original variants, with performance gaps ranging from +0.03 to 0.1 cross-entropy loss difference on The Pile-filtered (Tab.[) This comparable performance extends to standard language understanding benchmarks, where LN-frree models maintain accuracy within 1-2 percentage points from their original variants (Appendix F)) The only notable exception GPT-2 XL LN-ffre, which shows degraded performance on The Pile. A closer examination of the distribution of losses reveals that the higher averaged CE loss is driven by very small number of samples and that the 99.9 percentile ranges of GPT-2 XL LN-free and GPT-2 original are nearly identical for The Pile, indicating that the vast majority of sequences are handled similarly by both models. This suggests that GPT-2 XI. LN-ffree is highly overconfident on a small number of sequences that are present in The Pile but absent from The Pile-filtered dataset.
We also investigate whether the performance gap can be closed by simply fine-tnnig LN-free models for longer. Contrary to our initial expectations, we find that extending fine-tuning does not reduce the loss gap to vanilla models. Instead, the gap remains approximately constant throughout fine-tuning, suggesting that LN contributes small but persistent performance benefit that cannot be compensated by additional fine-tune ainettunig. We discuss potential mechanisms behind this behavior in Section [5.4 We also investigate if our methodology generalizes beyond the GPT2 model family, and successfully apply our removal strategy to Pythia 70M (Biderman et al. 2023 (see Appendix C
Table I: Overview of LN-free, vanilla fine-tuned, and original GPT-2 models. Reported values are mean pross-entrops over 10.2M tokens from The Pile and The Pile-filtered, and 4.5M tokens from the Open WebText (OWT) validation set. For each model size and dataset, the lowest loss is highlighted in bold, and the loss difference between the LN-free model and the best-performing model shown in brackets. See Appendix A for Hugging Face links to the models and summary of compute
requirements.
Model
FT steps
OWT (val)
The Pile
The Pile-filtered
GPT-2 Small original
3.1006
2.8450
2.7899
GPT-2 Small vanilla
300
3.0126
2.8511
2.8112 12
LN-free
300
3.0797 [+0.0671]
2.8852 [+0.0402]
2.8757 [+0.0858]
GPT-2 Medium original
2.8145
2.5163
2.5390
GPT-2 Medium vanilla
500
2.7390
2.5752
2.5724
GPT-2 Medium LN-free
500
2.7642 [+0.0252]
2.6579 [+0.1416]
2.6352 [+0.0962]
GPT-2 Large original
0
2.6623
2.5320
2.4347
GPT-2 Large vanilla
600
2.6240
2.6233
2.5074
GPT-2 Large LN-free
600
2.6384 [+0.0144]
2.7504 [+0.2184]
2.5159 [+0.0812]
GPT-2 XL original
2.5567
2.4436 F
2.3739
GPT-2 XL Vanilla
800
2.4799
2.4673
2.3821
GPT-2 XL LN-free
800
2.5052 [+0.0253]
130.2197
2.3992 [+0.0253]
MECHANISTIC INTERPRETABILITY ANALYSES ON LN- MODELS
Removing IN eliminates nonlinear dependencies between components and results in models where residual stream directions map linearly to output logits. In this section, we evaluate common
interpretability methods, such Direct Logit Attribution (DLA) (nostalgebraist 2020 Elhage et al., 202T, Wang et al., 2022b Nanda 2023b) and attribution patching [Nanda 2023a) on LN-free models and compare the results to their counterparts with LN.
*GPT-2 XI. original:: Median: 1.0103, 95th pere: [0.0005, 10.6193), 99.9th pere: |=0000000, 43.0064]
*GPT-2 XI. LN-ffree: Median: 1.0937, 95th pere: [0.0004, 10.7548), 99. 9th perc: [==0.0000, 48.6459]
5

---

Published as a conference paper at ICLR 2026
Activation patching on GPT-2 Small
Activation patching on GPT-2 Small vanilla finetuned
Activation patching on
GPT-2 Small LN-free finetuned
6-
7-
8-
10-
narmalized logit difference
11
Attribution patching GPT-2 Small
Attribution patching GPT-2 Small vanilla finetuned
Attribution patching
GPT-2 Small LN-free finetuned
D-
10-
11
Then
Sera
and
Katic
went
house
Sara
gove
compuree to
Sara
Katta Went
Sara
gove
campurat
to
Sari Thon
and Katie
Sara gov
campuret to
sjendoro
slendoo
Position
Position
Position
Figure 2: Activation patching and attribution patching applied on the residual stream at different
layers and positions on GPT-2 Small and the corresponding vanilla and LN-free versions.
5.1 DIRECT LOGIT ATTRIBUTION ON LN-FREE MODELS GIVES EXACT DIRECT EFFECT ON LOGITS
Direct Logit Attribution (DLA) is an approximation to the Direct Effect (DE) of component. The DE (Pearl 2022 Geiger al.) (024) is the effect of model component the outputs that is not mediated by intermediate components, and can computed by subtracting a component's output e from the residual stream after the final layer, and taking the difference in outputs,
DE(c) LN(r) Wu LN(r c) Wo,
where Wy denotes the unembedding, and LN the final LayerNorm. The DLA approximation is computed using the cached LN scale,
(LA(c) = LNenchea WU:
(5)
which effectively linearizes LayerNorm (LN).
calculated both DLA and DE on each attention head GPT-2 Small original, GPT-2 Small vanilla FT, and GPT-2 Small LN-free FT, 1,000 sequences consisting of 512 tokens from The fileffileeedd, for logits corresponding to the correct target token. To compare metrics, we used the Normalized Mean Absolute Error (NMAE which measures the average discrepancy between DLA and DE, expressed as percentage of the average magnitude of the DE. Our LN-free fine-tuned model achieves perfect 0.00% [0.00% 0.00%] (95% Confidence Interval CI) NMAE, whereas the original model exhibits an NMAE of 49.07% [29.92 92%, 66.10% 10%]. This result shows that removing LNs makes these methods mathematically equivalent, eliminating the need for inaccurate linearization approximations. See Appendix for more details.
5.2 ACCURACY ATTRIBUTION PATCHING LN-FREE MODELS DOES NOT SIGNIFICANTLY IMPROVE
Activation patching Meeg et al. 2022 Zhang & Nanda 2023
Heimersheim & Nanda 2024) is an
interpretability method used assess the causal roles of neural network components by transferring
'e calculate NMAE, using averages of ahsolute differences and DE magnitude rather than per-sample ratios, did observe consistent proportional relationship between these
across samples.

---

Published as a conference paper at ICLR 2026
activations from "clean" prompt that elicits correct model behavior into a "corrupted" prompt that typically leads to incorrect behavior. Formally, this can be expressed as:
(6) where (xx measures differences in model predictions (typically logit differences), and dy < at (4(eee)))) indicates replacing the corrupted activation with its clean counterpart at layer 1. While precise, activation patching is computationally expensive, scaling with the number of components tested. Attribution patching Nanda (2023a) addresses this approximating activation patching with a first-order Taylor expansion around the corrupted activation, requiring only two forward passes and one backward pass,
A [x at + (r((eote))) - f (Zcor),
(Tewn: ar t (Fctem)) - f Vor (Zeon) (as (z )) = Aatr-
As LN projects residual vectors onto a (dmodel 1) -ddimensional sphere after removing the mean component, it causes derivatives to vanish when patched directions align with the residual stream and is, therefore, source of attribution patching errors, i.e. discrepancies between attribution patching estimates and ground-truth activation patching results (Nanda 2023a described it for this reason as "a particularly thorny nonlinearity").
We investigate whether LN is the primary factor limiting attribution patching accuracy by comparing attribution patching across three models: GPT-2 Small, the corresponding LN-free fine-tuned, and vanilla fine-tuned. We focused on the residual stream preceding each transformer block, a location where attribution patching is known perform particularly poorly in models with LN. We used 480 IO1 Wang et al. (2022b) prompts, systematically varying names, places, and objects, with each prompt paired with counterparts covering all possible name orderings. To ensure alignment across inputs, all prompts had fixed token lengths and name positions. We applied both techniques and quantified how well attribution patching approximates activation patching across layers. We used normalized logit differences as the patching metric to enable robust comparisons across methods. Surprisingly, attribution patching yielded very similar results across layers in the three models (see Fig. 2 and despite removing LN, observed no improvement in attribution patching accuracy. For each layer, we quantified this by computing the sum of absolute attribution patching errors across token positions in the vanilla fine-tuned model, and subtracting the corresponding value from the LN-free model. This yielded a per-layer improvement score, where positive values indicate lower attribution error in the LN-free model. Averaged across layers, the improvement is μ -0.026, with standard deviation a = 0.082. This negative but informative result suggests that attribution patching's 's limitations likely arise from other more fundamental nonlinearities in the transformer architecture, namely the attention SoftMax or the MLP activation functions.
5.3 FIRST POSITION TOKENS ARE NO LONGER SPECIAL
A well-documented phenomenon in transformer based language models is the disproportionately high L2 norm of first position token's 's hidden representations (Xiao et al. 2024: Yona et al., 2025 Barbero et al. 2025). This characteristic has been identified as a key mechanism sinks." where the first token captures outsized portion of attention across multiple heads, affecting information flow throughout the network. While this mechanism appears to help standard mod-
"attention
els avoid representational collapse by controlling information mixing across layers, it introduces computational irregularities and potential vulnerabilities (Yona et al. 2025).
To investigate whether our models exhibit similar behaviours, we measured the L2 norm of first position tokens, compared to all other tokens, on 1,000 sequences consisting of up to 512 tokens from The Pile-filtered. LN-free models reveal a disruption of the typical first position token norm pattern. As illustrated in Fig. 3 the L.-ffree model maintains consistently moderate 1.2 norm values (~300 to 500) across all layers for the first token, in contrast to the significant norm inflation observed in models with LN. This more uniform norm across token positions represents fundamental shift from the standard architecture, where the first token's norm typically exceeds that of other tokens by close to order of magnitude. The largest first token norm growth in all three models was due to the attention head in layer 3, where norms grow from ~500 to 3,600 for the models with LN.
We also investigated the attention sink rate across models, defined as the proportion of attention heads where the first token attracts at least 30% of overall attention. For the original model, the sink rate was 55.3% [53.1 %, 58.1 1%] (95% CI), which dropped to 45.3% [42.0%, 48.5% for our LN-free
7

---

Published as a conference paper at ICLR 2026
GPTZ-Medium original
4200
4000
GPT2- MPrT-MMdium vanilla FT
GPTZ-Medium LN-free
-300
5000
12 Norm
2000
2000
1000
1000
Ron Toy
ma Ty
uny Ao
my
Figure 3: L.2 norm growth for first position tokens (left) versus other positions tokens (right) for GPT-2 Medium models. First token norms significant deviate from norms at other positions for models trained with LN. LN-free model treats first token norm similarly to other positions.
variant. Interestingly, while this represents notable reduction in sink rate, it is not proportional to the reduction we observed in 1.2 norms. This suggests that the relationship between relative token norm magnitudes and attention sink behavior is likely complex, with attention mechanisms potentially maintaining some degree of positional bias toward the first token even when its norm is substantially reduced.
This effect is likely due to the constant linear scaling applied by FakeLN. In models with LN, residual stream vectors are scaled by their individual standard deviations, meaning components trained to operate under normalized input conditions. Once LN is removed, this normalization is no longer enforced. To compensate, the model appears to adapt by reducing variability in token norms, such as between the first token and the rest of the sequence. Our auxiliary loss further encourages norm consistency by explicitly penalizing variation across positions. however, we did observe this fundamental change in norm behavior even in experiments without this loss term.
5.4 CONFIDENCE NEURONS ARE NEUTERED IN LN-FREE MODELS
When developing our LN-free model variants, we observed consistent pattern: models exhibited significant overconfidence compared their original counterparts. For GPT-2 Medium, the average entropy of the output distribution decreased from 2.86 [2.86, 2.87] (95% CI) in the original model to 2.53 [2.52, 2.54] in the LN-free version. Correspondingly, the expected calibration error, defined as the average absolute difference between the predicted confidence and accuracy, increased from 0.019 [0.018. 0.020] to 0.034 [0.033, 0.035]. Motivated by these observations, we investigated how the recently discovered "confidence neurons" (also referred to "entropy neurons") (Katz & Belinkov 2023 Gurnee et al. 2024 Stolfo et al. 2024) in the final MLP layer were affected by our LN removal strategy.
Following Stolfo et al. (0024) we define confidence neurons as neurons in the final MLP with (a) high weight norm, and (b) a uniform impact all output logits. We detail how confidence neurons were identified and further analysis in Appendix H We identified the same top-3 confidence neurons (1083, 1 1108, 3144) in GPT-2 Medium original, vanilla FT, and LN-free. To measure their importance in each model, we conducted mean ablations on 1,000 sequences consisting of 512 tokens in The Pile-filtered. For each neuron i, we replaced its input activation with its mean value across the dataset (z; - E[zz)). This intervention removes the neuron's 's contextual information while maintaining its average contribution. Figure 4 highlights the absolute change in cross-entropy loss when mean ablating each neuron. In the GPT-2 Medium original, all three neurons increase CE loss when ablated, with neuron 3144 showing the largest effect. In contrast, the impact is completely eliminated in LN-free model. This confirms that linearizing LN completely disables entropy neurons in the final MLP layer, further supporting previous work that identified LN's s non-linearity as their primary enabling mechanism (Stolfo et al. 2024). We also observed decrease in the effectiveness
8

---

Published as a conference paper at ICLR 2026
0.06
GPtD-ledumm original
GPT2 Medium vanila
GPT2-Med Ltkliee FT
Lo5s
0.08
Netron 1003
Neuror 1108
Neuron 3144
Figure 4: Absolute change in cross-entropy (CE) when ablating top-3 confidence neurons in GPT-2 Medium models. GPT-2 Medium original demonstrates a significant change in CE loss upon ablating, effect is significantly dampened in vanilla FT, and completely disappears in LN-free.
of confidence neurons our vanilla FT model, likely due to our fine-tuning hyperparameters, which we discuss further in Appendix H
DISCUSSION
6.1 LIMITATIONS
successfully remove LN from GPT-2 models. Here, we want to highlight common issues and possible limitations of this process. We find that the fine-tuning process when LNs are partially removed is, as expected, less stable. We find that the training loss can spike to high values on some inputs, which sometimes causes the training run to fail (irrecoverably high loss). A common failure observed are exploding gradients, which most often occur during LN; removal. Instabilities usually appear as a cascade of increasing gradient norms or exploding gradients in a single step.
While our LN-removal strategies developed on GPT-2 Small and Medium largely transfer to the Large and XL models, they required significant hyperparameter tuning. Additionally, an early version of our protocol without auxiliary loss worked for GPT-2 Small, but did not scale to larger models, suggesting that protocols don't always generalize across models.
As highlighted in Section 5.4 all of our LN-free models exhibit overconfidence compared their LN counterparts. While our experiments demonstrate that removing LN effectively neutralizes confidence neurons, the magnitude of the observed increase in overconfidence suggests additional contributing factors. It's possible that without the normalizing effect of standard LN, attention, and MLP components must now handle greater variability in residual stream inputs, potentially compromising their ability to contribute to appropriate uncertainty quantification.
Finally, we note that the LN-free models we open-source are not easily quantizable. We do not consider this a significant limitation as quantization is uncommon in mechanistic interpretability research, the primary intended use case for these models.
6.2 FUTURE WORK
We focused primarily on GPT-2 models, due their ubiquity in the interpretability community. In the future, we would like to expand our LN removal protocol to more recent models (see Appendix C and on expectations regarding generalizations to different architectures).
More models:
Parameter efficient fine-tuning: So far we used full fine-tuning, While this was feasible for GPT-2 sized models, want explore parameter efficient fine-tuning strategies in the future.

---

Published as a conference paper at ICLR 2026
Further protocol optimization: We noticed that the gap between removing the LN in different layers can be reduced for L. and MLP: in fact some experimental runs showed that we could remove those instances of LN in all layers simultaneously (only LN;, always required gaps).
Circuits interpretability: Attempts to create sparse computational graph to represent neural network are hindered by LN. It would be interesting to see techniques like Marks al. (2024) benefit from removing LN layers.
CONCLUSIONS
We showed that layer normalization can be gradually removed from transformer models with minimal performance loss using a fine-tuning procedure, demonstrating this on all GPT-2 models (and Pythia-70M). Our results demonstrate that nonlinearities such as LN are not necessary for GPT-2- class transformers to effectively model language at inference time. We detailed our procedure and the strategies used to address hyperparameter sensitivity. Applying interpretability techniques, we found that in LN-free models DLA becomes exact estimate of DE and first-token residual norms become comparable to those at other positions. Surprisingly, attribution patching does not improve in LN-frre models, suggesting its limitations stem from other nonlinearities. Finally, showed that LN-free models lack operational entropy neurons, contributing towards the more generally observed trend of model overconfidence. More generally, LN-free models enable more precise mechanistic analyses when LN's nonlinearity becomes particularly problematic and, additionally, can serve as testbeds for evaluating the impact of the nonlinearities introduced by LN on mechanistic interpretability techniques. Open-sourcing these models, we hope to contribute to mechanistic interpretability research.
ACKNOWLEDGMENTS
IS and MS conducted this research as part of the MARS program by the Cambridge AI Safety Hub (CAISH). Furthermore, CAISH provided compute resources. LB and GK conducted this research as part of the Supervised Program for Alignment Research (SPAR). GK was supported by Open Philanthropy.
AUTHOR CONTRIBUTIONS
MS and JS worked jointly on the fine-tuning code and scaled up the LN-removal. MS contributed ideas behind auxiliary loss and recomputation and did attention sink rate analysis. JS optimized the removal schedule and recomputation. JS and MS conducted final experiments for the LN-free models reported in the manuscript.
LB and GK worked jointly on the mechanistic interpretability experiments. LB led the creation the Pile-filtered dataset and the implementation of the attribution patching analysis. GK led the BOS token size and confidence neuron analyses. Both LB and GK contributed to the direct logit attribution experiments, with GK running the final experiments reported in the manuscript.
SH coordinated the project and provided advice and mentorship throughout.
All authors contributed to the writing.
REFERENCES
Emmanuel Ameisen, Jack Lindsey, Adam Pearce, Wes Gurnee, Nicholas L. Turner, Brian Chen, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. Circuit tracing: Revealing computational graphs in language models. Transformer Circuits Thread, 2025. URLhttps V/transfoxmer's caae--iiccuttt. wttp:ppppp///////trtrttee attribution. graphs heaeohss/eettan /tt.///tthooooooooooo html
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016.
10

---

Published as a conference paper at ICLR 2026
Federico Barbero, Alvaro Arroy,, Xiangming Gu, Christos Perivolaropoulos, Michael Bronstein, Petar Veličković, and Razvan Pascanu. Why do LLMs attend to the first token? arXiv preprint 02Xx::2500.002732. 2025.
Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O'Brien, Eric Hal- lahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. arXiv preprini
arXiv:2304.0137., 2023.
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: commonsense natural language. Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 7432-7439, 2020.
Reasoning about physical
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph. Zae Hatfield-Dodds. Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. https:/ttansformee.cciiittt htt.r/awws..ubo2222mooooommaaata features/index.htmll
Lucius Bushnaq, Stefan Heimersheim, Nicholas Goldowsky-Die Dan Braun, Jake Mendel, Kaarel Hänni, Avery Griffin, Jöm Stöhler, Magdalena Wache, and Marius Hobbhahn. The local interaction basis: Identifying computationnllly-eeeeeee and sparsely interacting features in neural networks. arXiv preprint arXiv:2405. 10928, 2024.
Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint 1rX1v:1905.10044, 2019.
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. https:/ttransforme-circuuitt wwwpuiit..pp///221111. pwtppput//raaeeeeaa wfw.e///wwwaaaeeww/////e/ee html.
Lucy Farnik, Tim Lawson, Conor Houghton, and Laurence Aitchison. Jacobian sparse autoencoders: Sparsify computations, not just activations. arXiv preprint arXiv:2502. 18147, 2025.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite. Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00m 0000007700000 2020.
Atticus Geiger, Zhengxuan Wu, Christopher Potts, Thomas Icard, and Noah Goodman. Finding alignments between interpretable causal variables and distributed neural representations, In Causal Learning and Reasoning, pp. 160-187. PMLR, 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptin Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katic Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul R. Barham, Tom Hennigan, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, Ryan Doherty, Eli Collins, Clemens Meyer, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, Jack Krawezyk, Cosmo Du, Ed Chi, Tze Cheng, Eric Ni. Purvi Shah, Patrick Kane, Betty Chan, Manaal Faruqui, Aliaksei Severyn, Hanzhao Lin, YaGuang Li, Yong Cheng, Abe Ittycheriah, Mahdis Mahdieh, Mia Chen, Pei Sun, Dustin Tran, Sumit Bagri, Balaji Lakshminarayanan. Jeremiah Liu, Andras Orban, Fabian Güra, Hao Zhou, Xinying Song, Aurelien Boffy, Harish Ganapathy. Steven Zheng, HyunJeong Choe, Ágoston Weisz, Tao Zhu, Yifeng Lu, Siddharth Gopal, Jarrod Kahn, Maciej Kula, Jeff Pitman, Rushin Shah, Emanuel Taropa, Majd Al Merey, Martin Baeuml, Zhifeng Chen, Laurent El Shafey. Yujing Zhang, Olcan Sercinoglu, George Tucker, Enrique Piqueras, Maxim Krikun, lain Barr, Nikolay Savinov, Ivo Danihelka, Becca Roelofs,
11

---

Published as a conference paper at ICLR 2026
Anais White, Anders Andreassen, Tamara von Glehn, Lakshman Yagati, Mehran Kazemi, Lucas Gonzalez, Misha Khalman, Jakub Sygnowski, Alexandre Frechette, Charlotte Smith, Laura Culp, Lev Proleev, Yi Luan, Xi Chen, James Lottes, Nathan Schucher, Federico Lebron, Alban Rrustemi, Natalie Clay, Phil Crone, Tomas Kocisky, Jeffrey Zhao, Bartek Perz, Dian Yu, Heidi Howard, Adam Bloniarz, Jack W. Rae, Han Lu, Laurent Sifre, Marcello Maggioni, Fred Alcober, Dan Megan Barnes, Shantanu Thakoor, Jacob Austin, Gabriel Barth-Maron, William Wong, Rishabh Joshi, Rahma Chaabouni, Deeni Fatiha, Arun Ahuja, Gaurav Singh Tomar, Evan Senter, Martin Chadwick, Ilya Kornkkov, Nithya Attaluri, Iñaki Iturrate, Ruibo Liu, Yunxuan Li, Sarah Cogan, Jeremy Chen, Chao Jia, Chenjie Gu, Qiao Zhang, Jordan Grimstad, Ale Jakse Hartman, Xavier Garcia, Thanumalayan Sankaranarayana Pillai, Jacob Devlin, Michael Laskin, Diego de Las Casas, Dasha Valter, Connie Tao, Lorenzo Blanco, Adrià Puigdomènech Badia, David Reitter, Mianna Chen, Jenny Brennan, Clara Rivera, Sergey Brin, Shariq Iqbal, Gabriela Surita, Jane I. Labanowski, Abhi Rao, Stephanie Winkler, Emilio Parisotto, Yiming Gu, Kate Olszewska, Ravi Addanki, Antoine Miech, Annie Louis, Denis Teplyashin, Geoff Brown, Elliot Catt, Jan Balaguer. Jackic Xiang, Pidong Wang. Zoe Ashwood, Anton Briukhov, Albert Webson, Sanjay Ganapathy, Smit Sanghavi, Ajay Kannan, Ming- Wei Chang, Axel Stjerngren, Josip Djolonga, Yuting Sun, Ankur Bapna, Matthew Aitchison, Pedram Pejman, Henryk Michalewski, Tianhe Yu, Cindy Wang, Juliette Love, Junwhan Ahn, Dawn Bloxwich, Kehang Han, Peter Humphreys, Thibault Sellam, James Bradbury. Varun Godbole, Sina Samangooei, Bogdan Damoc, Alex Kaskasoli, Sébastien M. R. Amold, Vijay Vasudevan, Shubham Agrawal, Jason Riesa, Dmitry Lepikhin, Richard Tanburn. Srivatsan Srinivasan, Hyeontaek Lim, Sarah Hodkinson, Pranav Shyam, Johan Ferret, Steven Hand, Ankush Garg. Tom Le Paine, Jian Li, Yujia Li, Minh Giang, Alexander Neitz, Zaheer Abbas, Sarah York, Machel Reid, Elizabeth Cole, Aakanksha Chowdhery. Dipanjan Das, Dominika Rogozińska, Vitaliy Nikolaev, Pablo Sprechmann, Zachary Nado, Lukas Zilka, Flavien Prost, Luheng He, Marianne Monteiro, Gaurav Mishra, Chris Welty, Josh Newlan, Dawei Jia, Miltiadis Allamanis, Clara Huiyi Hu, Raoul de Liedekerke, Justin Gilmer, Carl Saroufim, Shruti Rijhwani, Shaobo Hou, Disha Shrivastava, Anirudh Baddepudi, Alex Goldin, Adnan Ozturel, Albin Cassirer, Yunhan Xu, Daniel Sohn, Devendra Sachan, Reinald Kim Amplayo, Craig Swanson, Dessie Petrova, Shashi Narayan, Arthur Guez, Siddhartha Brahma, Jessica Landon, Miteyan Patel, Ruizhe Zhao, Kevin Villela, Luyu Wang, Wenhao Jia, Matthew Rahtz, Mai Giménez, Legg Yeung, James Keeling, Petko Georgiev, Diana Mineu, Boxi Wu, Salem Haykal, Rachel Saputro, Kiran Vodrahalli, James Qin, Zeynep Cankara, Abhanshu Sharma, Nick Fernando. Will Hawkins, Behnam Neyshabur, Solomon Kim, Adrian Hutter, Priyanka Agrawal, Alex Castro-Ros, George van den Driessche, Tao Wang. Fan Yang. Shuo yiin Chang, Paul Komarek, Ross Mcllroy. Mario Lučić, Guodong Zhang, Wael Farhan, Michael Sharman, Paul Natsev, Paul Michel, Yamini Bansal, Siyuan Qiao, Kris Cao. Siamak Shakeri, Christina Butterfield, Justin Chung, Paul Kishan Rubenstein, Shivani Agrawal, Arthur Mensch, Kedar Soparkar, Karel Lene, Timothy Chung, Aedan Pope, Loren Maggiore, Jackie Kay, Priya Jhakra, Shibo Wang, Joshua Maynez, Mary Phuong. Taylor Tobin, Andrea Tacchetti, Maja Trebacz, Kevin Robinson, Yash Katariya, Sebastian Riedel, Paige Bailey, Kefan Xiao, Nimesh Ghelani, Lora Aroyo, Ambrose Slone, Neil Houlsby, Xuehan Xiong, Zhen Yang, Elena Gribovskaya, Jonas Adler, Mateo Wirth, Lisa Lee, Music Li, Thais Kagohara, Jay Pavagadhi, Sophie Bridgers, Anna Bortsova, Sanjay Ghemawat, Zafarali Ahmed, Tianqi Liu, Richard Powell, Vijay Bolina, Mariko linuma, Polina Zablotskaia, James Besley, Da-Woon Chung, Timothy Dozat, Ramona Comaneseu, Xiance Si, Jeremy Greer, Guolong Su, Martin Polacek, Raphaël Lopez Kaufman, Simon Tokumine, Hexiang Hu, Elena Buchatskaya, Yingjie Miao, Mohamed Elhawaty, Aditya Siddhant, Nenad Tomasev, Jinwei Xing. Christina Greer, Helen Miller, Shereen Ashraf, Aurko Roy, Zizhao Zhang, Ada Ma, Angelos Filos, Milos Besta, Rory Blevins, Ted Klimenko, Chih- Yeh, Soravit Changpinyo, Jiaqi Mu, Oscar Chang, Mantas Pajarskas, Carrie Muir, Vered Cohen, Charline Le Lan, Krishna Haridasan, Amit Marathe, Steven Hansen, Sholto Douglas, Rajkumar Samuel, Mingqiu Wang. Sophia Austin, Chang Lan, Jiepu Jiang, Justin Chiu, Jaime Alonso Lorenzo, Lars Lowe Sjösund, Sébastien Cevey, Zach Gleicher, Thi Avrahami, Anudhyan Boral, Hansa Srinivasan, Vittorio Selo, Rhys May, Konstantinos Aisopos, Léonard Hussenot, Livio Baldini Soares, Kate Baumli, Michael B. Chang, Adrià Recasens, Ben Caine, Alexander Pritzel, Filip Pavetic, Fabio Pardo, Anita Gergely, Justin Frye, Vinay Ramasesh, Dan Horgan, Kartikeya Badola, Nora Kassner, Subhrajit Roy, Ethan Dyer, Victor Campos Campos, Alex Tomala, Yunhao Tang, Dalia El Badawy, Elspeth White. Basil Mustafa, Oran Lang, Abhishek Jindal, Sharad Vikram, Zhitao Gong, Sergi Caelles, Ross Hemsley, Gregory Thornton, Fangxiayu Feng, Wojciech Stokowiec, Ce Zheng, Phoebe Thacker, Çağlar Unlü, Zhishuai Zhang, Mohammad Saleh, James Svensson, Max Bileschi, Piyush Patil, Ankesh Anand, Roman Ring, Katerina Tsihlas, Arpi
Garrette,
12

---

Published as a conference paper at ICLR 2026
Vezer, Marco Selvi, Toby Shevlane, Mikel Rodriguez, Tom Kwiatkowski, Samira Daruki, Keran Rong, Allan Dafoe, Nicholas FitzGerald, Keren Gu-Lemberg, Mina Khan, Lisa Anne Hendricks, Marie Pellat, Vladimir Feinberg, James Cobon-Kern Tara Sainath, Maribeth Rauh, Sayed Hadi Hashemi, Richard Ives, Yana Hasson, Eric Noland, Yuan Cao, Nathan Byrd, Ie Hou, Qingze Wang, Thibault Sottiaux, Michela Paganini, Jean-Baptiste Lespiau, Alexandre Moufarek, Samer Hassan, Kaushik Shivakumar, Joost van Amersfoort, Amol Mandhane, Pratik Joshi, Anirudh Goyal, Matthew Tung, Andrew Brock, Hannah Sheahan, Vedant Misra, Cheng Li, Nemanja Rakićević, Mostafa Dehghani. Fangyu Liu, Sid Mittal, Junhyuk Oh, Seb Noury, Eren Sezener, Fantine Huot, Matthew Lamm, Nicola De Cao, Charlie Chen, Sidharth Mudgal, Romina Stella, Kevin Brooks, Gautam Vasudevan, Chenxi Liu, Mainak Chain, Nivedita Melinkeri, Aaron Cohen, Venus Wang. Kristie Seymore, Sergey Zubkov, Rahul Goel, Summer Yue, Sai Krishnakumaran. Brian Albert, Nate Hurley, Motoki Sano, Anhad Mohananey, Jonah Joughin, Egor Filonov, Tomasz Kępa, Yomna Eldawy, Jiawern Lim, Rahul Rishi, Shirin Badiczadegan, Taylor Bos, Jerry Chang, Sanil Jain, Sri Gayatri Sundara Padmanabhan, Subha Puttagunta, Kalpesh Krishna, Leslie Baker, Norbert Kalb, Vamsi Bedapudi, Adam Kurzrok, Shuntong Lei, Anthony Yu, Oren Litvin, Xiang Zhou, Zhichun Wu, Sam Sobell, Andrea Siciliano, Alan Papir, Robby Neale, Jonas Bragagnolo, Tej Toor, Tina Chen, Valentin Anklin, Feiran Wang, Richie Feng, Milad Gholami, Kevin Ling, Lijuan Liu, Jules Walter, Hamid Moghaddam, Arun Kishore. Jakub Adamek, Tyler Mercado, Jonathan Mallinson, Siddhinita Wandekar, Stephen Cagle, Eran Ofek, Guillermo Garrido, Clemens Lombriser, Maksim Mukha, Botu Sun, Hafcezul Rahman Mohammad, Josip Matak, Yadi Qian, Vikas Peswani, Pawel Janus, Quan Yuan, Leif Schelin, Oana David, Ankur Garg. Yifan He, Oleksii Duzhyi, Anton Algmyr, Timothée Lottaz Qi Li, Vikas Yadav, Luyao Xu, Alex Chinien, Rakesh Shivanna, Aleksandr Chuklin, Josie Li, Carrie Spadine, Travis Wolfe, Kareem Mohamed, Subhabrata Das, Zihang Dai, Kyle He, Daniel von Dincklage, Shyam Upadhyay, Akanksha Maurya, Luyan Chi, Sebastian Krause, Khalid Salama, Pam G Rabinovitch, Pavan Kumar Reddy M, Aarush Selvan, Mikhail Dektiarev, Golnaz Ghiasi, Erdem Guven, Himanshu Gupta, Boyi Liu, Deepak Sharma, Idan Heimlich Shtacher, Shachi Paul, Oscar Akerlund, François- Xavier Aubet. Terry Huang, Chen Zhu, Eric Zhu, EliCo Teixeira, Matthew Fritze, Francesco Bertolini, -aaaa--lcooora Marinescu, Martin Bölle, Dominik Paulus, Khyatti Gupta, Tejasi Latkar, Max Chang, Jason Sanders, Roopa Wilson, Xuewei Wu, Yi-Xuan Tan, Lam Nguyen Thiet, Tulsee Doshi, Sid Lall, Swaroop Mishra, Wanming Chen, Thang Luong, Seth Benjamin, Jasmine Lee, Ewa Andrejczuk, Dominik Rabiej, Vipui Ranjan, Krzysztof Styre. Pengcheng Yin, Jon Simon, Malcolm Rose Harriott, Mudit Bansal, Alexei Robsky, Geoff Bacon, David Greene, Daniil Mirylenka, Chen Zhou, Obaid Sarvana, Abhimanyu Goyal, Samuel Andermatt, Patrick Siegler, Ben Horn, Assaf Israel, Francesco Pongetti, Chih-Wei "Louis" Chen, Marco Selvatici, Pedro Silva, Kathie Wang, Jackson Tolins, Kelvin Guu, Roey Yogev, Xiaochen Cai, Alessandro Agostini, Maulik Shah, Hung Nguyen, Noah Ó Donnaile, Sébastien Pereira, Linda Friso, Adam Stambler, Adam Kurzrok, Chenkai Kuang, Yan Romanikhin, Mark Geller, ZJ Yan, Kane Jang, Cheng-Chui Lee, Wojeiech Fica, Eric Malmi, Qijun Tan, Dan Banica, Daniel Balle, Ryan Pham, Yanping Huang, Diana Avram, Hongzhi Shi, Jasjot Singh, Chris Hidey, Niharika Ahuja, Pranab Saxena, Dan Dooley, Srividya Pranavi Potharaju, Eileen O'illl, Anand Gokulchandran. Ryan Foley, Kai Zhao, Mike Dusenberry, Yuan Liu, Pulkit Mehta, Ragha Kotikalapudi, Chalence Seffaanek-hhrader Andrew Goodman, Joshua Kessinger, Eran Globen, Prateck Kolhar, Chris Gorgolewski, Ali Ibrahim. Yang Song. Ali Eichenbaum, Thomas Brovelli, Sahitya Potluri, Preethi Lahoti, Cip Baetu, Ali Ghorbani, Charles Chen, Andy Crawford, Shalini Pal, Mukund Sridhar, Petru Gurita, Asier Mujika, Igor Petrovski, Pierre-Louis Cedoz, Chenmei Li, Shiyuan Chen, Niccolò Dal Santo, Siddharth Goyal. Jitesh Punjabi, Karthik Kappaganthu, Chester Kwak, PaLlavi LV, Sarmishta Velury, Himadri Choudhury, Jamie Hall, Premal Shah, Ricardo Figueira, Matt Thomas, Minjie Lu, Ting Zhou, Chintu Kumar, Thomas Jurdi, Sharat Chikkerur, Yenai Ma, Adams Yu. Soo Kwak, Victor Ähdel, Sujeevan Rajayogam, Travis Choma, Fei Liu, Aditya Barua, Colin Ji, Ji Ho Park, Vincent Hellendoorn, Alex Bailey. Taylan Bilal, Huanjie Zhou, Mehrdad Khatir, Charles Sutton, Wojeiech Rzadkowski, Fiona Macintosh, Konstantin Shagin, Paul Medina, Chen Liang, Jinjing Zhou, Pararth Shah, Yingying Bi, Attila Dankovics, Shipra Banga, Sabine Lehmann, Marissa Bredesen, Zifan Lin. John Eric Hoffmann. Jonathan Lai, Raynald Chung, Kai Yang, Nihal Balani, Arthur Bražinskas, Andrei Sozanschi. Matthew Hayes, Héctor Fernández Alcalde, Peter Makarov Will Chen, Antonio Stella, Liselotte Snijders, Michael Mandl, Ante Kärrman, Pawet Nowak, Xinyi Wu, Alex Dyck, Krishnan Vaidyanathan, Raghavender R, Jessica Mallet, Mitch Rudominer, Eric Johnston, Sushil Mittal, Akhil Udathu, Janara Christensen, Vishal Verma, Zach Irving, Andreas Santucci, Gamaleldin Elsayed, Elnaz Davoodi, Marin Georgiev, Ian Tenney, Nan Hua, Geoffrey Cideron, Edouard Leurent, Mahmoud Alnahlawi, Ionut Georgescu,
13

---

Published as a conference paper at ICLR 2026
Nan Wei, Ivy Zheng, Dylan Scandinaro, Heinrich Jiang, Jasper Snoek, Mukund Sundararajan, Xuezhi Wang, Zack Ontiveros, Itay Karo, Jeremy Cole, Vinu Rajashekhar, Lara Tumeh, Eyal Ben- David, Rishub Jain, Jonathan Uesato, Romina Datta, Oskar Bunyan, Shimu Wu, John Zhang, Piotr Stanczyk, Ye Zhang, David Steiner, Subhajit Naskar, Michael Azzam, Matthew Johnson, Adam Paszke, Chung-Cheng Chiu, Jaume Sanchez. Elias, Afroz Mohiuddin, Faizan Muhammad, Jin Miao, Andrew Lee, Nino Vieillard, Jane Park, Jiageng Zhang, Jeff Stanway, Drew Garmon, Abhijit Karmarkar, Zhe Dong, Jong Lee, Aviral Kumar, Luowei Zhou, Jonathan Evens, William Isaac, Geoffrey Irving, Edward Loper, Michael Fink, Isha Arkatkar, Nanxin Chen, Izhak Shafran, Ivan Petrychenko, Zhe Chen, Johnson Jia, Anselm Levskaya, Zhenkai Zhu, Peter Grabowski, Yu Mao, Alberto Magni, Kaisheng Yao, Javier Snaider, Norman Casagrande, Evan Palmer, Paul Suganthan, Alfonso Castaño, Irene Giannoumis, Wooyeol Kim, Mikolaj Rybinski, Ashwin Sreevatsa, Jennifer Prendki, David Soergel, Adrian Goedeckemeyer, Willi Gierke, Mohsen Jafari, Meenu Gaba, Jeremy Wiesner, Diana Gage Wright, Yawen Wei, Harsha Vashisht, Yana Kulizhskaya, Jay Hoover, Maigo Le, Lu Li, Chimezie Iwuanyanwu, Lu Liu, Kevin Ramirez, Andrey Khorlin, Albert Cui, Tian LIN, Marcus Wu, Ricardo Aguilar, Keith Pallo, Abhishek Chakladar, Ginger Perng. Elena Allica Abellan, Mingyang Zhang, Ishita Dasgupta, Nate Kushman, Ivo Penchev, Alena Repina, Xihui Wu, Tom van der Weide, Priya Ponnapalli, Caroline Kaplan, Jiri Simsa, Shuangfeng Li, Olivier Dousse, Fan Yang. Jeff Piper, Nathan Ie, Rama Pasumarthi, Nathan Lintz, Anitha Vijayakumar, Daniel Andor, Pedro Valenzuela, Minnie Lui, Cosmin Paduraru, Daiyi Peng, Katherine Lee, Shuyuan Zhang, Somer Greene. Duc Dung Nguyen, Paula Kurylowicz. Cassidy Hardin. Lucas Dixon, Lili Janzer, Kiam Choo, Ziqiang Feng, Biao Zhang, Achintya Singhal. Dayou Du, Dan McKinnon, Natasha Antropova, Tolga Bolukbasi, Orgad Keller, David Reid, Daniel Finchelstein, Maria Abi Raad, Remi Crocker, Peter Hawkins, Robert Dadashi, Colin Gaffney, Ken Franko, Anna Bulanova, Rémi Leblond, Shirley Chung, Harry Askham, Luis C. Cobo, Kelvin Xu, Felix Fischer, Jun Xu, Christina Sorokin, Chris Alberti, Chu-Cheng Lin, Colin Evans, Alek Dimitriev, Hannah Forbes, Dylan Banarse, Zora Tung, Mark Omernick, Colton Bishop, Rachel Sterneck, Rohan Jain, Jiawei Xia, Ehsan Amid. Francesco Piceinno. Xingyu Wang, Praseem Banzal, Daniel J. Mankowitz, Alex Polozov, Victoria Krakovna, Sasha Brown, MohammadHosseem Bateni, Dennis Duan, Vlad Firoiu, Meghana Thotakuri, Tom Natan, Matthieu Geist, Ser tan Girgin, Hui Li, Jiayu Ye, Ofir Roval, Reiko Tojo, Michael Kwong, James Lee-Thorp, Christopher Yew, Danila Sinopalnikov, Sabela Ramos, John Mellor, Abhishek Sharma, Kathy Wu, David Miller, Nicolas Sonnerat, Denis Vnukov, Rory Greig, Jennifer Beattie, Emily Caveness, Libin Bai, Julian Eisenschlos, Alex Korchemniy, Tomy Tsai, Mimi Jasarevic, Weize Kong, Phuong Dao, Zeyu Zheng. Frederick Liu, Fan Yang. Rui Zhu, Tian Huey Teh, Jason Sammiya, Evgeny Gladchenko, Neje Trdin, Daniel Toyama, Evan Rosen, Sasan Tavakkol, Linting Xue, Chen Elkind, Oliver Woodman, John Carpenter, George Papamakarios, Rupert Kemp, Sushant Kafle, Tanya Grunina, Rishika Sinha, Alice Talbert, Diane Wu, Denese Owusu-Afriyie, Cosmo Du, Chloe Thornton, Jordi Pont-Tuset, Pradyumna Narayana, Jing Li. Saaber Fatehi, John Wicting, Omar Ajmeri, Benigno Uria, Ycongil Ko, Laura Knight, Amélic Héliou, Ning Niu, Shane Gu, Chenxi Pang. Yeqing Li, Nir Levine, Ariel Stolovich, Rebeca Bantamaria-Fernandee
Sonam Goenka, Wenny Yustalim, Robin Strudel, Ali Elqursh, Charlie Deck, Hyo Lee, Zonglin Li, Kyle Levin, Raphael Hoffmann, Dan Holtmann- Olivier Bachem, Sho Arora, Christy Koh, Soheil Hassas Yeganeh, Siim Pöder, Mukarram Tariq, Yanhua Sun, Lucian Ionita, Mojtaba Seyedhosseini, Pouya Tafti, Zhiyu Liu, Anmol Gulati, Jasmine Liu, Xinyu Ye. Bart Chrzaszez. Lily Wang, Nikhil Sethi, Tianrun Li, Ben Brown, Shreya Singh, Wei Fan, Aaron Parisi, Joe Stanton, Vinod Koverkathu, Christopher A. Choquette-Choe Abe Ittycheriah, Prakash Shroff, Mani Varadarajan, Sanaz Bahargam, Rob Willoughby, David Gaddy, Guillaume Desjardins. Marco Cornero, Brona Robenek, Bhavishya Mittal, Ben Albrecht, Ashish Shenoy, Fedor Moiseev, Henrik Jacobsson, Alireza Ghaffarkhah, Morgane Rivière, Alanna Walton, Clément Crepy, Alicia Parrish, Zongwei Zhou Clement Farabet, Carey Radebaugh, Praveen Srinivasan, Claudia van der Salm, Andreas Fidjeland, Salvatore Scellato, Eri Latorre- Chimoto, Hanna Klimezak-Plucinfs Mendolicchio, Lexi Walker, Alex Morris, Matthew Mauger, Alexey Guseynov, Alison Reid, Seth Odoom, Lucia Loher, Victor Cotruta, Madhavi Yenugula, Dominik Grewe, Anastasia Petrushkina, Tom Duerig, Antonio Sanchez, Steve Yadlowsky, Amy Shen, Amir Globerson, Lynette Webb, Sahil Dua, Dong Li, Surya Bhupatiraju, Dan Hurt, Haroon Qureshi, Ananth Agarwal, Tomer Shani, Matan Eyal, Anuj Khare, Shreyas Rammohan Belle, Lei Wang. Chetan Tekur, Mihir Sanjay Kale, Jinliang Wei, Ruoxin Sang, Brennan Sacta, Tyler Liechty, Yi Sun, Yao Zhao, Stephan Lee, Pandu Nayak, Doug Fritz, Manish Reddy Vuyyuru, John Aslanides, Nidhi Vyas, Martin Wicke, Xiao Ma, Evgenii Eltyshev, Nina Martin, Hardie Cate, James Manyika, Keyvan Amiri,
Yunjie Li, TJ Lu,
David Bridson, Dario de Cesare, Tom Hudson, Piermaria
14

---

Published as a conference paper at ICLR 2026
Yelin Kim, Xi Xiong, Kai Kang, Florian Luisier, Nilesh Tripuraneni, David Madras, Mandy Guo, Austin Waters, Oliver Wang. Joshua Ainslie, Jason Baldridge, Han Zhang, Garima Pruthi, Jakob Bauer, Feng Yang, Riham Mansour, Jason Gelman, Yang Xu. George Polovets, Ji Liu, Honglong Cai, Warren Chen, XaangHai Sheng, Emily Xue, Sherjil Ozair, Christof Angermueller, Xiaowei Li, Anoop Sinha, Weiren Wang, Julia Wiesinger, Emmanouil Koukoumidis, Yuan Tian, Anand Iyer, Madhu Gurumurthy, Mark Goldenson, Parashar Shah, MK Blake, Hongkun Yu, Anthony Urbanowicz, Jennimaria Palomaki, Chrisantha Fernando, Ken Durden, Harsh Mehta, Nikola Momchev, Elahe Rahimtoroghi, Maria Georgaki, Amit Raul, Sebastian Ruder, Morgan Redshaw, Jinhyuk Lee, Denny Zhou, Komal Jalan, Dinghua Li, Blake Hechtman, Parker Schuh, Milad Nasr, Kieran Milan, Vladimir Mikulik, Juliana Franco, Tim Green, Nam Nguyen, Joe Kelley, Aroma Mahendru, Andrea Hu, Joshua Howland, Ben Vargas, Jeffrey Hui, Kshitij Bansal, Vikram Rao, Rakesh Ghiya, Emma Wang. Ke Ye, Jean Michel Sarr, Melanie Moranski Preston, Madeleine Elish, Steve Li, Aakash Kaku, Jigar Gupta, Ice Pasupat, Da-Cheng Juan, Milan Someswar, Tejvi M., Xinyun Chen, Aida Amini, Alex Fabrikant, Eric Chu, Xuanyi Dong, Amruta Muthal, Senaka Buthpitiya, Sarthak Jauhari, Nan Hua, Urvashi Khandelwal, Ayal Hitron, Jie Ren, Larissa Rinaldi, Shahar Drath, Avigail Dabush, Nan-Jiang Jiang, Harshal Godhia, Uli Sachs, Anthony Chen, Yicheng Fan, Hagai Taitelbaum, Hila Noga, Zhuyun Dai, James Wang, Chen Liang, Jenny Hamer, Chun-Sung Ferng. Chenel Blkind, Aviel Atias, Paulina Lee, Vít Listík, Mathias Carlen, Jan van de Kerkhof, Marcin Pikus, Krunoslav Zaher, Paul Müller, Sasha Zykova, Richard Stefanec, Vitaly Gatsko, Christoph Hirnschall, Ashwin Sethi, Xingyu Federico Xu. Chetan Ahuja, Beth Tsai, Anca Stefanoiu, Bo Feng, Keshav Dhandhania, Manish Katyal, Akshay Gupta, Atharva Parulekar, Divya Pitta, Jing Zhao, Vivaan Bhatia, Yashodha Bhavnani, Omar Alhadlaq, Xiaolin Li, Peter Danenberg, Dennis Tu, Alex Pine, Vera Filippova, Abhipso Ghosh, Ben Limonchik, Bhargava Urala, Chaitanya Krishna Lanka, Derik Clive, Yi Sun, Edward Li, Hao Wu, Kevin Hongtongsak, Ianna Li, Kalind Thakkar, Kuanysh Omarov, Kushal Majmundar, Michael Alverson, Michael Kucharski, Mohak Patel, Mudit Jain, Maksim Zabelin, Paolo Pelagatti, Rohan Kohli, Saurabh Kumar, Joseph Kim, Swetha Sankar, Vineet Shah, Lakshmi Ramachandruni, Xiangkai Zeng, Ben Bariach, Laura Weidinger, Tu Vu, Alek Andreev, Antoine He, Kevin Hui, Sheleem Kashem, Amar Subramanya, Sissie Hsiao, Demis Hassabis, Koray Kavukcuoglu, Adam Sadovsky, Quoc Le, Trevor Strohman, Yonghui Wu, Slav Petrov, Jeffrey Dean, and Oriol Vinyals. Gemini: A family of highly capable multimodal models, 2024. URL BBLLLtttss///rv...rr////////////
Aaron Gokaslan and Vanya Cohen. Openwebtext corpus. @0peneebTextCorpus 2019.
Akshat Gupta, Atahan Ozdemir, and Gopala Anumanchipalli. Geometric interpretation of layer normalization and comparative analysis with rmsnorm, 2025.
Wes Gurnee, Theo Horsley, Zifan Carl Guo, Tara Rezaei Kheirkhah, Qinyi Sun, Will Hathaway, Neel Nanda, and Dimitris Bertsimas. Universal neurons gpt2 language models. Transactions on Machine Learning Research, 2024.
Stefan Heimersheim. You can remove gpt2's LayerNorm by fine-tuning. In NeurlPS Workshop: Interpretable Al: Past, Present and Future, 2024.
Stefan Heimersheim and Neel Nanda. How to use and interpret activation patching, arXiv preprint arXiv:2404.15255, 2024.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 448-456. Lille, France, 07-09 Jul 2015. PMLR. w.t.press/v3//.. v htt////ffffe...hcec
Shahar Katz and Yonatan Belinkov. Visit: Visualizing and interpreting the semantic information flow of transformers. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp.
14094- 2023.
Connor Kissane, Robert Krzyzanowski, Arthur Conmy, and Neel
Nanda.
Sparse autoencoders work attention layer outputs, Jan 2024.
URL
15

---

Published as a conference paper at ICLR 2026
Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. On the biology of large language model. Transformer Circuits Thread, 2025. URL pub/
www.ooreeerrrrrrieeeeee circuits
graphs sizaphs/biolgyy www.ooyo.ooooo..ooo..oo.oooooo.. html
Samuel Marks, Can Rager, Eric Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. arXiv preprint arXiv:2403. /9647, 2024.
Callum McDougall, Arthur Conmy, Cody Rushing. Thomas McGrath, and Neel Nanda. Copy Suppression: Comprehensively Understanding an Attention Head. arXiv e-prints, art. arXiv:2310.04625, October 2023. doi: (0-485500arxxi22311..42222
Kevin Meng. David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associ- ations in GPT. Advances in Neural Information Processing Systems, 36, 2022. arXiv:2202.05262.
Noa Nabeshima. Tinymodel: A tinystories Im with saes and transcoders, 2024. URL https Accessed: 2025-05-15.
//github. catchuu..om//ooeee
Neel Nanda.
Attribution patching: Activation patching at industrial scale, Mar
Ncel Nanda. Exploratory analysis demo http:////wc.ottttttt/////ooaeeee http://awaa..aa.aaaa.aaeaaaaaaaa
wsspyaAnaallllsl_S_aa ysss__ems_________el_ 1pynb 2023b.
Neel Nanda and Joseph Bloom. Transformerlens.
https://github.cm
2022.
nostalgebraist.
interpreting gpt: the logit lens, Aug 2020.
URL
https://www. /wwp:///wwww.// al
htwp:ummorrrossooeeeeeeeoo http://wwww..oooppss/s/ceeeeeeee interpreting gpt the the- logeellogieeeeee lens
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Sermadett-Shapiro. Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu. Joost Huizinga. Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, tukasz Kaiser, Ali Kamali, Ingmar Kanitscheider. Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Lukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning. Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David
16

---

Published as a conference paper at ICLR 2026
Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko. Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk. David Mély, Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo. Hyeonwoo Noh, Long Ouyang, Cullen O' Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng. Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong. Tolly Powell, Alethea Power, Boris Power, Elizabeth Prochl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder. Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher. Felipe Petroski Such, Natalic Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayveriyya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang. Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan. Wojciech Zaremba, Rowan Zellers, Chong Zhang. Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang. William Zhuk, and Barret Zoph. Gpt-4 technical report, 2024.
Judea Pearl. Direct and indirect effects. In Probabilistic and causal inference: the works of Judea Pearl, pp. 373-392. 2022.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106,
2021.
Alessandro Stolfo, Ben Wu, Wes Gurnee, Yonatan Belinkov, Xingyi Song, Mrinmaya Sachan, and Neel Nanda. Confidence regulation neurons in language models. arXiv preprint arXiv:2406.16253 16254, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023. URL URL htttt:::/// //ap::://aa http///axvvv..o org/ abs/ 2202. 11:002.1399 13971
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017
Jiaxi Wang. Ji Wu, and Lei Huang. Understanding the failure of batch normalization for transformers in nlp. Advances in Neural Information Processing Systems, 35:37617-376300 2022a.
Kevin Wang, Alexandre Variengien, Arthur Commy. Buck Shlegeris, and Jacob Steinhardt. Inter- pretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint w:Xiv:2211.00520 2022b.
Eric
Winsor. Re-examining layernorm.
Alignment Forum, 2022.
URL
https: www. www.aaaiggnnnnnonom forum. httg///osss////vv/////o////// re- www.mmmiiiinnn-ayyyyooooo
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming
language models with attention sinks. Advances in Neural Information Processing Systems, 37, 2024. arXiv:2309.17453.
Itay Yona, Ilia Shumailov, Jamie Hayes, Federico Barbero, and Yossi Gandelsman. Interpreting the repeated token phenomenon in large language models. arXiv preprint arXiv:2503.08840 2025.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
17

---

Published as a conference paper at ICLR 2026
Biao Zhang and Rico Sennrich. Root mean square layer normalization, 2019.
Fred Zhang and Neel Nanda. Towards best practices of activation patching in language models: Metrics and methods, arXiv preprini arXiv:2309. 17X1:22399.1042,, 2023.
Jiachen Zhu, Xinlei Chen, Kaiming He, Yann LeCun, and Zhuang Liu. Transformers without normalization, 2025.
18

---

Published as a conference paper at ICLR 2026
Appendix
A CODE AND MODEL AVAILABILITY
The LN removal code is available on GitHub removing- layenggg-lyymmmmmmmm
httts://github.com wttp:luw..ooooooooouoooooe rat
Table 2: Hugging Face links for models used and generated in this manuscript. fine-tuning (FT) steps and GPU hours are shown.
Furthermore,
Model
FT Steps
FT GPU Hours
Link
GPT-2 Small original
N/A
but /hugg Cace copopenai commosityy/pt.
GPT-2 Small vanilla
300
butgas /huugg//uuggggg Care 2 anall
GPT-2 Small LN-free
300
1.5
hctpes/ /ngpge//tog..gt........ http://pww.......cce/e////////e/
GPT-2 Medium original
N/A
GPT-2 Medium vanilla
500
2.5
bctpa: httppss/hhuggggaaae httpp///g....aaaao..cccaaaaaeeea http:://wwaa..aaaacoeo////////ts
GPT-2 Medium LN-free
500
3.5
h/togs//hogg" nigh htt.
GPT-2 Large
N/A
bctpa: hctpas//hagginotacee http:://.........o.oooooooooooaa
GPT-2 Large vanilla
600
6.5
bet pos //hugg Face, wtt.//////caccoooo wttp://///cche////////2./
116603
GPT-2 Large LN-free
600
8
bot bott ng wwtpcch/e////cgppccccecc Large
GPT-2 XL original
N/A
httpo:/ http:////w.gg.......ooo
GPT-2 XL vanilla
800
14
bulpps wwg.aggggggggooaoooe htgp
GPT-2 XL LN-free
800
26
bukpps / /hwggggggggnn Cace wttpaa/www.ccaaeeeeeeeeeeeeeeeee Cttpchhhhhhheettteeeee x1_1NP sw000
Other Compute Requirements: The evaluation and interpretability experiments require negligi- ble amount of compute (on the order of few GPU hours).
19

---

Published as a conference paper at ICLR 2026
B BLOCKWISE LN-REMOVAL SCHEDULES
All schedules use a warmup phase, cosine learning rate decay schedule, and continue fine-tuning for some iterations after LN removal is completed. Recomputation and auxiliary loss applied to all schedules. The removal steps in the schedule are configured by start, gap and number of layers hyper parameters Tab. See Tab. for how these affect the final schedules.
Hyperparameter
Small
Medium
Large
XL
Original GPT-2 model
gpt2
gpt2-medium
gpt2-large
gpt2-xl
Micro Batch Size
32
22
18
Gradient Accumulation Steps
16
23
18
28
Batch Tokens Per Step
524288
518144
516096
516096
Weight Decay
0.01
0.01
0.01
0.01
Learning Rate
0,0006
0.0006
0.0003
0.0001
Min Learning Rate
0.0003
0.0003
0.00004
0,00002
Aux Loss Weight
0.1
0.1
0.03
0.01
Gradient Checkpointing
true
true
false
false
GPU memory
80GB
80GB
180GB
180GB
Number of Layers
12
36
48
Warmup Steps
25
10
15
20
Max Steps
300
500
1200
1200
Start LNMLP
20
20
30
50
Start LN
44
68
174
242
Start LN,
68
116
318
434
Start LN,
104
188
534
722
Gap LNMLP
2
Gap LN
2
2
4
4
Gap LN,
3
6
Table 3: Comparison of GPT-2 Small, Medium, Large, and XL LN-free Hyperparameters
Small (12 layers)
Medium (24 layers)
Large (36 layers)
XL (48 layers)
Step
Removal
Step
Removal
Step
Removal
Step
Removal
20
LN
20
LN
MIP
30
LN LN
ML.
50
LN
MLP
22
IN
22
IN
34
54
LN
MLP
42
LN
66
LN23
170
LN35
238
LN47
MLP
MLP
44
LNO
gk
68
LNO LN
gk
174
LNO
gk
242
LND
ok
QK
46
IN
gk
70
gk
178
LN
gk
246
LN
qk
66
LN
114
LN
314
LN35
430
LN47
qk
4k
qk
68
LND
116
LN
318
IND
434
LND
71
LN!
119
LN!
324
LN
440
LN
V
101
LN
185
LN
528
LN35
716
LN
Final
104
LN
188
LN
534
IN
722
LN
Table 4: LN removal schedule for GPT-2 Models (Small, Medium, Large, and XL). Values correspond to fine-tuning steps when particular LN is removed. Gaps between removal events are uniform within each LN group.
20

---

Published as a conference paper at ICLR 2026
B.1 EMPIRICAL GUIDELINES FOR HYPERPARAMETER SELECTION
Based on our empirical observations during the development of the LN-removal procedure, we identified several common failure modes and here provide effective mitigation strategies:
Exploding loss after removal events: If the main loss spikes dramatically shortly after removing a LN module, this typically indicates that the removal schedule is too aggressive. The most effective mitigation is to increase the gaps between removal events within each LN group.
Loss degradation after complete LN removal: When the loss begins increase after all LN modules have been removed, we found it beneficial to increase the minimum learning rate of the learning rate schedule.
Sudden failures at some fine-tuning step during the query/key path LN removal schedule: In this specific case, we found it beneficial monitor standard deviation values for drop replacement, modify the EMA smoothing factor, change the learning rate, and change the auxiliary loss weight.
These strategies proved effective for achieving stable LN removal across all model sizes. Additionally, the auxiliary loss and EMM estimation significantly helped in reducing sensitivity to hyperparameter choices compared approaches without these components.
B.2 SUBLINEAR SCALING OF FINETUNING DATA REQUIREMENTS
Analyzing the total number of tokens used during LN removal across different GPT-2 variants (Fig. T and T it is possible to observe sublinear scaling of fine-tuning data requirements with model size. From results:
GPT-2 Small (124M params): 157M tokens (300 steps 524k tokens/step)
GPT-2 XL (1.5B params): 413M tokens
880 steps x 516k tokens/step)
This shows 12x increase in parameters required only a 2.6x increase in fine-tuning tokens.
21

---

Published as a conference paper at ICLR 2026
C GENERALIZATION BEYOND GPT-2 FAMILY
To evaluate whether our LN removal methodology extends beyond the GPT-2 model family, we conducted additional experiments using Pythia 70M (Biderman et al. 2023). We compared two fine-tuning approaches on The Pile dataset: vanilla fine-tuning with LN intact, and our proposed fine-tuning strategy where LN is removed.
The results demonstrate comparable performance degradation to our GPT-2 findings. The vanilla fine-tuned model achieved cross-entropy loss of 3.80 on the TeenWeeText evaluation set, while the IN-free variant achieved a cross-entropy loss of 3.89. This 0.09 increase in loss aligns closely with the performance gap observed in GPT-2 Small (124M parameters), indicating that our methodology exhibits consistent behavior across different transformer architectures. On the HellaSwag benchmark, the original Pythia 70M model scored 0.2679, the vanilla fine-tuned model achieved 0.2667, and the LN-free model obtained 0.2636, showing minimal performance degradation on this alternative evaluation.
These findings provide evidence that our LN removal approach generalizes beyond the specific GPT-2 architecture to other transformer- based language models.
22

---

Published as a conference paper at ICLR 2026
D RELATIONSHIP BETWEEN LN AND RMSNORM AND GENERALIZATION TO RMSNORM MODELS
R.M.Nomm is defined
RMSNorm(x) x
7
(8)
H
h=1
As such LN and RMSNorm are nearly equivalent operations, differing solely in three aspects:
LayerNorm includes a bias term B, which can however be "folded" (i.e., absorbed into the parameters of an adjacent linear operation).
Subtracting j before rescaling, LayerNorm effectively removes the (1, 1, l,... component of the activation vector.
LayerNorm rescales by the standard deviation of the activation component values rather than the RMS. However, after removing the (1, 1, ..... projection, rescaling by the standard deviation is equivalent rescaling by the RMS up factor of / which well be "folded" into adjacent parameters.
Since LayerNorm is applied before every model component and before the unembedding. the net effect is that LayerNorm models do not utilize the 1, I,....) dimension of their residual stream and apply RMS normalization on the remaining dimensions. For further discussion, see Gupta et al. (2025). Given this near-equivalence, expect our procedure to generalize models using RMSNorm.
23

---

Published as a conference paper at ICLR 2026
E
THE PILE-FILTERED
When evaluating models on the Pile (Gao et al. 2020 we observed unusually high cross-entropy losses for specific tokens. To investigate this, we compared token frequency distributions between 1 million samples from this dataset and Open WebText (Gokaslan & Cohen 2019) both pretok- enized with GPT-2. We identified tokens that appeared The Pile but not in Open Web which corresponded to sequences with high cross entropy loss. We filtered out sequences containing any of these tokens, and created a small 10,000-example filtered subset of The Pile. The filtered dataset, along with token metadata and generation scripts, is made on the Hugging Face Hubhttpa https //huggingface. co/ dataset s lucabaroni/ http/lpplllloooo wwteeeeeeeeeeeeeeeee-eee
token id
token
count
197
\t
4,260,185
628
\nin
1,382,601
1849
Axa0
1,090,135
201
\r
725,891
191
Ax03
50,457
200
Ax0c
49,412
5624
Axa0
40,045
4603
\xa0Xxa
9,374
205
Xx11
5,169
203
Ax0f
4,177
Table 5: Top 10 most frequent tokens present in The Pile and missing Open WebText.
E.I GPT-2 XL LN-FREE HIGH Loss SAMPLES ON THE PILE
We reported a very high mean CE loss (130.22) for GPT-2 XL LN-Free on The Pile. However, the median and 99.9 percentile range are very similar to GPT-2 XL original. Three samples are responsible for the high mean CE loss for GPT-2 XL LN-Free on The Pile. We list these samples below. These samples contain token token sequence At such tokens, the model has absurdly high CE losses, up to 5 million, i.e., the model is overconfident that the true next token will not be the next. For the three samples, the first token prediction with CE loss larger than 50 are "IxOc", "It", and "'n" respectively. The last token of the sequence leading up to the token with high errors "In" for all three samples, indicating that these specific tokens and token combinations are causing overconfidence in the model. Further inspection reveals that these high CE losses derive from very large negative logits. These outliers occur because, in rare cases involving certain tokens not present in the fine-tuning dataset, the norm of residual stream vectors before unembedding explodes, Interestingly, we observed this phenomenon only in GPT-2 XL.
present in OWT and are listed in Tab.
Sample 1:
Sample 2726 out at has tokens with loss 50,
First taken with CE loss > 50:200 at position 11. Decodad:"
Decosed (unicado_oseeet cecccodd_occseeii 'xce"
Bequence of last 5 Tokena for pradiecion:220 220 220 1357 198 11
Docodod:'
Decodad (unicoda_eacapoo."" 11Aa
220:8/A, 220:7.6214733, 220:7, 388017, [Token:Lossi
220:0.21067815, 220:0. 22010.0828788, 07294927
220:0.06P81219, 1367:9.5636, 19813.8313624, 203:38.273285, 42138:11.611189, 250:5.8252113, 31839, 9021:17.
286:5.927439, 9480:9.354071, 972:3, 4310:4.679943, T61:10. 407:0.808:4852, 307:0. 3940:4.7689095
'Specifically we used
paapLoogy-ple--unoyyyiioo 'Specifically
we used
24

---

Published as a conference paper at ICLR 2026
12:1.346435, 91.90:9.200DE2,
2173:7.3507837,
B03:0.88767885,
2011
428:2.462277
19811.3526363,
198:0,00169402E,
7662:1.6574776,
11:0.8758041,
262:1.
2069115.1991183
28622 38040:3.
19303:9,20058, 2657:4.18554, 3199:1.149225,
1682:8.386018, 2058:0.2735605,
007:3.1076966,
422:0.1759622,
198:2.1329598, 198:0.0001505,, 36208:13.958035, 4537:22.094295, 16412:8.
475:3.527892, 422:0.22476129,
262:0.25115574,
3892:8.823268, 17941:5. 4260:11.620002, 2799:5.924607, 180.3.
27159:10..149113 2192:5.48372'5,
286:1.2630095,
8235: 0235:9.437175,
357:1.502765,
48716.1692253, 27055, 39:5,992879,
106.15.501152,,, J83816.2311013,,
447:4,11596,
251:0.2123165, 828:5.8962223,
198:3.1704738, 198:3.05913,
1187:11..422755
13:2. 4C6:8.
13:4. 501826,
1403:6.3063745
13:2.2845055,
14436:11.
12:9.5589272,
26192:13.
11:229189519,
49171:9.920289,
8949:10.02035,,
15:24452814,, 19143:3.181636,
13:2.0333782,
28229:9.
11:1.51247E2,
9786:9.335802,
5183:8.786331,
55500122 27622:3.355222,
11:1,7278297,
27937:13. 612004,
22:11.208633,
1540E:1.:2247,,
1160:8.236332,
466::00612097,
357:3.7660804, 12288:7.9838665
828:5. 7518573,
542:3.
199:4. 198:4.0900915,
198:0.052418C5, 1829:12.192535,
22:22277, 1115:7.43645, 13788:11.599928,
2758322, 861757.3080463,
60578, 13793:10, 22312:10.30144 301445,
11:1. 411.5555,,
262:2.8934842, 18628:10.360179
251111,
622777.2655045, 11:2.16625088
280:2,5125C53 198:4.63<0384,
15:22.3188238
1169:5.9166457, 5094:8.
6.90:6.
(809:6.
25:4.863468, 628:3,801043.
8.:651
220:3. 819473,
220:4,2062297,
220:4,961951.
220:4. 995181, 220:5.
382:3.
4988:9.8524C3,
5059075, 6414:9.438324, 351:3.
2665:7.610552
1403618. 626756, 286:4.936455.
25212.6956165,
3872:3.572085,
6275:6.6044493
4347:11.077166, 1799:6.
4776774 198:3.295391,
220:5.0134106.
228:6.064687,
220:5.9405313,
220:4. 2.2479,,
223:3.2302897,, 220:2.
155347
2:04,,,
286:3.
$235:10,5037035 11:2.1030743,
743:7. 743:2.7276053,
13040:13.952915
10494:
9E4:1558. 5077,,
6647:2454. 7053, 353:082.05888 .5588, 743:1939.8294, 307:1388.4974,
3805:2580.665,
198:469.1405,
220:
220:1138.9397,
220:1082.
220:1259. 226:1304.2882.
383: 988228, $038:2638..102,
286:456.9895,
3283:2395.4785,
503:1655.8503
262: 63543, 8617:3220.733,
238:830.44545,
(28:1525.5525,
995:797.9059,
3911:4231,72,
4093:2158. 9766, 289; 4985:2501. 727,
743:1726.
1982450.30127,
220:1403.6113
220:2175.758,
287,
220:1938. 5784, 220:1560.34S1,
220:1166,444,
38040:4204,648
16494:8649.662,
587:1345.0178,
19503:3773.4755
2457:2025.
3173:1838.
353:568.34094,
262:590.882 8824, 4986:2062.4525
13847:2279.
389:1038.8554,
5038:2869.269,
2801779.
198:396.6925,
220:881.5336,
220:1608.3259,
220:2106.2659,
220.1566..009,
220:1546.5249,
220:1340,,75,, 3283:2450.
508:1587.5262, 428:13:8..421,
685:892.65913, 3911:
4083:2057.4734
188:216.41724,
198:246.9541, 19E9:2581.
471:1104,4808,
13:41.819535,
50:1369.8, 13:150. 74365, 54:1408.8376
13:1 64563,
5480;2291,254,
15136s' 765,
16:2138.022
66:190" 7491, 11:178.259, 2608:
471:1437,7157,
Decuded::
ablicw and comment proceduses of $ 553 need not be followed. PlainLiCr puinta put Lba. in this
case, the statutory aut autharity to pronulgate interin final rules astuallly comes not from the RFPAFA but Fron the Health Insurance Portabil lity and Accountabl lity Act of 1996 ("HPPAA",, Pub. L. No. 104-191, 55 101, 102, 402, 110 Btat. 1935, 1851, 1976, 2082 (1996), which incorgorated threw sall adention ical provisions into ERISA, the Internal ReowEuu Codee and the ieblic tealth Service Act:
The Secretary, consistent with section Accountability
the Hessthh Care Portability and 1996, may promulgate suct regolations may necessary or appropriate to carry ous the provisioas this (part). The Seuretary may promaigste any interin tinal rules as the Secretary deternines are appropriate to out this [part).
29. U.S.C. 1191o, 25 U.S.C. 3 9833 treplacing "part" with "chapter"), and 42 U.S.C, 300gg-
iseplacing "part" with "auschaptor").. 4 Pleintict argues that Congress only intenced give the Secretaries authority to pace Interin final rules relating to HTPAA and not the MIPARA, which was pessed txelvv years lster, Rovever, the HIPARA's substantive previsions amencrents to the same sections at BRISA, the Internal Revenue Code, and the Public Reaith Service Act
HIPM provisions cited abave, and statutory ciearly giver the Secretaries authority to premolgate interin final, rules to earry out these sections. Theretore, the Court tinds that Congress has authorized the Secretories to "promuigste any interin tinol rules as thety! [ctormine[] are appropriate to carry cut the" REFAEA, Finding thst Congress authorized promulcation interin final rules does not end the inquiry, Althouga the APA recogrizes that Congress may nodity the notice and comment
governed
This roqulatory authorsty covers part of Subtitle 2 of Title I of 2RISN 129 U.s.c. ss 1101-91c1, Chapter 100 or the Internal Beveaue Code (26 u.s.c. 25 9801-33), and Part A or Title XIVII the Publle Bealth Service Act U.S.C. 300g0 to 30009-92).
12
procedures calied for by 5 353, states "[s]ebsequcat statute not be held supersede oz nodicy [8 553] except to the extent that it does so exprassly." U.S.C. & 539. "[r]he import of the 539 instruction is that Congress'a iatent to substantive change cleas."
Acp'n of Data Processing Serv, orgs.. Inc. v. ad. of Bovernore, 745 2.2d 677, 686 ID.C. cir. 1985). The statutozy provisions authorizing interin final ruies in this do menticn notics
25

---

Published as a conference paper at ICLR 2026
and comment any othez aspect of the ABA. suct. case, the D.C. Circuit defined the relevant standard as "Mtether Congress has cstabliched procaduzes so clearly dictoren: from those
required by the APA that nust have intended to displace the nozn." Asiana Mirlines v. E&A, 134 P.3d 393, 397 (D.C. ciz. 19981.
aefendantl ooly on two cases in which the D.C. Cisouit held that the notice cumment provisions of 5 553 Mero abrogated by specitic statuzory provisions authorizing inzerim rinal reles. Bee Asiana Airlines v. Fed. Aviation Admin., F.3d 393 (D.C. ciz. 1998), Mechodist Sosp. of Baczamente v. Shalala, 38 P.3d 1225 (D.C. cix, 1994). In Methodiat Rospital of Secramento, the court was faced with
Sample 2:
Sample 1323 out at 108 has tokens with GE lose > 5C, Virst token with OF lass > 50:197 at position 10. Desosed:'
Decosed lunicode, escager:"
Sequence of last S. Tokens for ion:222 4251 5175 13 198 Decosed:' string array.
Recoded lunicode escege):" string arrey-In'
[Token:Loss]
IC03:N/A, 1003:11.26558, 9126:11.3620B2
46621:13,591033, 35515-12, 95421,, 4131:77 617999
3704,
198:1.6627543,
197:51-155723, 12235:16.
69:10.
7663:9.20375,
176311
8347:10.834765,
198:3.47483,, 92:13.36(931, 199:1.
9331817,,,
1903:3,
368:1. 2918,,9 3798:3.19929.,
178:2.765933
45356:11. 567278, 40109:11,
9860:9.337919, 25111 5190241,
649:1, 1564222 543:4.84
360:2.3216953,
207:3.686756,
973:0. 74535253,
284:3.0334966,
392073, 257:1.6750616, 581:7.6475534, 5171:5.633069,
198:3,4956757,
1902:1367115,, 29951882022033.
12:3.9833105, 3019, 5662:5.
314177.331666,, 13:2.6122014,
198:0.8083212
1303:15.
198:2.0173892,
1003:15, 395724, 2155015.244333. 25:0. 19102191, 210:2.1424259, 318:1.3225417
257:1.5655422
275:9.388821,
03:4.2324853, 10210:5. 020052, 7552:5.
19702:11 923C24, 822:0.8639222, 33084:C_970956,
68376416, 78519.2063/bb
375363783686,
1250119. 020502, 210461, 14:0, (16667,, 250:5.0 6433:4.456625,
6312:3.195785,
14:2.777247,
67:4.2537035, SC98:0.37573352, 17752:5. 198:2.2561002, 1903:12.218852
257:1. 936492,,
2639:8.357916, 9839:0.183291,, 6637:3.1122556,,
13:1.564305,
198:9.2961991,
20195:21.239:223,
338:1,6365192,
19:0. 024241818, 3199:2.4091002,
45J26:10. (3511,, 40.09:
762554, $96116,12136,
5996.5091047,
060755047564477, 1750515.3587077,
8641:1.850085
8:6.0505694, 273365,, 49:4.241515,
3755:2.2(25796, 272:1.7413552,
t536:8.1885225,,
40109:9.711792,
957003,
19823 197:35.356222, 7783:13. 6022213, 1222:7.56527A, 19:5.103052,
3198:12,716011
252:9,562133, 2336:12.330081
40109:13. 20:5, 1223515, 3258843, 39:13,279553, /A65:12.282539,
5354493,,,
2512:9.293197,
18:374,
7465:14.
T0633, 198:3.7612552,
52:9.5572633, 198:0.632967,
5126459, 22186:11.7562035
252299,
3419:595915.1, 1391:129406,
198:2:275.512, 197:436295.1, 1003:157350,20
9729:140387.66,
5875
628.79266.625, 2393:106
389151231.
691,73382.914, 24284:177729.25,
356:59283.35,
2639:149387.16,
11602:2190E4, 43, 13:6765.0703, 198:32494.742,
197:403424.3,
32152
15039:195013.9,
01::62211207,,
37:97575.016,
1225:110197.266,
1000:3488L..1, 34, 5439:269758.94 94, 19089:212632.28,
628:112185.05, 197:436578,7,,
1489414
36834:172249.12
60105:219531.65
9203:188346.6,
0.299:237815.0,
5344:139417,
1600:119173.45,
20789:137373.7,
47619:202613.2 5261:148838.85,
20109:222167.31,
2769:212062.98, 45991:286692.55,
828:93572. 62, 9701:161337.11,
8163298.71, 198:25669.29, 197:66331866,
34020:100055.65
38800:193890.8
10109:273160.
7203:120339.91,
2220:195155.9,,
17602:17922..3 78, 28455:165867.17,
160::77894,,
20789:196518.0,
8912:206197.88,
46047:205311.12,
22027:217080.91, 40109:232314.2, 5769:212539.55,
(5991:271713.0,
928:92536.35,
3101:131772.14,
9:71338,03, 198:5289.922,
187:479748.00 06, 34320:161961.17, 38804:183568388,
4C109:324803. 62, 1203:168302.4,
1562:152243.08,
1958:311552.6.
27372:174300.02
1600:175331.13 20789:198232.96,
2673:1214E1.22,
1958:134500.2, 45555:281647.2.
40109:218465.22.
Decoded:
Block hashes string array.
BlockBashes l]string
BewdlescanBuucaccdd reluins Instance which used Lasue
JSCN-BPC command.
/7
// HOTE: This is bled ont ported Cron github www///eeeeddddddddddee... Jaon. zequires websocket connection.
tune NesRescanBiooksCnd (blockHashes [latzing) #RascanBlockstnd !
sw. dlescaniBuukksnnn Blucklashhes blockleshees)
fune 1n {
// The commands in this file are only usable by wabsockets. flags :- CPWebaocketCnlly
MustRegiatercnd ("auchenticata', (rAuthenticatachal) (aill, flagst Must Regist wrtdrnnd ( (flsdtxfilseer,, ltterd (sill, flagsi
MualRegzattzzo (Phol Eyykookks" (Nllt ifyBlock (aill, flagsl
(Hltt tytheetrarsactinnenn Muxt RegixterCnd ("not fyrrofyrrwwwve wed", (XXos iyyllfeee owdCnd) (nil), flags)
Regiaterrnd (fnoti ranesctreneet
Inil), flage)
MaatReg Sabeernn (fhoLify coliifyapent ",
Cypoity (r11),, flaga)
Most Regi stwrčnd ("session", (nSessionCnd) (ni flagsi
Beg wrön (*stogrot fyblocks", (eStopNot iryllockaCedl (ail), flags)
Must Reglawaznd ('atopooti Eyneut canaccianaationn
(18topMoLi www.glloiiillltSsssseeeenno
(nil), flags)
Must Regi atwrrnd (*atognot i fyspert", " 14Stoplist i Ordy (nil), flegsi
Must Beg wrnnd ("stogrot fyrecwived", (xStoplltt Byyee (oddrn)) tnil), flags)
Must Regi aterrnn ("rescan", (sReecandns) inil), slage)
Must Regi wrdd Freceeblocke". (xRescenfl oocksldd (iil1, flagst
26

---

Published as a conference paper at ICLR 2026
Paschless Execution: Fithting Prasidential Lawlesprass
The first Dav days rolling my nav book, Paithlesa Exscution, have been axtilarating with fav shings gracitying and hunbling than the venderful revicw by one of my very favorites, 20 Rodia's own Boggz Bimea.
It has been uoliccing to see hov nany people really are ala:med-zather than indifferent, as I woreled--to the presiem of ranpant the ssparation pewars, which is threstened by President
prosidential lewlessness. People really do grasp Obana's i usurpation of the powers of the states and osler fuderal depar swally as the key to protecting our liberties. Too much accumiiation of power in ona government ofricial's hand-moat.cciaci the Pranars observed, the feining of the legislative and excoutive power in single department or personmris the road to tyrancy.
Whan peopla geaso that, they similarly greap that presidaacial Laxcessnesa not a consorvative veraas likeral issus, nor Republicon versus tencerat, It is a question of wacther we still appire to be a rapublie under the rule of law instead of aub jects undes powsidetiaa whin, IC they ace not knocked down, the precedents that President Mhana is selling for imperial exscutive sovar will be availabla orlentationn That eught to frighten all Aner Americars, not just appoments of the current prexident's y policies
exploitation by every future president, zecardless party iddoiecical
I make a austained attengt in the book to explain that Liecachennt-tha ultimate cons:itutional respense to presidantial thousand ingeactabll offenses, but if there public will that the president senoved, impeaciment is nonstarter. The political case for senoval is the one that
laul
politicel renedy, not legal
strong
aphill, Bstablishing the isqal case for inpascimatt LLe., denonstrating that hich crimas and misdemaaeess have been cond is the eany part.// The label and act iora expect to be in flex container. Sincw this curgonent adds another
1/ veapping layar to the ndc-shackbar purface, it should also insludo flax display.
ans uds simple snack bar { display: flox;
"IL has like the Alane at tinas, Nothing vent. Zoz ca. It fools like we have lost but the final is over tvo lega and we have to be dell dighthhed with the overall lne.
Liverpon1 tirst-tean captain Breven Garrard and central dafundar Janio carragher ware quickly touch after the win and Baighvay added; "They have foll lowed us all the way through.
"They cexted before every game they have texted again after
"They acu steeped in the history of this club and know what it neans Lo win this Loccommett."
City's acedany chief Jin Cassell
Sample 3:
Sample 9535 out at 10k has tokens with GE lose 50, First token with CE less > 50:199 at position 155. Decoded:'
Decodad jesicodd_eccasee:" laa
Sequenee oz Last 5 Tokens for (r9d 19704 9705 20379 198
Decodad: (2:25)::////1111111111
Decosed [unicode_escsse)
[oose:Loass
407:8/A, 407:4.6931055, 162(:7.360232,
326:1
380401,
2030:7.6741133
262:1. 2626:4,106626,
3788:4.8768866, 13:1.0567981, 1002:2.57059,
345:0.35921186,
779-3.5662775,
428:2.6385352.
3788:0.4169199,
287:1.3137745,
257:0.
1720:1.4579158,
12:0.9270898,
213:3, 48:22:1.5712257,
2396925,
282:0.093141652,
1120:0_15413ED5
(0014:0.295766226
3015331,
072:0.
318:6.05636958,
067:55007545187,
2872:0.050823112,
12:043214
198:0. 7484702,
17:13,849214, 13:0.13205925, 978:7.044759,
6400:2.0508624,
2723:2.2134705, 635012.22331,,
127610.85757795,
42CELL,, 30722:1.
7498:0.11422599
E5::C.11422855
094:0.00481102, 11:0.3558381, 290:0008005334, 1276:0.
107:0.C8.67527,,
307:0.025236622,
(G22::0,1220072
376:0.
355:0.0029704511,
852:0.14899838, RC2019788,
2858:0.022728355.
2.055022,, 0310.043,7017, 198:0. 211312,,, 17:6.378
13:0.03099801334,
77010.5271791,
4003:1.
075377
CC7:0. 201329, CC7:0.201329, 307:0-013352291, 9611:0. 393:0.C392538, 1429400119510234,
122:0.C3159716, 397:0.C38671132,
0223:0.2433001 5032:0.28856535,, 13:0.014534266
198:0.0928015,
4476637
158:0.:0734685,
198:0,9003922977,
460277T, 4027766, 49T0L:0.
45700:2.
49TE6:1
45106:1.113343.
9483:009489326,
9705:2.
1705:2.115188,, 1151383, 14055:1
1015871,,
35343:20. 283399, 51550.5159568,
1836:10.64370.
197,30.43997,,
305223, 1299:0.67363411, 2418:3.
8119438, 2.038.17
756991,
6242:7.9582387. 33:0.28353172,
2927:11. 1227:11.29194,, 1386:2.9536972,
19:1.0574700,
01871,, 1135:12.6299985
197.33,
35:6.926921, 376977
560585, 197:31,852261,
3185: 746413, 34:4, 736834, 4475849,
5538:5.085874
32883:6,
69899
1004:0. 860001533, 3430.34 72:112452292,,
8194944,,, 183579.853144,
197632.
3913.1674745,
9800:5.4635477,
197:33.27188 197:33.442223..
38910128.7021G,
201859.
37757,, 21096:2.12277422
19810.78 1635:3.943309,
67.33.11875
100211, 1475:15.054752,
197659.80469 197:38.404465
21339:16.17667,
11:3,2231097,
352:6.5712805,
12.
17838578 2240:13.490033
31844,, 1191:1
7258412,,
49704:23. 5610522 4650472..66547,,
97744119.025005
49704:20.011
49704:21 655386, 49704;40. 67674 9705:22. 20379:27.
198:8.23015,
198:399.37865,
29704:808.
49704 74054, 19704:722.9033
49704:811. 80115, 49704:698.3036,
19704:730.5885 9705:677.8175,
19375,
3985,,
1002:409.E4636,,
40348:599. 6119, 4932:453. 198:66.265115,
93063, 361:344.89172,
4684423,6663,,
891 4477447.8888,
11593:46,
3185:463.
33:837, 42:323.4264, 3838:367,594,
35833:424.9655,
(E:187.75824,
3063:536.
41237:574.64325,
62:188.
29:61808. 834:1737813.5,
198:58819.
2:501469.34,
12086:475518.622
11593:430123.34,
5,88:436203.5, 34:149311.
62:225199.66, 3888:499326.16.
32853:46378.555
5,
865):506636.12,
41231:671244.25, 52:3301155.44, 39:505198,0, 8, 834:412493, 628:283896.55,
197:1610916.9, 9249:487438.94,
640:274647.44,
5862:508110,75,
18320:E01000.5, 62:200508.23, 17514:568054,25,
317;109115.234,
6242:375E80. 62, 2749:538623.9
1787766766
1058:368174.3.
147014455588,, 30582
196:122744225,
22777727,,,
10:353159.03, 03, 198:107419.88.
187:1258664.6, 197:1605486.2,
197:1546371.4, 197:1209663.6,
197:1201286.6,
197:11184335,
3838:524715.44
33883:448043.62
4891:
3419:4D4699,94, 94, 1058:314273.58, 12301:389488.2,
Recoded:
must not clain that wrote the original soltware. If you usw this softvare in product, acknool acknovledgnent, in the product cocurentation would be appresiates but is not required. 2, Aeree source versions most be pi ainly marked as such, and most not be misrepreeented as being the original softxare.
27

---

Published as a conference paper at ICLR 2026
notice may not or from any disteibation.
Concains code for an AABB collider.
Visic GPC_MMBBco111der
Yaushax PiNzem Terdinan
idace Januszy, 1st, 2002
Inglude Guard
Vitndef -PC_AAABCOLII
Edatine -FCC.MMBBCCLIIERRRLLL
struet APC ValuneCache
(MBBCache() (1
SusBox.nContor.eee (1)
Fal
I/ Cachidd fages signelure
CollisionkaaB Fat3ox) //1s Box used when pazforning the quezy resulting in cached faces // User sett inns
flow: FalCueEf; 1/1x aladius2 multipline used Cal spleee
class APC AABBCollide: public Volumecollider
public:
1/ Conatiuctoz /. Deatiactos
AABBCoo Lider
virtual
AARBBCol ()
Generic onl sitinn query for generic OPCODY models. After the call, access the rexu with Bl
reekrrecchesPrinttsves TerTachedPriniitieee
Aparam cecne [in/out] box cache
Aparam box
[in] collixion AAMI in world
Aparam [in] Spcade
Areturn true success
lvarning ing SCA-P NCT эиРPоRтт.. The natrices mist contain rotation & translation in parts only.
2001
1011ecc cache, const Co111910mA8886 box, ccast Model6 model);
17
CoLiicceh cachc, const Collisionta25s box, ccast MBBIISG (226)
protected:
Calonnnns mñoo;
//t Query box in Icerter, excents) form
Foint
//36
mMax;
Query
point
1/ Teaf description
Foint
nLeazverts(3]; //14 Triangic vertices
1/ Internal
nethode
AAMBCo
nodel;
vold
Sollldd Iconst MMBBNcLeatticde nodel]
vold
Collide Loonet ieaeQuantizedllodo nodel;
_ollldd
AMSBQuantiieddkceeannn
MMBT7ecNode aodel
rold
CoLLideNoPrimii tiseeess (const MMBCoolieeirntooee node);
vold
collideMoPrinit..eo tivetes (const MMBBNozaaflldde node
28

---

Published as a conference paper at ICLR 2026
F EVALUATION ON STANDARD BENCHMARKS
We additionally evaluated our models on four widely used benchmarks: BoolQ CClrk et al. 2019). HellaSwag (Zellers et al. 2019). PIQA (Bisk al. 2020), and WinoGrande (Sakaguchi et 2021 These tasks assess general language understanding, commonsense reasoning, and pronoun resolution. Tables report normalized accuracy for each model before and after LayerNorm removal. LN-free models maintain performance comparable to their baselines, with only minor variations across tasks.
Task
GPT-2 XL original GPT-2 XL vanilla FT
GPT-2 XL LN-free FT
BoolQ
61.8
62.2
61.9
HellaSwag
50.9
49.8
48.8
PIQA
70.5
70.5
69.9
WinoGrande
58.3
57.5
56.1
Table 6: Accuracy of GPT-2 XL model variants on BoolQ, HellaSwag, PIQA, and WinoGrande.
Task
GPT-2 Large original
GPT-2 Large vanilla FT
GPT-2 Large LN-free FT
BoolQ
60.5
62.1
62.0
HellaSwag
45,4
43.4
42,8
PIQA
69.2
68.7
69.3
WinoGrande
55.3
56.2
54.6
Table 7: Accuracy of GPT-2 Large model variants on BoolQ, HcllaSwag, PIQA, and WinoGrande.
Task
GPT-2 Medium original
GPT-2 Medium LN-frre FT
BoolQ
58.6
59.9
HellaSwag
39.4
37.4
PIQA
66.4
65.6
WinoGrande
53.1
51.5
Table 8: Accuracy of GPT-2 Medium model variants BoolQ, HellaSwag, PIQA, and WinoGrande.
Task
GPT-2 Small original
GPT-2 Small LN-free FT
BoolQ
48.7
52.0
HellaSwag
31.1
30.2
PIQA
62.5
61.4
WinoGrande
51.6
50.9
Table 9: Accuracy of GPT-2 Small model variants BoolQ, HellaSwag, PIQA, and WinoGrande.
Overall, these results show that LN-free models maintain comparable performance standard benchmarks, supporting their use in interpretability studics.
29

---

Published as a conference paper at ICLR 2026
G DIRECT LOGIT ATTRIBUTION BECOMES EXACT
To quantify the discrepancy between Direct Logit Attribution (DLA) and the Direct Effect (DE), we compute the Normalized Mean Absolute Error (NMAE) through two-stage process. For each attention head in the model, we calculate:
NMAE, EE DLA, -EEM EE=IDE |.EA
100%
where, i indexes individual sequences in The Pile-Filtered dataset, / indexes attention heads, DLA,, h and D E are the corresponding DLA and DE values for sequence and head We then average across all attention heads to obtain the overall NMAE:
Nhenda
NMAE
NMAE₁
(10)
Nheads
Am
The original model exhibits an NMAE of 49.07% [29.92%, 66.10%) (95% Confidence Interval CT), indicating that Direct Linear Attribution (DLA) estimates deviate from direct effect measurements by approximately half of the true effect magnitude on average across all attention heads. The vanilla fine-tuned model demonstrates an even larger discrepancy with an NMAE of 57.85% [38.52%, 7..52%]. In contrast, the LN-free fine-tuned model achieves a perfect 0.00% [0.00%, 0.00% NMAE, empirically confirming that removing the non-linearity introduced by LN climinates the discrepancy between DLA and direct ablation methods. This result validates that without LN's non-linearity, the two attribution methods are mathematically equivalent, eliminating the need for linearization approximations, which can be significantly inaccurate.
To ensure our overall NMAE metric was not biased by a few significant outliers, we visualized the per-head NMAE values across all attention heads in GPT-2 Small models (Figure 5). In models with LN, the disagreement widespread across most attention heads rather than driven primarily by a small number of outlers. Later layers showed NMAE values exceeding 100%. contrast, the LN-free model shows no disagreement between both methods, with NMAE values of zero across all heads.
Per-head NMAE (%) between DLA and Direct Effect
baseline
finetuned
noLN
150
10
Per-head NMAE (%)
100
Layer
4
0
5 10
5
10
5
10
Attention Head
Figure 5: Per-head NMAE (%) between DLA and Direct Effect across all attention heads in GPT-2 Small models: baseline (left), vanilla fine-tuned (middle), and L.-ffree fine-tuned (right). Significant deviations occur across most attention heads in models with LN. The LN-free model shows no difference across heads, demonstrating that DLA and DE are equivalent.
30

---

Published as a conference paper at ICLR 2026
H CONFIDENCE NEURONS
As mentioned in Section 5.4 confidence neurons exhibit two key characteristics: (a) high weight norm, implying importance despite weight decay regularization, and (b) approximately constant contribution to all next token logits, suggesting minimal impact on token prediction. These seemingly contradictory characteristics matrix We. The effect of confidence neurons on output logits is mediated by this normalization, mechanism absent in LN-free models.
reconciled by the final LN, between Wout, and the unembedding
These neurons regulate confidence by writing high-norm vectors that project onto an effective nullspace the unembedding matrix. When these vectors increase the residual stream norm, the final LN scales everything down uniformly, making the output distribution more uniform while preserving token rankings. To identify (b), neurons that preserve token logits ranking, we followed Stolfo et al. (2024) and calculated Logit Var( Wout,)), the variance in the normalized projection of the neuron s weights with each token the unembedding matrix:
WUWaut,
LoritVar(Woan.)) Var [ [ulldim= [Wou,ell
(11)
where WWllddm=1 is the vector of norms of columns of the We matrix. Confidence Neurons (CN) maximize the ratio of (a) and (b):
Wout.
CN(1) Logit (Won,)))
(12)
Figure summarizes CN identification in both GPT-2 Small and GPT-2 Medium models: the same identical set confidence neurons persist as across all model variants (we chose to highlight the top-7), including LN-free models where their theorized mechanism of action is absent. These neurons maintain their characteristic high weight norm and low logit variance signature despite fine-tuning and even the removal of LN.
Small
Normal
Nomral
10
Entropy
10
Encropy
Entropy
Logivarte out!
10
20
10:4
10
12
[[W outi
[|W outi]
1| puti]
GPT2 Wedium original
GPT2 Medium virilla FT
GPT2 Wedium Ui-feee FT
Normal
Normal
Normal
Log ous)
10m4
20ml
10n4
10"
8
10
12
14
16
8 10
12
16
16
10 12 14
16
16
[|w_out]]
[|w_put]]
Figure 6: Identification of confidence neurons in GPT-2 Small (top) and GPT-2 Medium (bottom) across different model variants: original pretrained models (left), vanilla fine-tuned models (middle), and LN-free fine-tuned models (right). The same confidence neurons (highlighted in red) persist across all model variants, exhibiting characteristically high weight norms and low logit variance.
Having observed identical confidence neurons across all model variants, we next investigated whether their effective nullspaces modified by performing Singular Value Decomposition (SVD)
31

---

Published as a conference paper at ICLR 2026
each model's unembedding matrix. Figure shows the normalized singular values (solid lines), revealing similar nullspace patterns, though fine-tuned variants exhibit slightly smaller effective nullspace. The cosine similarity between top confidence neurons and singular vectors (dashed lines) demonstrates these neurons predominantly project onto the nullspace in all variants, with some non- negligible overlap in transitional regions where singular values approach zero. This may explain why our vanilla fine-tuned model has less effective confidence regulation when mean ablated. Interestingly, the LN-free model maintains an almost identical nullspace and cosine-similarity pattern to the vanilla fine-tuned model, despite having no ability to affect logit rankings.
0,200
0.200
GPT2 Small original
GPT2 Medium original GPT2-Medium vanilla FT
GPT2-Small vanilla FT
0.175
LN-fred
0.175
GPT2-Medium Ui-free
0.150
0.150
Nornaliee Singular Values
Normalized Singulan Values
0.125
0.125
0.100
0.100
0.075
0.075
0.050
9.000
0.925
0.025
0.000
0.000
720
730
740
750
760
900
950 1000
1010
1020
Singular Vector Index
Singular Vector Index
Figure 7: SVD of the unembedding matrix for GPT-2 Small (left) and GPT-2 Medium (right) across model variants. Solid lines show normalized singular values, revealing similar nullspaces across variants, though fine-tuning appears to make the effective nullspace slightly smaller. Dashed lines represent the cosine similarity between the top confidence neuron (584 for Small, 1083 for Medium) and each singular vector. These neurons predominantly interact with the nullspace in all variants, with overlap in regions where singular values approach zero in the fine-tuned models.
To test whether confidence neurons maintain their functional impact across model variants, we performed mean ablation on these neurons (similar to the total effect described in Stolfoo et al. (2024). and measured the resulting change in cross-entropy loss. Figure shows the absolute change loss when ablating the top-3 confidence neurons in each model. The original GPT-2 Small and GPT-2 Medium models exhibit substantial variation when these neurons are ablated. Without the context- specific LN scaling these neurons provide, the models predicted logit distributions significantly change. The vanilla fine-tuned models show reduced but still notable effects, suggesting these neurons less effective due our fine-tuning strategy. This reduced effectiveness may be related to the slightly smaller effective nullspace, though further investigation is needed to confirm this relationship. The LN-free models show almost no variation, implying that these neurons have no effective mechanism to impact final logits despite maintaining their structural characteristics.
To empirically verify that confidence neurons primarily work by modifying the entropy of outputs, we cumulatively ablated the top three confidence neurons in GPT-2 Medium across all variants. Figure illustrates the results. In the original model, ablating all three neurons decreases entropy by over 3% while changing cross cros-eettropy loss by only 0.1%-a a 30x difference in magnitude. The vanilla fine-tuned model shows a similar but reduced effect, consistent with our earlier observations of its slightly degraded confidence regulation capability. Again, the LN-free model exhibits no change in either metric. These results directly demonstrate that confidence neurons function by modulating distribution entropy through LN scaling, with minimal impact on which tokens are predicted, allowing them to regulate model uncertainty without changing token rankings. We also investigated whether cumulative confidence neuron ablation of GPT-2 Small vanilla fine-tuned model could yield identical CE loss and entropies to the AN-free model. While the entropies matched (approximately 2.785) when ablating the top-3 neurons, there remained an absolute difference of approximately 0.06 (2%)
32

---

Published as a conference paper at ICLR 2026
GPtl-Small
0.06
MM12-Melumm orignal
Small
GP12-Medium UK-teee
0.05
CE Losi
0.00
Niuuron SBA
hauron 2123
Neumn 2870
Neums 10R3
Neuron 3128
Neurcn 3746
Figure 8: Change in CE loss upon mean ablation of top-3 confidence neurons for GPT-2 Small (left) and GPT-2 Medium (right). The original models (blue) show substantial loss changes when these neurons are ablated, indicating their significant role in confidence regulation. The vanilla fine-tuned models (yellow) exhibit reduced but still notable effects. The LN-free models (green) show almost no change in loss when the same neurons are ablated, confirming that without LN, they lack the mechanism to directly affect output logits.
in CE loss, implying that the general trend of overconfidence in LN-free models arises from more complex mechanisms beyond simply disabled confidence neurons the final MLP.
80
LoGa
0.09
Change Entrope 1961
-1.0
-004
CPM2-Medium
-20
Change
-0.00
-0.00
Figure 9: Cumulative effect of ablating the top three confidence neurons in GPT-2 Medium. Left: Relative change in CE loss. Right: Relative change in entropy. The original model (blue) shows a disproportionately large impact on entropy compared to CE loss, demonstrating these neurons primarily regulate distribution confidence rather than token predictions. The vanilla fine-tuned model (yellow) shows reduced effects, while the LN-free model (green) shows no measurable change in either metric.
33

---

Published as a conference paper at ICLR 2026
1 IMPACT STATEMENT
Our work investigates the role of Layer Norm transformer-based language models, showing that can be entirely removed from all GPT-2 models with minimal performance loss. This contributes the broader interpretability agenda by removing nonlinearity and reducing complexity and entanglement. Our results do not move the frontier of model capabilities; thus, we do not expect our work to create novel risks. In contrast, our work may support safer and more transparent model development by making more tractable and accurate mechanistic interpretability techniques. As with other interpretability advances, there remains the possibility that our work could be used to develop more capable Al systems. However, we believe the release of I.-ffree GPT-2 models will primarily serve researchers working to understand model internals and improve the transparency of current architectures.
34

---

Published as a conference paper at ICLR 2026
J STATEMENT ON LLM ASSISTANCE.
We used LLM-based coding assistance to draft tests and make small code edits. While writing the manuscript, used LLM tools polish wording, improve clarity, and generate LaTeX math expressions. All suggestions were reviewed and integrated by the authors.
35


Reviews:
[{"review_id": "3C80jrPSAT", "reviewer": "Reviewer_6KTD", "summary": "The authors show that LayerNorm modules can be removed from GPT-2 models through fine-tuning, with only a small increase in validation loss across all model sizes. The resulting LN-free models are released on Hugging Face. They also conduct a series of experiments to examine how removing LayerNorm affects several well-known phenomena (such as direct logit attribution, attribution patching, first-token high L2 norm, and confidence neurons). Their analysis reveals that some of these phenomena originate from the presence of LayerNorm, while others remain unaffected by its removal.", "questions": "I think the paper is very interesting and well written.\n* Did you attempt to train an LN-free model larger than GPT-2 XL? It would be helpful to understand whether the proposed method scales beyond 1.5B.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* The paper is very well written and easy to follow.\n* The experiments and baselines are thoughtfully designed and make it easy to understand the impact of each decision involved in fine-tuning the LayerNorm-free models. The approach is evaluated across multiple GPT-2 model sizes, strengthening the conclusions.\n* The paper analyzes several interpretability techniques and phenomena to examine the effects of removing LayerNorm. This enables a controlled investigation into whether well-known behaviors in language models are driven by LayerNorm itself or emerge from other underlying mechanisms. The analyses are well executed, and the conclusions are very interesting.\n* all their LN-free models are open-sourced and publicly available on Hugging Face", "weaknesses": "* A key limitation is that the analysis remains largely focused on the GPT-2 family. Although the authors include results for Pythia-70M, this provides only limited evidence of generality, and it is unclear whether the findings extend to newer or larger architectures.\n* The largest model evaluated is GPT-2 XL (1.5 B parameters), which is relatively small.\n* The title (“Transformers Don’t Need LayerNorm at Inference Time”) feels overclaiming, as the experiments are limited to GPT-2 models and up to 1.5B parameters.\n* Some mathematical notation is introduced without adequate definitions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KLtx6zyU5s", "reviewer": "Reviewer_KPh1", "summary": "The authors show that all layernorm layers in a transformer can be removed via fine-tuning from every GPT-2 model with only a small increase in validation loss (e.g. +0.03 cross-entropy loss for GPT-2 XL). They then study these models in interpretability tasks, showing downstream effects such as improvement in logit-attribution interpretations, improvements to approximation methods like attribution patching, and inactivity of confidence neurons. The work is novel because authors replace LN with a purely linear alternative as compared to earlier works which use non-linear alternatives.", "questions": "1. Did the authors consider training models without LN? How would this compare to their fine-tuned variant as well as the original GPT-2 model? \n2. What is the main takeaway for interpretability researchers?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I enjoyed reading this paper – I learned something from it. I think others will too! \n- The authors carefully dissect LayerNorm layers, find the sequence of removing these layers with several hyper-parameters, and provide a linear transformation equivalent whose loss is only minimally/negligibly worse as compared to the original. They then rigorously test out the effect of removing the layer-norm on important ideas in the literature – such as the effect on direct-logit-attribution as well as attribution patching, attention sinks, confidence neurons. The authors introduce a simple auxiliary loss term to stabilize loss during fine-tuning and do extensive empirical tests to find the best hyper-parameters for the sequence in which Layer-Norms should be removed.\n- Authors find that this removal can be scaled sub-linearly to larger models.\n- The work suggests exciting new directions of research, on changes and improvements to the transformer architecture with an interpretability lens.", "weaknesses": "1. The empirical cost of finding the hyper-parameters for LN removal seems to be really high, and the benefits of this removal don’t seem to be significant. Could training a transformer with the fake-LN be a better approach, rather than figuring out the best way to fine-tune away from it?\n2. The results are predominantly on GPT-2 – unclear how this transfers to very large LMs that are in use today.\n3. Unclear what the main takeaway is, in terms of whether the LN should be removed or not. The predominant advantage seems to be that logit attribution can be direct, but is the whole process of empirically understanding the best sequence/hyper-parameters to remove the LN worth the trade off?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LxzqhVwkKm", "reviewer": "Reviewer_L48A", "summary": "This article demonstrates that LayerNorm (LN) is not essential for Transformer inference, introducing a novel fine-tuning protocol that successfully removes all LN layers from the entire GPT-2 family (up to 1.5 billion parameters) with only a minor performance drop. The resulting LN-free models make a significant contribution to mechanistic interpretability, rendering Direct Logit Attribution (DLA) mathematically exact and reducing its approximation error from nearly 50% to 0%. The study also uses these models to causally confirm that \"attention sinks\" and \"confidence neurons\" are adaptive mechanisms dependent on LN's non-linearity. Finally, the work reveals LN's previously under-appreciated role in robustness and calibration, as the LN-free models become overconfident.", "questions": "1. Could you explain how you separated LN_{q,k}​ and LN_{v}​? Since the input to the entire attention block is typically normalized, wouldn’t that mean the keys, queries, and values all receive the same normalized input?  \n2. Have you tried training a reasonably large language model without LN? If so, how does its performance compare to LN-enabled versions?\n3. Beyond the mechanisms discussed in the paper, do you think the removal of LN could significantly affect any known circuits or mechanisms? For instance, could the IOI circuit become more distributed due to the absence of LN?\n4. Researchers often apply the final LayerNorm when using LogitLens. Do you think that would no longer be necessary if LN were removed?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. Nonlinearities in transformer architectures are a major challenge for mechanistic interpretability. This work takes an important step toward addressing that issue by replacing LayerNorm with a linear alternative.\n2. It not only provides a method for removing LN without significantly impacting the model’s overall performance, but also releases a suite of LN-free models that could be valuable to the mechanistic interpretability community.\n3. It performs a thorough empirical evaluation to show that model capabilities remain largely intact after removing LN, based on both perplexity and benchmark results.\n4. It provides insightful results about the impact of LN on common mechanistic interpretability techniques like direct logit attribution and attribution patching.\n5. It also details the impact of the removal of LN on first token norm, attention sink, and confidence neurons.\n6. It is well well-written paper which is easy to follow and provides the required details.\n7. I also appreciate the broader direction of conducting mechanistic interpretability research to modify and potentially improve model architectures.", "weaknesses": "1. As noted in the paper, relying on a single family of models and not evaluating larger models raises concerns about the broader effectiveness of the proposed LN removal method, especially given the fine-tuning involved and the need for extensive hyperparameter tuning. For example, both GPT-2 and Pythia models use standard LayerNorm, whereas many newer models employ RMSNorm, which limits the immediate applicability of the findings to architectures such as Llama or Mistral.\n2. While the impact of removing LN is assessed from multiple perspectives. There is more to be done to fully understand what LN actually does and how it helps in language modeling. For instance, this work showed that removal of LN makes the model overconfident, which needs to be better understood and mitigated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C9iNyAkzF2", "reviewer": "Reviewer_vbJk", "summary": "This paper investigates how LN can be removed from LLMs, and develops a step-wise procedure to do so. By removing LNs one by one, while continuing to (pre-)train using CE loss and an additional loss, the authors manage to remove LNs with minimal harm to model performance. They do so for all GPT-2 models and Pythia 70m. The authors then study 2 mech interp techniques thought to be hindered by LN and find that one performs better without LN (DLA) and another does not (AtP). The authors also discover that the confidence neurons discovered by prior work no longer exist in LN-free models.", "questions": "- Do you have any results for / did you consider the variants of attribution patching (integrated gradients, AtP*)? They seem worth discussing\n- Why use IOI for attribution patching? It seems like you could really just use any data (and even better, a more general dataset), since attribution patching can approximate the effect of any activation patching experiment.\n- Where exactly do you show that \"the amount of fine-tuning data needed for LN removal grows sublinearly with model parameters\" (22-23)?\n- Comment: Change 318-19 to say something like \"Nanda (2023a) describes it for this reason as \"a particularly thorny nonlinearity\". The current formulation feels a little too much like leaning on Neel Nanda's particular authority as a well-known individual, which is not super appropriate for an academic paper (or otherwise).\n- messed-up parentheses in citations (079-80)", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper (assuming it is extension of work I have seen previously) tackles an original question: can we remove LN from LLMs? It answers the question in the positive, quite clearly showing that LN can be so removed. The experiments fairly convincingly show that models' overall performance is not greatly harmed, and that this makes certain mech interp techniques more accurate. This is somewhat significant, as LN is generally considered an inconvenience by mech interp researchers.", "weaknesses": "- This paper could go a little further in proving that LN-less models are (almost) just as good as ones with LN. For example, it could show that such models can also be fine-tuned or quantized.\n- This paper doesn't seem to address fine-tuned / instruction-tuned / RLHF'd / reasoning models at all, even though they are quite often studied by mech interp researchers. Similarly, the models that the paper does address are rather small (though I think that's reasonable, considering the cost of using larger ones). It also doesn't study any models that use RMSNorm.\n- In general, my main concern about this paper is its practicality. Even if the number of extra training steps grows sublinearly with model size, this procedure seems inconvenient and compute-intensive to perform. Moreover, as much interpretability work derives its value from studying real-world models that people actually use, the value of studying these LN-free models will likely be limited by the fact that they will see little practical use. This means that this paper will probably be an academic curiosity more than a game-changing development; however, I still think it's interesting enough to publish.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)