Title: Do Language Models Use Their Depth Efficiently?
Venue: NeurIPS 2025
Accepted: yes
Keywords: transformer

llm

large language model

residual stream

depth

casual interventions

compositionality
Abstract: Modern LLMs are increasingly deep, and depth correlates with performance, albeit with diminishing returns. However, do these models use their depth efficiently? Do they compose more features to create higher-order computations that are impossible in shallow models, or do they merely spread the same kinds of computation out over more layers? To address these questions, we analyze the residual stream of the Llama 3.1, Qwen 3, and OLMo 2 family of models. We find: First, comparing the output of the sublayers to the residual stream reveals that layers in the second half contribute much less than those in the first half, with a clear phase transition between the two halves. Second, skipping layers in the second half has a much smaller effect on future computations and output predictions. Third, for multihop tasks, we are unable to find evidence that models are using increased depth to compose subresults in examples involving many hops. Fourth, we seek to directly address whether deeper models are using their additional layers to perform new kinds of computation. To do this, we train linear maps from the residual stream of a shallow model to a deeper one. We find that layers with the same relative depth map best to each other, suggesting that the larger model simply spreads the same computations out over its many layers. All this evidence suggests that deeper models are not using their depth to learn new kinds of computation, but only using the greater depth to perform more fine-grained adjustments to the residual. This may help explain why increasing scale leads to diminishing returns for stacked Transformer architectures.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates whether Transformers use depth efficiently, i.e., whether larger depth is utilized for higher-order computation or simply spreading computation. The authors analyze the residual stream in the LLaMA 3.1 model family. Through a series of empirical studies, they conclude that increased depth does not result in new computational capabilities but instead improves residual adjustments.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper tackles a fundamental and timely research question: how effectively Transformers leverage their depth. This has direct implications for architecture design, interpretability, and scaling strategies.\n2. The experiments are extensive. Multiple analysis techniques are employed to examine the effect of depth, and the results are presented clearly.\n\n**Weaknesses**\n1. The layer contribution metrics used in the analysis are introduced based on intuition, but their theoretical justification is limited. It would strengthen the paper to explain why these metrics are valid or whether they align with any existing formal frameworks.\n2. The connection between depth utilization and task complexity is underexplored. The paper primarily evaluates tasks that could plausibly be solved by relatively shallow models. It remains unclear whether the conclusions would hold for tasks that inherently require deeper computation.\n\nQUESTIONS:\n1. Can the authors provide further justification or theoretical grounding for the layer contribution metrics used in the analysis?\n2. Do the findings still hold for problems that cannot be solved by shallow models and inherently require deeper computation? For example, is a 10-layer model only able to solve problems whose solutions can be represented by models with effective depth no greater than, say, 5 layers? Or would it learn to utilize depth more effectively in this case?\n3. Prior work [1] [2] suggests that CoT prompting improves reasoning by increasing the effective computational depth of LLMs. Do the conclusions of this paper contradict such findings? Or do they suggest that CoT compensates for poor depth utilization in standard forward passes?\n4. Is the inefficiency in depth usage due to architectural limitations of Transformers (i.e., expressive capabilities) or to properties of the training process (e.g., optimization bias or data-induced inductive bias)?\n\n[1] Feng et al. Towards revealing the mystery behind chain of thought: a theoretical perspective. NeurIPS 2023.\n\n\n[2] Merrill et al. \"The expressive power of transformers with chain of thought.\" ICLR 2024.\n\nLIMITATIONS:\nSee Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the depth efficiency of modern large language models through the lens of mechanistic interpretability. By inspecting the residual stream representations of LLMs through logit lens and by measuring their causal contributions to model outputs via causal intervention, they found that layers in the second half contribute much less than those in the first half. Moreover, for semantically more complex problems, LLMs fail to utilize more layers to solve them, suggesting a lack of compositionality in model reasoning. Another probing analysis suggests that deeper layers are not performing \"new\" operations compared to early layers. Together, these empirical results suggest that LLMs are not using deeper layers efficiently, which may also explains the diminishing return of scaling up transformer by adding more layers.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The relation between neural network model depth and performance is an important research problem, especially in the era of LLMs where scaling up model size often lead to improved general performance. \n\n* The paper applied various mechanistic analysis tools to study LLM residual streams, and the experiments yield converging evidence that LLMs are not leveraging its deeper (especially second-half) layers efficiently.\n\n* The paper presents an inspiring discussion section by drawing relations between their interpretability findings and LLM reasoning capabilities using Chain-of-Thought and Latent Thinking.  \n\nWeaknesses: \n* While the analysis results presented in the paper are interesting, their empirical implications still seem vague to me. In particular, I found the argument that \"the second half of the transformer models seem wasteful\" a bit unsupported. According to Fig.4, without the second half layers, the model would yield an output distribution whose top-5 most likely token is very different from those predicted by the full model -- this should be considered as an essential step in computing the output probability distribution, as opposed to an incremental \"refinement\" step. \n\n* I'm not fully understanding the logical relation between the claimed LLM depth inefficiency and the potential ineffectiveness of latent thinking methods. Even if non-recurrent transformers are inefficient in using deeper layers, a recurrent one pretrained from scratch may still evolve in a different way. The analyses in this paper cannot offer insights in answering this question.\n\nQUESTIONS:\nSee the weaknesses section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the relevance of different layers in a large language model when prompted to solve mathematical and more general reasoning tasks. Through a sequence of analyses involving the average norms and cosine similarities of terms which define the residual stream, causal interventions, linear mappings between residual streams of different models, the authors draw the main conclusion that the second half of the layers in a language model is used to make local changes to the residual stream required for accurately predicting the current token, with this half of layers having little influence on the prediction of future tokens. An analysis of the MoEUT architecture shows that universal transformers may not suffer from the same limitations.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper presents a sequence of interesting observations which all point in coherence to the main conclusions, that (a) non-uniform transformers tend not to utilize the second half of the layers when predicting future tokens and that (b) the second half of the layers tends to be utilized to make smaller refinements of the residual stream necessary for predicting the current token.\n- One must commend the sheer amount of experimental evidence the authors provide, all of which cohesively (although perhaps not unshakeably conclusively) points to the final drawn conclusions. This also causes me to believe that the paper is entirely technically solid. I find  zero reason to doubt any of the presented evidence.\n- Observations made in this paper can an impact on future research on architectures, training objectives, and optimizers. I personally believe that the observations made in this work are valuable.\n- The linear mapping of the residual streams from all of the layers of one model to all of the layers of another model is, i find, a very interesting experiment.\n\n**Weaknesses**\n\n- Some of the evidence presented is not entirely convincing. For example, while Figure 8 does show a diagonal pattern, I would be hesitant to draw too strong of a conclusion from it. I find that it is not entirely clear what it means to \"stretch out\" the same computation over several layers. Of course, I acknowledge the difficulty of providing exact explanations of the internal computations of large language models, and the sheer amount of partial evidence provided in this work does point to the conclusions drawn by the authors. I would just be cautious about stating that deeper models \"stretch out\" the same computation that shallower models perform over several layers, especially considering more formal results which provide results on the transformer depth necessary to perform certain computations (See e.g. the logarithmic dependence in Liu et al. 2023). Another claim that I find somewhat unconvincing is the statement that, paraphrasing lines 248-250, *The fact that MoEUT tends to make use of more depth in its computation than a non-universal transformer **confirms the advantage of shared-layer models.*** I do not believe that this fact really confirms anything.\n- I believe that the text could be reformatted so that fewer references to sections and figures in the appendix are necessary. For example, in section 3.5., the four examples are not at all explained in the main text, being completely relegated to the appendix. I understand that, again, given the sheer amount of material the paper presents, it is difficult to adhere to the 9-page limit, but I still believe that there is room for improvement in this regard.\n- I believe that a better overview and explanation of the MoEUT results would have been in order. It is rather unsurprising that a universal transformer utilizes its layers in a drastically different manner than a non-uniform transformer does. It would have been interesting if this higher layer utilization could be linked to higher extrapolation performance in a more systematic manner than just listing the one example of Mul/Div Multiple Longer.\n- The claimed \"refinement of the current token's probability distribution\" sounds like a very interesting phenomenon, and I believe that it could have been addressed in more detail. But again I must acknowledge that one paper cannot do it all, so this weakness is rather to be taken as my opinion on what the ideal paper would do. \n- As essentially every experimental analysis, this one could also benefit from a wider set of benchmarks. Additionally, I have to question its relevance, given that nowadays chain-of-thought based approaches seem to be the standard for the type of task the authors investigate.\n\nQUESTIONS:\n**Questions**\n\n- While it was my impression that the observation of \"the second half of the layers refining the current token's probability distribution\" is framed somewhat negatively, could this not also be a positive property of non-universal transformers?\n- One sentence in the abstract poses the question of whether transformers *create higher-order computations that are **impossible** in shallower models*. Do you have any formal results confirming that certain computations that you investigate are indeed **impossible** in shallower models?\n- I must ask the authors what novel observations their analysis brings? I am aware that, even if not every observation is entirely novel, the analysis is valuable due to its scope and thoroughness, hence I am not interested in rejecting on the trivial ground of novelty. I am simply wondering which observation is entirely novel, given certain overlap of this work with at least that of *Lad et al., 2024*.\n\n\nI do not doubt that we can lead a meaningful discussion after which I will consider increasing my score. I would in particular appreciate a discussion on the originality of your approach, seeing as it's one of the grading criteria.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper asks whether modern pre-layernorm Transformer language models (Llama-3.1 and Qwen-3) truly exploit additional depth for new computations or simply spread the same work across more layers. Using a pack of residual-stream analyses and causal interventions—including layer-wise contribution analysis, causal layer-skipping, LogitLens & prediction-overlap, complexity probes with a depth-score metric, cross-model linear mapping, and an alternative-architecture (MoEUT) check—the authors uncover a sharp phase split: the first half of the network drives most residual change, while the second half mainly fine-tunes the current-token logits. These findings imply that stacked-Transformer LLMs under-utilize their later layers, devoting them to incremental logit sharpening rather than hierarchical composition, which helps explain diminishing returns from depth scaling and motivates architectures or objectives that make deeper networks more effective.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- Insightful multi-angle analysis: The paper fuses a diverse set of diagnostics. Since every tool points to the same phase-split conclusion, the evidence base is persuasive.\n- Practical relevance for depth scaling: By demonstrating that later layers in very deep Transformers mostly perform incremental logit-sharpening, the study offers a explanation for the diminishing returns observed in depth-scaling curves. This insight can inform both model-design choices and future research on more depth-efficient objectives or architectures.\n\n### Weaknesses\n- Connection to the latest literature is thin: Several closely related lines of work are not discussed. Incorporating results from Mix-LN [1], Peri-LN [2], selective attention pruning [3], and Massive Activations [4] would situate the contribution more firmly within the current discourse and might even sharpen the interpretation of the phase-split finding.\n\n- Architectural bias toward pre-LN models: All experiments use pre-layer-norm Transformers; it therefore remains unclear whether the conclusions extend to post-LN or other variants such as the OLMo architecture [5] or [2]. A small-scale replication on at least one non-pre-LN baseline, or a principled argument for transferability, would substantially strengthen the claim of generality.\n\n- Interpretability of massive activations remains unexplored: Since residual-path signals can accumulate large magnitudes (as documented by Sun et al. [4]), tying the observed logit-sharpening behavior to activation growth patterns could provide a more mechanistic explanation of how and why later layers lose representational diversity.\n\n- Clarity of key visualizations: Several figures—most notably the depth-score heat-map—lack fully labeled axes, color bars, or legends, forcing readers to consult the appendix for basic interpretive cues. Self-contained captions would improve accessibility.\n\n- Limited evidence for alternative architectures: The MoEUT comparison relies on only one or two checkpoints. Extending the study to a broader range of shared-parameter or other architectures would give greater empirical weight to the “look beyond pure stacking” recommendation.\n\n- Root cause of logit-smoothing remains open: The paper compellingly documents what happens in the back half of the network but stops short of explaining why. Some competing hypotheses are neither disentangled nor tested experimentally such as  (1) the next-token objective provides diminishing gradient signal at depth, and (2) architectural bottlenecks curb the formation of new representations \n\n[1] Li, Pengxiang, Lu Yin, and Shiwei Liu. \"Mix-ln: Unleashing the power of deeper layers by combining pre-ln and post-ln.\" ICLR2025.  \n[2] Kim, Jeonghoon, et al. \"Peri-LN: Revisiting Normalization Layer in the Transformer Architecture.\" Forty-second International Conference on Machine Learning. 2025.\n[3] He, Shwai, et al. \"What matters in transformers? not all attention is needed.\" arXiv preprint arXiv:2406.15786 (2024).  \n[4] Sun, Mingjie, et al. \"Massive activations in large language models.\" COLM2024.  \n[5] OLMo, Team, et al. \"2 OLMo 2 Furious.\" arXiv preprint arXiv:2501.00656 (2024).\n\nQUESTIONS:\nCovered in the Strengths and Weaknesses section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates whether deeper layers in modern LLMs contribute to more complex computations or simply distribute the same processes across more layers. Overall, their results suggest that current deep LLMs do not use additional layers to perform fundamentally new computations. Instead, depth is primarily used for finer adjustments of representations, possibly explaining diminishing performance returns from simply increasing model depth. The authors analyze the residual stream and layer contributions, finding that layers in the second half of the network contribute much less than the first half. Additionally, skipping later layers has minimal effect on future predictions while they don’t find evidence that deeper models use additional depth to compose multi-hop reasoning or complex intermediate steps. Their findings also suggest that deeper models spread existing computations over more layers instead of introducing new types of computation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The paper provides a thorough analysis of depth-wise computations in large language models, approaching the problem from several perspectives\n- The insights provided in the paper have practical implications for future research and model design.\n\nWeaknesses:\n- The experimental evaluation relies on relatively small sample sizes and limited datasets (e.g. on some experiments, 10 random examples from GSM8K), which may constrain the generalizability of the findings.\n- Similarly, the analysis is limited to only the Llama and Qwen model families, leaving open the question of whether the results generalize to other architectures.\n\nQUESTIONS:\n- The behavior of the Qwen models appears to differ somewhat from that of the Llama models (e.g., Figure 15, Figure 18e). Could the authors offer any explanations for these differences? It might also be valuable to analyze additional model families to better understand whether the observed patterns are consistent across architectures.\n\n- Are there notable differences in layer-wise behavior across different prompts, datasets, or tasks? While the paper includes some experiments on datasets like MQuAKE and MATH, several analyses rely on only 10 sampled prompts from GSM8K. It could strengthen the findings to include a broader range of prompts and datasets to assess the robustness and potential variability of the results.\n\n- In the discussion in Section 5, the authors suggest that the second half of the layers might be wasteful, as these layers primarily perform distribution matching. However, could this distribution matching still be necessary, given that the prediction distribution halfway through the network remains substantially different from that of the final layer?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}