Hierarchical Analysis: Monotonicity of Layerwise performance in Large Language Models

Authors: Jianfa Lai, Weihao Lu, Dongming Huang, Hehu Xie, Jie Luo, Qian Lin
Primary area: interpretability and explainable AI
Keywords: Monotonicity, Layerwise Performance, Large Language Models (LLMs), Hierarchical Analysis

Abstract:
We introduce a quantitative framework to evaluate how Large Language Models (LLMs) learn tasks across all layers, revealing a `monotonicity phenomenon'. Specifically: 
i) performance at each layer consistently improves from one layer to the next on the pre-training set, and 
ii) this improvement is consistently observed across various downstream tasks. This monotonicity phenomenon indicates that LLMs effectively capture complex hierarchical features across diverse datasets. For example, our study on the abstraction of concepts using linear representations in word embeddings shows that the clarity of these abstractions progressively increases with each layer.
Finally, by leveraging this monotonicity, we can significantly reduce inference time and memory requirements by selecting the most appropriate layer, thereby enhancing the efficiency of LLMs in real-world applications.

Decision: reject
Reviewer scores: 5, 6, 5, 5

Reviews and discussion:
<thread>
creation date:
1730254332644

thread_type:
official_review

<thread_object>
creator: reviewer_xuzj

<content>
summary:
This paper explores the layerwise performance of Large Language Models (LLMs) and uncovers a "monotonicity" phenomenon: LLMs consistently improve their performance from one layer to the next, both on pre-training data and across various downstream tasks. The authors attribute this to the LLMs’ability to capture complex hierarchical features and propose leveraging monotonicity to optimize inference time and memory usage by selecting the most appropriate layer embedding for a given task.

strengths:
The paper introduces the concept of “monotonicity” in LLMs, and combine ideas from hierarchical analysis and linear representation to explain the emergence of monotonicity, offering a fresh perspective on LLMs’ capabilities.

weaknesses:
1.The conclusion of "monotonicity" presented in this paper aligns with common intuition—that predictions made using deeper layer outputs generally yield better results. Therefore, proving this seems to offer limited value.

2.The paper lacks sufficient discussion and citation of related works, including studies on layer pruning[1][2][3][4] and LayerSkip[5]. All of these works consider the redundancy in the model's layers and remove the unnecessary ones to reduce the number of parameters and speed up computation.

3.The quality of the writing in the paper could be improved. For instance, the font size in the figures is too small, and Figure 3 unnecessarily includes two identical images.
[1]Men, X., Xu, M., Zhang, Q., Wang, B., Lin, H., Lu, Y., ... & Chen, W. (2024). Shortgpt: Layers in large language models are more redundant than you expect. arXiv preprint arXiv:2403.03853.
[2]Chen, X., Hu, Y., & Zhang, J. (2024). Compressing large language models by streamlining the unimportant layer. arXiv preprint arXiv:2403.19135.
[3]Kim, B. K., Kim, G., Kim, T. H., Castells, T., Choi, S., Shin, J., & Song, H. K. (2024). Shortened llama: A simple depth pruning for large language models. arXiv preprint arXiv:2402.02834.
[4]Song, J., Oh, K., Kim, T., Kim, H., Kim, Y., & Kim, J. J. (2024). SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks. arXiv preprint arXiv:2402.09025.
Elhoushi, M., Shrivastava, A., Liskovich, D., Hosmer, B., Wasti, B., Lai, L., ... & Wu, C. J. (2024). Layer skip: Enabling early exit inference and self-speculative decoding. arXiv preprint arXiv:2404.16710.

questions:
1. Why is accuracy used to evaluate the performance of pre-training GPT-2’s  checkpoints? Perplexity is a more commonly used evaluation metric for this purpose.

2. This paper highlights that only the early layers are necessary to achieve decent performance on certain downstream tasks. This is similar to LayerSkip[1], which employs a specialized training method to achieve good performance using only the first half of the model's layers. However, this paper does not mention LayerSkip. Additionally, the downstream tasks selected are simple classification tasks that could be effectively handled by a 110M BERT model. This raises the question of how well the proposed method would perform on more complex classification and generation tasks, such as MMLU, GSM8K, or HumanEval.
[1]Elhoushi, M., Shrivastava, A., Liskovich, D., Hosmer, B., Wasti, B., Lai, L., ... & Wu, C. J. (2024). Layer skip: Enabling early exit inference and self-speculative decoding. arXiv preprint arXiv:2404.16710.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730656991821

thread_type:
official_review

<thread_object>
creator: reviewer_xvcr

<content>
summary:
This paper evaluates LLMs' layerwise performance by focusing on the phenomenon of monotonicity, that is the performance at each layer consistently improves from one layer to the next one during pretraining. With extensive experiments on different datasets, they found that the layerwise performance improvement happens also on the dataset level, showing that the ability of LLMs to learn complex features during training is not limited to the data they were initially trained on but is a more generalizable characteristic to new tasks. They use the concept of linear representation in word embeddings to focus on consistent patterns and found that the monotonic effects exist with each layer. They suggest to focus on certain layers which are most appropriate for a certain task to save time and memory requirements.

strengths:
Proper dataset selection and task design for the experiment.

weaknesses:
1. I am afraid the main finding of the paper could be a bit superficial, as it talks about the monotonic effect, that is the performance improvement across layers in LMs. This finding is less interesting, although extensive experiments were conducted. Readers could be more interested in the "why" of this monotonic effect. More analysis regarding the explainablility could be required. 

2. There is a lack of details of experimental results, with only introducing the experiments in section 5. More content of results analysis should be conducted, with comparisons of performance of different layers (layer combinations), models, tasks.

questions:
NA
</content>
</thread_object>

</thread>
<thread>
creation date:
1730816609229

thread_type:
official_review

<thread_object>
creator: reviewer_szb1

<content>
summary:
The paper addresses a phenomenon the authors call "monotonicity" in large language models (LLMs). This phenomenon shows that intermediate layers of pre-trained LLMs, when fine-tuned, can achieve comparable results on downstream tasks like classification or regression, similar to using the last layer's embedding. The authors analyzed the performance of various datasets applied to LLMs such as GPT-2, Llama 3.1, Phi 3.5 mini instruct, and GPT-Neo. The results showed that while later layers enhance downstream task performance, the performance curve stagnates several layers before the final layer. Thus it might be useful to use only those layers for inference as it can save computational as well as time cost.

strengths:
- The paper is well written with a clear structure and good to understand.
- It outlines pracical advantages which can be used as a framework that is easy to apply by saving inference costs. This can be used for further academic research when dealing with LLMs

weaknesses:
- While mentioned in the text no results for regression tasks are reported
- While comparing LLMs of different providers (GPT2 – OpenAI, Llama 3.1 – Meta, Phi 3.5 – Microsoft etc.) a more extensive research by comparing models of the same provider with different sizes e.g. Llama 3.1 8B vs. Llama 3.1 1B would provide even deeper insights
- More detailled information oft he dataset collection would be useful to understand the set-up (e.g. label distribution)
- The only reported performance metric was accuracy depending on the label distribution (see above) different metrics might be more valuable

questions:
- What was the label distribution of your downsampled datasets?
- Which datasets did you used for regression tasks and what are the results?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731994024091

thread_type:
official_review

<thread_object>
creator: reviewer_tnpt

<content>
summary:
In this paper, a hierarchical analysis of LLMs is presented to illustrate the phenomenon of monotonicity in layerwise performance. The study shows that this progressive improvement in test accuracy, observed across successive layer in pre-training data and generalizes to a variety of downstream tasks.

strengths:
1. Many state-of-the-art LLMs are compared in experiments and many datasets from diverse domains are used in evaluation. 
2. The experiments are reasonable and the results are convincing.

weaknesses:
1. It'd be great if more experiments on the application of monotonicity could be conducted, for example, using more metrics other than test accuracy, and including more datasets.

questions:
How does the size of training data affect the monotonicity of layerwise performance?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732270410259

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Further Experiments Addressing Reviewers' Concerns

comment:
Thank you for your thoughtful suggestions and insightful comments. Many reviewers expressed interest in exploring additional metrics and experiments, and we are pleased to provide further clarifications and results in this response. The following experiments and their results are detailed in the **Supplementary Material**:

**Perplexity in GPT-2**:  
We evaluated the performance of GPT-2 using Perplexity on the OpenWebText dataset. The results demonstrate that the monotonicity principle remains effective across various metrics, even after the training process.

**Log-Loss for Classification**:  
To assess classification performance, we calculated the log-loss on the Banking77 dataset across three models. The results indicate that the monotonicity property is consistently upheld across these models, even with this metric.

**MMLU Question Answering - Humanities**:  
We conducted experiments on the MMLU Humanities question-answering benchmark to evaluate two models. The findings confirm that, while no accuracy flattens is observed in the final layers due to the complexity of this dataset—limiting the proposed method's ability to accelerate the inference process—the monotonicity principle remains valid for this task.

**Regression Data**:  
For regression tasks, we evaluated the Yelp dataset for rating prediction (\(Y = \{1, 2, 3, 4, 5\}\)) using Mean Squared Error (MSE) as the evaluation metric. The results confirm that the monotonicity property is consistently maintained across all three models tested on this regression dataset.

We hope these additional experiments and results effectively address the reviewers' concerns and further demonstrate the robustness of our approach across various tasks and evaluation metrics. Thank you again for your valuable feedback and consideration.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732356614237

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Follow-Up on Reviewer Responses

comment:
Dear ACs and Reviewers,

I hope this message finds you well. I am writing to inquire about the next steps for our submission, for which we have carefully addressed all of the reviewers' concerns in our submitted rebuttals. 

We sincerely appreciate your time and support throughout this process. Awaiting your guidance, we stand ready to implement any recommendations you may have.

Best regards
</content>
</thread_object>

</thread>
<thread>
creation date:
1732596413888

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
Dear ACs and Reviewers,

I hope this message finds you well. We kindly ask the reviewers to provide their comments on the authors' reply to the initial feedback, focusing on whether the responses adequately address the concerns raised and improve the manuscript. 

If the reviewers believe that additional experiments would be beneficial to address any outstanding issues, we encourage them to request these experiments now, as November 26th is the final date for authors to upload a revised PDF with any new results. After this date, authors will only be able to post replies on the forum, and no further experiments can be added. 

Your timely feedback and suggestions are crucial for ensuring the manuscript reaches its highest potential within the given timeline. We sincerely appreciate your time and support throughout this process.

Best regards
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper argues that LLM layerwise performance is "monotonic", i.e., deeper layers yield more accurate performance on downstream tasks. This is intuitive, given that deeper networks typically offer stronger performance, it would seem likely to follow that deeper layers within networks would similarly constitute stronger representations. This is a point made by both xvcr and XUzJ. Still, it is valuable to establish this empirically, even if not surprising. That said, the work does not engage adequately with existing related research, as pointed out by XUzJ.