Title: Exploring Context Window of Large Language Models via Decomposed Positional Vectors
Venue: NeurIPS 2024
Accepted: yes
Keywords: Positional Vector

Context Window

Large Language Model

Length Extrapolation

Context Window Extension
Abstract: Transformer-based large language models (LLMs) typically have a limited context window, resulting in significant performance degradation when processing text beyond the length of the context window. Extensive studies have been proposed to extend the context window and achieve length extrapolation of LLMs, but there is still a lack of in-depth interpretation of these approaches. In this study, we explore the positional information within and beyond the context window for deciphering the underlying mechanism of LLMs. By using a mean-based decomposition method, we disentangle positional vectors from hidden states of LLMs and analyze their formation and effect on attention. Furthermore, when texts exceed the context window, we analyze the change of positional vectors in two settings, i.e., direct extrapolation and context window extension. Based on our findings, we design two training-free context window extension methods, positional vector replacement and attention window extension. Experimental results show that our methods can effectively extend the context window length.


Peer reviews:
comments: SUMMARY:
This paper disentangles positional vectors from the hidden states of a pretrained Transformer language model to facilitate the understanding of length extrapolation. After a series of analyses, this paper proposes two context extending techniques. Experiments show that the proposed methods lower the perplexity on the task of language modeling.

STRENGTHS:
It's always good to have a mechanistic interpretability view of the hidden states of language models. The findings presented in this paper might inspire follow-up work along this direction.

WEAKNESSES:
The experiments presented in the current draft are not convincing enough to me. See questions below.

QUESTIONS:
1. Instead of continue training from the tinyllama model, I think training models from scratch using the 50B tokens budget will make the results in this paper more convincing. This is because you can get rid of the ripple effect of the originally used rope positional embeddings. Maybe your models were trying to unlearn rope during the continue training stage?
2. Apart from testing perplexity scores on the task of language modeling, I highly recommend the authors adding the experiment of needle in a haystack, otherwise I do not know if the models are really using all the tokens.
3. How do you decide the values of alpha and lambda in section 4.1 and 4.2? In addition, the temperature scaling technique was also used in several other places [1, 2] with explanations of how they did temperature selection.

[1] YaRN: Efficient Context Window Extension of Large Language Models
[2] Attention Alignment and Flexible Positional Embeddings Improve Transformer Length Extrapolation

LIMITATIONS:
See above.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-06

comments: SUMMARY:
This paper proposes a mean-based decomposition technique to analyze the formation and effect of positional encodings in LLMs. It then uses these results to propose methods to extend the context window, resulting in models that generalize better to longer texts.

STRENGTHS:
1. This paper is very well-written, and the main findings are properly highlighted.

2. This paper not only explains how positional vectors are formed, but also introduces methods to interpolate them based on the findings.

3. Experiments are performed to show that the new methods result in better perplexity scores beyond the context window.

WEAKNESSES:
I believe this contribution is novel and insightful enough, and there is no apparent weakness.

QUESTIONS:
1. The legends and graphs in Figure 4 overlap.

LIMITATIONS:
The authors have adequately addressed the limitations of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper dives into the inner workings of how transformer-based language models handle positional information. By decomposing hidden states into semantic and positional vectors, the authors give a series of analysis about how the positional information are encoded and propagated through layers. I believe this work offers valuable insights for understanding the positional information within the transformer architecture.\n\nSTRENGTHS:\nVery detailed and clear analysis about how the positional information is encoded and propagated within the transformer architecture, and to the best of my knowledge, I have not seen similar work before. I particularly enjoyed reading Figure 2 and 3, which shows how positional information is propagated through layers and goes beyond the window size, and shows how the manipulation of the positional embedding causally influence the attention patterns, particularly removing the attention sink.\n\nWEAKNESSES:\nThere are few points that I would like to suggest here to make the paper even stronger.\n\n- Section 4 feels weak and unnecessary. The performance of replacing the positional vector, if my understanding is correct, seems to be much worse than Dynamic NTK. Given the current mainstream approach is modifying the base of Rope (like YaRN), which is much easier than the approach proposed by this work, I do not think this work’s proposed context extension will be accepted by mainstream model builder.\n- That being said, I think the in-depth analysis of the positional embeddings are strong enough for me to give an acceptance (I learned a lot from it), so **I would strongly suggest removing the content of section 4, and use its space for more experimental analysis of the positional vectors**\n\nThere are a few important problems that I believe will receive the communities’ attention and worth being addressed:\n\n- Although this paper shows the positional information can propagate through layers (Figure 2), in practice, many work found that models with window attention cannot pass the needle in a haystack test, and this is why Mistral 1.5 changed its attention back to full attention. It would be insightful if the authors can discuss the relationships between positional information and needle-in-a-haystack performance (because needle in haystack is what makes long-context models useful), i.e., why window attention cannot pass needle in haystack even it does have the correct positional information?\n- This paper’s analysis is restricted on TinyLLaMA, but TinyLLaMA is not a widely used open-source model, thus casting the doubt whether this discovery of this paper will hold for other model families, particularly mainstream open-weight models like LLaMA 3, Mistral, QWen or Yi. I would strongly suggest the authors verify the behavior of positional embedding on either LLaMA 3, Mistral, QWen, or Yi.\n\nCurrently I’m given a borderline accept, and I will consider increasing my scores if the authors could either (1) discuss the relationship between positional vectors v.s. needle-in-a-haystack or (2) verify that the properties of positional vectors hold for LLaMA 3, Mistral, QWen or Yi (any 2 out of the 4).\n\nQUESTIONS:\nsee the above weakness section\n\nLIMITATIONS:\nsee the above weakness section", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-17"}