What Rotary Position Embedding Can Tell Us: Identifying Query and Key Weights Corresponding to Basic Syntactic or High-level Semantic Information
Authors: Yiting Chen
Junchi Yan
Venue: NeurIPS 2024 poster (2024)
Keywords: Large Language Model.+Rotary Position Embedding.+Self Attention
Primary area: natural_language_processing

Abstract: Transformer-based large language models (LLMs) have successfully handled various tasks. As one fundamental module in Transformers, position encoding encodes the positional information of tokens in a sequence. Specifically, rotary position embedding (RoPE), one of the most widely used techniques, encodes the positional information by dividing the query or key value with $d$ elements into $d/2$ pairs and rotating the 2d vectors corresponding to each pair of elements. Therefore, the direction of each pair and the position-related rotation jointly determine the attention score. In this paper, we show that the direction of the 2d pair is largely affected by the angle between the corresponding weight vector pair. We theoretically show that non-orthogonal weight vector pairs lead to great attention on tokens at a certain relative position and are less sensitive to the input which may correspond to basic syntactic information. Meanwhile, the orthogonal weight vector pairs are more flexible regarding the relative position, which may correspond to high-level syntactic information. Empirical evidence supports the hypothesis that shallow layers of LLMs focus more on local syntax and deep layers focus more on high-level semantics. Furthermore, we show that LLMs fine-tuning mainly changes the pairs of weight vectors that are nearly orthogonal, i.e., the weight corresponding to high-level semantics, which enables the reduction of the number of trainable parameters during fine-tuning without sacrificing performance. We propose a method namely Angle-based Weight Selection (AWS) to reduce the fine-tuning overhead and verify the effectiveness of the proposed method on widely used Alpaca fine-tuned Llama-2.

What Rotary Position Embedding Can Tell Us: Identifying Query and Key Weights Corresponding to Basic Syntactic or High-level Semantic Information
Yiting Chen, Junchi Yan Dept. of CSE & School of AI & MoE Key Lab of AI, Shanghai Jiao Tong University
(sjtuucyt, yanjunchi @sjtu.edu.cn
Abstract
Transformer-based large language models (LLMs) have successfully handled vari- ous tasks. As one fundamental module in Transformers, position encoding encodes the positional information of tokens in sequence. Specifically, rotary position embedding (RoPE), one the most widely used techniques, encodes the positional information by dividing the query or key value with d elements into d/2 pairs and rotating the 2d vectors corresponding to each pair of elements. Therefore, the direction of each pair and the position-related rotation jointly determine the attention score. this paper, we show that the direction of the 2d pair is largely affected by the angle between the corresponding weight vector pair. We theoreti- cally show that non-orthogonal weight vector pairs lead to great attention on tokens at a certain relative position and are less sensitive to the input which may corre- spond to basic syntactic information. Meanwhile, the orthogonal weight vector pairs more flexible regarding the relative position, which may correspond high-level syntactic information. Empirical evidence supports the hypothesis that shallow layers of LLMs focus more on local syntax and deep layers focus more on high-level semantics. Furthermore, we show that L.LMs fine-tuning mainly changes the pairs of weight vectors that nearly orthogonal, i.e., the weight corresponding to high-level semantics, which enables the reduction of the number of trainable parameters during fine-tuning without sacrificing performance. We propose a method namely Angle-based Weight Masking (AWM) to reduce the fine-tuning overhead and verify the effectiveness of the proposed method on widely used Alpaca fine-tuned Llama-2.
Introduction
have achieved impressive success and attracted considerable attention towards analyzing the Transformer structure and enhancing fine-tuning efficiency [[4]]]]
Large language models 36 26 35
18 [5]. As one fundamental module in a Transformer, position encoding encodes the positional information of tokens in sequence. Various position encoding have been proposed including absolute position encoding [36]] 8 26], and relative position encoding 30 (7], etc. Among them, one of the most widely used position encodings is rotary position embedding (RoPE) [31] which is used in various popular LLMs such as Llama [35), Mistral [66. GLM 9) etc. Instead of applying function to the input, RoPE applies a transformation the query and the key of each attention layer, which provides us with a window to investigate how the LLMs utilize the position information. In
*Corresponding author. This work in part supported by NSFC (92370201, 62222607) and Shanghai Municipal Science and Technology Major Project under Grant 2021SHZZD7X0102
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

this paper, we analyze the query and the key weight matrix in the self-attention module of LLMs using RoPE and propose a simple method to reduce the computational cost of LLM fine-tuning.
As the name "rotary" indicates, RoPE divides the elements of queries and keys into pairs and rotates each pair 2D vector with certain angle determined by the position. One simple fact is that the direction of the 2D vector before rotation and the position-relate rotation jointly affect the attention score. Note that the direction of the 2D vector is determined by the corresponding weight vector pairs, As illustrated in Fig. [ a simple analysis shows that non-orthogonal weight vector pairs with large absolute cosine similarity values are less sensitive the input and will draw greater attention to certain positions. It provides an indicator of how the model utilizes the position information. We empirically show that attention heads with large absolute cosine similarity between weight vector pairs focus more on basic syntactic information while attention heads with near-zero cosine similarity between weight vector pairs focus more on high-level semantics. In linguistics, formally, syntactic information refers to the arrangement of symbols or words according to the rules of a formal system or language 10] and semantic information pertains to the meaning and interpretation of words, phrases, and sentences (28) In studying how deep learning models utilise syntactic and semantic information, previous works (34] empirically show that shallow layers of LLMs focus more on basic syntactic information and deep layers of LLMs focus more on high-level semantics. Our experimental results align with previous works and further support the hypothesis. In a more fine-grained perspective, we further show that within each attention head, the angles between weight vector pairs also vary.
By comparing the fine-tuned version with the pre-trained version of LLMs, we further show that the weight in the query and the key are mainly changed on nearly orthogonal weight vector pairs during fine-tuning, and the non-orthogonal weight vector pairs are barely changed. This implies that fine-tuning the query and the key in LLMs mainly changes the weights corresponding to high-level semantic information and does not change the weights corresponding to basic syntactic information in the query and key. We conjecture that it is because the pre-trained LLMs are already good enough in processing basic syntactic information and only need to be tuned on how to process high-level semantic information for downstream tasks. Therefore, we propose to fix the non-orthogonal pairs of weight vectors in the query and key of each layer in the pre- trained models to reduce the number of trainable parameters during fine-tuning. We conduct experiments on widely used models and datasets to verify the effectiveness of our method. We show that our method could effectively reduce the number of trainable parameters while maintaining or even boosting the performance of the fine-tuned model. We summarize the contributions of this paper in the following:
We provide new perspective to investigate how LLMs with RoPE utilize the positional information and theoretically show that non-orthogonal weight vector pairs divided by RoPE are less sensitive to input and draw greater attention to certain relative positions.
We empirically show the angles between the weight vector pairs in the query and the key could serve as an indicator of whether the pair of weight vectors focuses on basic syntactic information or high- level semantics. Non-orthogonal weight vector pairs focus more on basic syntactic information while nearly orthogonal weight vector pairs focus more on high-level semantic information.
Since RoPE divides the elements in the query or key values into pairs, we empirically show that fine-tuning LLM mainly changes the orthogonal pairs of corresponding weight vectors. Based on our findings, only orthogonal pairs of weight vectors are changed during fine-tuning, we propose a method to reduce the number of trainable parameters during LLM fine-tuning and verify its effecctivnnes on widely used models and benchmarks.
2 Preliminaries and Related Works
Position Encoding. After the seminar work (36), various Transformer- position encoding methods have been proposed to incorporate position information into the function calculating query, key, and value. One typical way is adding the input with vector depending on the position of the input vector. Learnable position embedding is introduced where the position embedding is trainable during training (36] g (6). The sinusoidal positional encoding was introduced by the authors of (36) where the vector is generated by sinusoidal function. Instead of using absolute position, another branch of work proposes relative position encoding (30 27 Rotary position embedding (RoPE) (31
was proposed to incorporate position information by rotating the pairs of elements in the query and the key, which was widely used different L.LMs such as Llama (35), Palm 22, Mistral [6).
GLM 1 etc. Recent works on long-contes Transformers also show the advantage of RoPE on input

---

RoPE
ROPE
000
cosa
RoPE
Multiply
cosa <
Projected
OOOD
Input: Space (R&)
Query/Key Weight
2D Space (IR2)
Figure 1:
Illustration of how the angle between weight vector pairs in the query or the key affects RoPE. The larger absolute cosine similarity cos al, the direction of the projected 2D) vector is more fixed which leads to high attention certain relative positions regardless of the input.
In this paper, we provide a new weight vector angle perspective to investigate how LLMs with RoPE utilize the positional information and empirically show that we can identify weights corresponding to processing basic syntactic information or high-level semantic information. Please refer to Sec. for more details.
length extrapolation
Farameter-Ffffciintt Fine-Tuie Tuning. Different from full fine-tuning that fine-tunes all the parameters, parameter-efficien fine-tuning (PEFT) methods aim at reducing the number of parameters fine-tuned reduce the computational cost and prevent overfitting on fine-tuning data [25]. Various PEFT
methods have been proposed such as adapter tuning (14), prefix tuning [1]. and prompt tuning [18]
Adapter tuning proposes adding small module each layer of the pre-trained model. Prefix tuning and prompt tuning propose adding additional tunable prefix tokens to the input. Notably, LoRA [5
was proposed to use a low-rank matrix to approximate parameter updates.
Other Methods to Reduce Computational Cost of LLMs. Besides the PEFT methods, many other methods are proposed to accelerate the L.MSs. Quantization (5) 7 is common technique reduce the memory footprint of I.MMs. ( propose double quantization to further extend the application of
quantization from inference to fine-tuning. Another common technique is pruning where parameters
or modules are pruned to reduce the computational cost [7 [3]. Recently, [12] shows that deep
layers of LLMs can be pruned without degrading much performance. Orthogonal these previous methods in reducing the computational cost of LLMs, we provide a method to identify weights in the query and the key that does not require updating during fine-tuning. Refer Sec. 4 for more details.
Definition of RoPE. For an input sequence S = [xm where x, R is the input regarding the m-th token. The self-attention with RoPE 31 generates the query and the key with
qm (Tm m) = R W Wm..
(1)
km (a m) R Wa.m:
where qm and km is the query and key of the m-th token, the Wg and W₁ is the weight matrix and Rd. ; is the rotary matrix defined as following
sin m0 cos ml0 sin má 0
0
0
0
0
cos m0, 0
0
0
0
cos m0,
sin m0,
0
0
Rdd
sin m62
cos m02
0
:
0
0
0
0
CoS m00
- -sn a/2
0
0
0
sin m0d/2
cos m6d/2
(2)
3

---

The è is predefined as @ = {0,- 10000 2(1 2/1-1//4 1//4 W// (1,2, ,d//]}.
3 Investigating the Angle between Weight Vector Pairs in RoPE
In this section, we first provide a simple analysis regarding how the angle between the pair of weight vectors will affect the RoPE and then provide empirical results regarding how the LLM utilizes the positional information with different attention heads and across layers.
3.1 A Simple Analysis Regarding the Angle between Weight Vector Pairs
Let w and T denote the i-th row vector of W, 4 and W4 such that W,, (w w wa) T and
w T. As shown in Eq. RoPE rotates each pair (w (24-1) (20) and (wr, (2 Xm: W Xm) with m&, As we take the inner product of query and key, the direction of 2D pairs and the position-related rotation jointly determine the attention weight. According the definition in Eq. we have
WR = (wl.
(2 ke w
Xm Wy Xm)
(2))T
d/2
qn Km
Xn, w. (
xn)ll Il k 13
Xm, W Xm Ill cos (1 Ymm + (n m m)8:)
(2))
(3)
Where m. corresponds to the direction of 2D vector (wa (2i-1)T
(22) Xn, wq (24) xn.) and mm. corresponds
the direction of 2D vector (w(? Xm: W (2))
(2T
Xm).
Let us consider the angle nhlq and k Since the analysis for the query and the key the same, we use {g. k} to represent either the query or the key. Let xa (.... denotes the projection Xm onto
the subspace held by w 2i-1 and w (q..t)" Suppose the angle between w and wiq.yy is at (g.8) {g.k]
[g.kk
and the angle between w 21-1 and x² (0.6) is S (m.(q.))>
have
(g.k)
Wg (2
[ (n.)) |lI w (axyll cos( (m.m (g.k) a aaa)))
tan o fg.kg
Xm
(4)
(2i-1)
x m. w 1 cos((3 (3m.)aa))) m.ta.)))
[g.b))
Derive Eq.
we get
||w23 (g.tg)
(cos + sin dlasy tan Pm pmntass)))
mmm (g.k) = arctan
(5)
[[ (g.13)
As shown in Eq. 5 the larger sin ar (g,6)] the larger impact B (,, would have on (g..) It indicates that the sin (g is small and the cos [q.k)] is large for both the query and the key, the attention weight would be less sensitive to the input and draw greater attention to certain relative positions since the directions of the projected 2-d vector (w (2i-1)T Xm. w [g. Xm) of every token [q.k)
(2:)
close. In extreme condition, when the two weight vectors are in the same direction opposite direction, sin at (g.k) O, we have 7 [g,[g.)) arctan [
which means the direction of the
(g.k)]
2-d vector (w (2i-1)T Xm w (2:)T [g.k]
(g.h) xm) is fixed for any Xm
Note that as vectors in high-ddmensional space, due to the curse of dimensionality, these weight vector pairs nearly orthogonal when randomly initialized. Therefore the non-orthogonal weight vector pairs are non-trivial such that the model learns non-orthogonal weight vector pairs to emphasize tokens certain relative positions. In the following sections, we empirically show that non-orthogonal weight vector pairs widely exist in LLMs and provide a new perspective for us to investigate LLMs.
3.2 Attention Visualization for Different Attention Heads
To verify the conjecture in Sec. 3.1 that large absolute cosine similarity | ccos oi) corresponds to basic syntactic information and small cos a corresponds to high-level semantics, we visualize the
arotan
Arctan
'The focused relative position is

---

sdsss .212
Attention head with the Attention head with the largest average cos smallest average cos (a) Llama2-7b-chat
Attention head with the Attention head with the largest average cos of smallest average (b) Mistral
Figure 2: Attention visualization of the attention heads with the largest or the smallest average absolute cosine similarity cos a across the weight vector pairs in RoPE, where the key is on the left, and the query is the right. The lower transparncyy means higher attention score. We demonstrate the results for the first layer of support that higher cos a leads attention on basic syntactic information and lower cos a:l leads to attention on high-level semantic information. For more results of different models at different layers please refer to Appendix B
and Mistral-7B Dastral-7--Inttuct--00 Instruct-v0.2. The results empirically
attention of attention heads with different average absolute cosine similarity cos af in this section. With simple questions such as "What is the capital of France?" as input, we record the attention score of different attention heads and visualize the score by setting the transparency of the lines connecting the key and the query accordingly. The lower transparency corresponds For clarity in the figure, we limit the number of tokens to 20.
higher attention score.
As shown in Fig. 2 present the attention visualization of the attention heads with the largest the smallest average absolute cosine similarity in the 1st layer of Llama-2-7b-chatt (35) and Mistral- 7b-Instruct-v0..2 [16]. For the attention head with large absolute cosine similarity value across the weight vector pairs (0.54 on average for the 1st layer of Llama2-7b and 0.65 on average for the stt layer of Mistral-7b), attention mainly on tokens of prepositions or articles that may correspond more to the basic syntactic information. Notably, attention to special tokens is high. Special tokens mainly correspond to syntactic information, such as the end of the input prompt or the start of the answer by LLMs. In contrast, for the attention head with a small absolute cosine similarity value across the weight vector pairs in RoPE (0.24 on average for the 1st layer of Llama2-7b and 0.29 on average for the 1st layer of Mistral-7b), the attention is on every token of the phrase which may correspond more to the high-level semanties. For more results different layers or different models, please refer to Appendix. B
3.3 Analyzing Weight Vector Pair Angles Across the Layers
In this section, we propose investigate how LLMs utilize positional information from RoPE across the layers from the weight vector angle perspective. For each layer, we calculate the average absolute value of cosine similarity across all the weight vector pairs in the layer.
As shown in Fig. 3] we report the results of each layer of different LLMs. For each LLM, generally, the average absolute cosine value decreases to very low value after the first several layers (typically 3 layers) and stays low for the rest of the layers until it slightly increased at the last layer. This phenomenon agrees with the hypothesis that the LLMs first process the information about local syntax the first several layers and then process the high-level semantic information [4 B. While the angle between weight vectors pairs in RoPE also provides a new perspective complementary to the probing tasks 34 designed to show that models like BERT g process the information layer-by-layer following the traditional NLP pipeline. It also explains the success of fastly training a smaller model using the first several layers of a pre-trained LLM [29] where the first several layers contain the weights responsible for processing basic syntactic information, which plays vital role in the model.
5

---

Average absalute simllarity
2-13p
Average absolute cosine similant Falilantty
Jama 700-01
aama 7-70077
Janre 3 60
Came
04
Vole Lumrn
Nets Livrrs 3-720
chorg
02-
10
20
33
40
70
80
25
20
35
40
50
50
70
Layers
Layers
(a) Results for the query
(b) Results for the key
Figure 3: We report the average absolute cosine similarity of the query and the key across the layers of different LLMs. We show that, for all the LLMs we investigate, the average absolute cosine similarity drastically decreases after the first several layers and stays small until the last layer.
1st layer
5th layer
10th layer
20th layer
Figure 4: We show the cosine similarity of each weight vector pair in the query and the key of different layers of the Llama2-7b [35]. Each column corresponds to different layers. The top row the results for the query and the bottom row is the results for the key. The attention heads are separated with vertical red lines. We show that even within one head, the cosine similarity of different weight vector pairs differs. More results on other layers and other models are in Appendix
The last layer generates the output in certain format, which leads a slight increase in the average absolute cosine value. This also agrees with the empirical results in previous works (37 21 that the sparsity of activation changes the middle from sparse dense, and the layers the middle could be pruned. (3] also shows that the layers in the middle are the mostly changed layers during fine-tuning. Sec. we further extend the result that the orthogonal pairs of weight vectors (cos 0) are the mostly changed weights during fine-tuning and propose method to reduce the number of trainable parameters during fine-tuning.
3.4 Investigating the Cosine Similarity Across Weight Vector Pairs
Beyond the coarse grain results (attention head-wise as in Sec. 3.2 and layer-wise as in Sec. 3.3). the angles between weight vector pairs could provide more fine- aeneggranndd view. In this section, propose to investigate the distribution of cosine similarity between weight vector pairs in the query and the key. As shown Fig. 4 we report the cosine similarity, the cos a, of weight vector pairs of the query and key in the first layer of Llamma-7b [35]. In the figure, we separate different attention heads by vertical red lines. The cosine similarity changes drastically across the heads or even across the pairs of weight vectors in the same head. This means that in one head, there is still division of labor, such that some of the weight vector pairs draw more attention certain relative positions while the other weights are more flexible. More results of different LLMs are in Appendix B

---

1st layer
10th layer
20th layer
30th layer
Figure 6: Comparison between the base model Mistral-7B [16] and fine-tuned version WizardLM-
2 [38]. For each sub-figure, the scatter figure in the middle demonstrates the cosine similarity between weight vector pairs where each point corresponds
weight vector pair; the x-axis corresponds to the results for the base model, and the y-axis corresponds to the fine-tuned model. the histogram on the top and left, we report the average La weight distance between the weight vectors of the pre-trained model and the fine-tuned model. The y-axis of the histogram on the top and the x-axis of the histogram on the left correspond to the L2 weight distance. Generally, fine-tuning merely changes the angle between weight vector pairs. Besides the first several layers, the weight change mainly happens on weight vector pairs that are nearly orthogonal. More results are provided in Appendix
Still, we find that the angles between the pairs of weight vectors in query and key are highly re-
100-
lated to each other. As shown in Fig. the cos a of pairs of weights of the query is positively cor- related with the cos a of pairs of weights of the
key with Pearson correlation 0.86. It makes sense since it requires both the angle of the pro- jected 2D vector of the query and the key to be nearly fixed to assure high attention on certain relative positions. Initialized be both nearly orthogonal, we conjecture that the angles be- tween the query and the key weight vector pairs may be changed simultaneously during train- ing. For results of other layers and other models, refer Appendix B
000
675
00
1.00 0.75
-5.50
-0.25
0.50
0.25
0.50
0.75
Query
Figure 5: Correlation between cosine similarity of the query and key weight vector pair of the 1st layer of Llama2-7b. Each dot represents two corresponding weight vector pairs in query and key. The x-axis corresponds to the cosine similarity of the weight vector pair in the query and the y-axis corresponds to the cosine similarity of the weight vector pair in the key. (Pearson's r 0.86)
4 Reducing the Trainable Parameters During Fine-tuning
In this section, we first show that only the weight vector pairs that are nearly orthogonal (with 0) are changed during training by com- paring the base version and finetuned version (e.g. chat instruct version) of the same LLM. We then propose an efficient and effective method, namely Angle-base based Weight Masking (AWM), that fixes the non-orthogonal query and key weight vector pairs defined RoPE to reduce the number of trainable parameters while maintaining or even boosting the performance during fine-tuning, Since our proposed method fixes parameters at fine- grained weight row vector level, it orthogonal to LoRA (15), the widely used parameter-efficient fine-tuning method. The experimental results verified the effectiveness of AWM.
7

---

Llama-2-7b V.S. Llama-2-7b-chaat
Llamm-2-77b V.S.
Llama-2-13b V.S.
Llama-3-8b V.S.
Mistral-7b V.S.
Alpaca-7b-chat
Allma-2--13--che
Llama-3-8b-Instruc"
AizardL.-2
Figure 7: Average difference of weight vector pairs in the query and key across the layers of different LLMs (including Llama-2 [35,, Alpaca [33. Llama-3 [5 Mistral [16], and WizardLM (8)). We show that the average difference increases after the first several layers and stays high, which the
result of the fact that only near orthogonal weight vector pairs are updated during fine-tuning.
4.1 Comparing Different Versions of the Same LLM
Generally, most L.Ms are trained in a metrainnfinnture paradigm where the language model firstly pre-trained large dataset with pre-train task such predicting the next token and then finetuned to accommodate better the needs for downstream tasks. Generally, there will be many different versions of the same LLM released such the base version of the LLM after pre-training and the finetuned version e.g. the chat version. By comparing the parameters of two different versions of the same LLM, we could investigate how fine-tuning changes the parameters.
We present the results of comparison between the base model Mistral-7B-v0.1 [16] and its fine-tuned version WizardL.M-2 (8 in Fig. 6 Each sub-figure contains three figures. With the scatter figure the middle, we show the cosine similarity between weight vector pairs in the base model and the fine-tuned version, where the x-axis corresponds to the base model and the y-axis corresponds to the fine-tuned model. Generally, the fine-tuning barely changes the angle between weight vector pairs. means that the weights responsible for processing basic syntactic information high-level semantic information in the base model are still responsible for processing the corresponding basic syntactic information or high-level semantic information after fine-tune tuning. With the histogram figure the top and left of each sub-figure, we further present the average L2 distance between weight vector pairs in the base model and the fine-tuned model. For the figure the top, the x-axis corresponds to the cosine similarity of the weight vector pairs in the base model and the y-axis corresponds to the weight distance. Similarly, for the figure the left, the x-axis corresponds to weight distance, and the y-axis corresponds to cosine similarity results for the fine-tuned model.
Generally, find that the weight distances are high for weight vector pairs with nearly zero cosine similarity. indicates that fine-tuning mainly changes the weights corresponding to high-level semantic information processing and the weights corresponding to low-level information such as syntax are merely changed. Intuitively, it results from the fact that pre-trained LLMs are good enough at processing basic syntactic information and fine- fine-tuning changes in how they process high- level semantic information for downstream tasks. We further show the average distance between weight vector pairs across the layers of different base and fine-tuned models in Fig. Z As result of the fact that only near orthogonal weight vector pairs are updated during fine-tuning, the average distance increases after the first several layers and stays large, which is aligned with the results in Fig. 3 As previous work 23 also found that the layers in the middle are the most updated part during fine-tuning, we provide new perspective on further understanding the phenomenon. Beyond that, since we could effectively calculate the cosine similarity of weight vector pairs, it enables an efficient method reduce the number of trainable parameters during fine-tuning, Note that being barely changed during fine-tuning indicates non-orthogonal weight vector pairs are important in processing basic syntactic information, which also agrees with empirical results in [12].
We have also conducted experiments on more LLMs and received similar results including Ilama2 35 Alpaca 33 and Ilamaa3 (35 Please refer Appendix B for more results.
4.2 Reducing the Trainable Parameters on Query and Key
Our observation provides us with an efficient way to reduce the trainable parameters during training. According to the results in See. 4.1 the non-orthogonal weight vector pairs the query and key weight matrix do not need to be updated during fine-tuning, therefore we can reduce the number of
8

---

Table 1: Results of Llama-2 fine-tuned on the query and value in attention module with LoRA and AWM. We report the evaluation results on TruthfulQA 20), GSM8K 4. and HellaSwag 39 We also report the threshold of AWM and the portion of the masked query weight vector pair.
TruthfulQA
GSM8K
HellaSwag
Fixed weight
Model
Threshold
mel
mc2
acc
acc
vector pairs (%)
Llama-2-7b
26.07
39.45
4.85
57.99
L)
33.41
49.48
10.92
58.81
LoRA(r=8) AWM(Ours)
34.27
50.37
11.30
58.79
0.01
77.20
LoRA(r=8) + AWM(Ours)
34.27
49.98
11.90
58.79
0.005
85.36
&LoRA(==8)- + AWM(O
35.00
10.69
58.91
0.001
92.12
+ +oRRA(=22)
32.19
48.64
10.99
58. 73
+ LoRA((=2) +
AWM(Ours)
33.41
49.78
11.30
58.91
0.01
77.20
+ +LoRA(r=2) + AWM(Ours)
33.29
49.14
12.81
58.83
0.005
85.36
+LoRA(r=2) + AWM(Ours)
33.66
49.98
11.37
58.93
0.001
92.12
1lama-2-13h
23.75
33.41
19.33
60.89
LoRA(r=8)
30.72
44.74
18.42
61.31
LoRA(r=8) AWM(Ours)
30.48
45.40
19.41
61.45
0.01
79.32
LoRA(E=8) + AWM(Ours)
30.84
45.09
19.26
61.29
0.005
87.04
LoRA(E=8) + AWM(Ours)
32.07
45.88
19.48
61.28
0.001
93.40
trainable parameters before the fine-tuning is conducted and only update the nearly orthogonal weight vector pairs during fine-tuning. Therefore, we propose the method, namely Angle-based Weight Masking (AWM), to reduce the number of parameters during fine-tuning. With threshold set as т, we only update the weight vector pairs w w and w 2F (q.k) with cos ax < where a is the angle
[g.k)
between w 2i-1 and w25 "q. The detailed algorithm is in Alg.
(a.k)
Since our method orthogonal to many popular parameter efficient fine-tuning methods [15 6 that reduce the compu- tational cost of fine-tuning an LLM, in this section, we conduct our experiments with LoRA [15.. We finetune Llama2-7b and Llama2- 13b (35 on Alpaca [33] using LoRA and AWM. The query and the value weight of each model are fine-tuned with LoRA Alpaca for 3 epochs. For details on hyperparameter settings and re- sults, please refer to Appendix A
Algorithm Angle-based Weight Masking
Input: Query and Key parameters Og {w) (wig,zl E [0, d)}, other parameters Oo.
threshold and loss on fine-tuning dataset C(.)
OT + θo
for w 21-1
2
(g.k) E do
"g.k3'
Wark) (g.k)
In Table D we present the evaluation re- sults of LoRA combined with our AWM. We evaluate the fine-tuned model on TruthfulQA (20). GSM8K (4. and Hel- laswag 33 TruthfulQA measures the ten- dency of the model reproduce common falsehoods online. GSMSK a test di- verse grade school math problems. Hel- laswag is test of commonsense inference.
cos 1 [w....) x) II w (g.k)
if cos then
θr, + U [ 2i-1
w
{g.k}' [g.k)
end if end for
O + miner, L(OTr)
We Im-eval (II) and follow the setting in Open LLM Leaderboard to evaluate LLMs on these datasets where models are evaluated in 0-shot setting on TruthfulQA, 5-shot GSM8K, and 10-shot on Hellaswag. As shown in Table D our proposed method could largely reduce the trainable parameters in the query while maintaining or even boosting the performance. As shown in Table we also conduct experiments following the setting in LoftQ [19] where we fine-tune Llama-22 2-7b. Mistral-7B, and Phi-2 on wikitext-2 and GSM8K. Generally, our proposed method improves the performance while reducing the number of trainable parameters. We conjecture that the performance is boosted because the orthogonal weight pairs in the query corresponding to basic syntactic information are fixed, which prevents the model from overfitting the basic syntactic information in the fine-tuning dataset. Generally, the benefits of our
http::/hhngiiniiiccccococchhhhhe Jlm leaderboard Jendebbbrrrr

---

Table 2: Results of LLMs fine-tuned on WikiText-2 [22] and GSM8K 4 with LoftQ [9 and AWM. Models are fine-tuned through causal language modelling on training sets and are tested on validation/test sets. We also report the threshold of AWM and the portion of the masked query weight vector pair.
WikiText-2
GSM8K
Fixed weight
Model
Threshold
perplexity
acc
vector pairs (%)
Llama-2-7b LoftQ
5.518
39.20
+ LoftQ + AWM(O)rs)
5.485
40.86
0.01
77.20
+ LoftQ AOrr()ur))
5.483
38.44
0.005
85.36
+ LoftQ AWrr()urs)
5.480
38.06
0.001
92.12
Mistral-7B + LoftQ
6.423
54.51
+ LoftQ AWrr))
6.335
55.12
0.01
19.72
+ LoftQ AWrr()
6.340
55.88
0.005
21.56
+ LoftQ AWM(Ourr)
6.337
55.80
0.001
23.04
Phi-2 + LoftQ
9.553
18.75
+ LoftQ AWM(Ours)
9.766
51.71
0.01
32.44
+ LoftQ AWM(Ours)
9.829
52.01
0.005
37.00
+ LoftQ AWM(Ours) 9.836
52.92
0.001
40.72
proposed method are twofold: 1) further reduce the number of trainable parameters during fine-tuning, and 2) fix the orthogonal query and key weight vector pairs prevent overfitting.
5 Conclusion and Limitation Discussion
We provide a new perspective to study how LLMs with Rotary Position Embedding (RoPE) utilize the position information. We show that non-orthogonal query and key weight vector pairs in RoPE draw higher attention to certain relative positions regardless of the input, which may correspond to processing basic syntactic information while the orthogonal query and key weight vector pairs in RoPE are more flexible with the relative position, which may correspond to processing high-level semantic information. Our analysis and various empirical results at the layer level, attention head level, and neuron level are provided Sec. 3 By comparing the pre-trained model and the fine-tuned model, show that fine-tuning mainly updates propose a method to reduce the number of trainable parameters during I.LM fine-tuning in Sec. 4
orthogonal weight vector pairs and further
The limitation of the proposed method is that it only applies to the query or the key weights which may limit the number of reduced trainable parameters. Specifically, the query and key weight vector pairs are near orthogonal at random initialization, which makes the non-orthogonal query and key weight vector pairs non-trivial. The phenomena presented previous works and this paper imply that the not-orthogonal query and key weight vector pairs play important role in LLMs using RoPE, which requires further investigation and we leave for future works. This paper provides a new perspective investigating large language models (LLMs), which have the potential to make LLMs more accessible and effective for various applications. However, we must also address the potential risks associated with their misuse.
References
[1] Tom Brown, Benjamin Mann. Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
[2] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1
113, 2023.
10

---

[3] Kung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprini wrXiv:2309.03883 2023.
[4] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 14,1648,,, 2021.
[5] Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Llm.int8): 8-bit matrix multiplication for transformers at scale. CoRR, abs/2208.07339) 2022.
[6] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2024.
[7] Tim Dettmers and Luke Zettlemoyer. The case for 4-bit precision: k-bit inference scaling laws. In International Conference on Machine Learning, pages 7750-7774. PMLR, 2023.
[8] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre- pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Papers), pages 4171-4186. Association for Computational Linguistics, 2019.
[9] Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprini arXiv:2103. 10360, 2021.
[10] Victoria Fromkin, Robert Rodman, and Nina Hyams. An Introduction to language. An Introduction to Language, 2014.
[11] Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang. Laria Reynolds, Hailey Schoelkopf. Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou, A framework for few-shot language model evaluation, 12 2023.
[12] Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Daniel A. Roberts. The unreasonable ineffectiveness of the deeper layers. CoRR, abs/2403.17887, 2024.
[13] Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28, 2015.
[14] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nip. In International conference on machine learning, pages 2790-2799. PMLR, 2019.
[15] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OeenReview.net, 2022.
[16] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.0682. 2023.
[17] Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. Advances in neural information processing systems, 2, 1989.
[18] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Chengging Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACI/IJCNLP 2021, (Volume 1: Long Pupers), Virtual Event, August 1-6, 2021, pages 4582-4597 Association for Computational Linguistics, 2021.

---

[19] Yixiao Li, Yifan Yu, Chen Liang, Nikos Karampatziakis, Pengcheng He, Weizhu Chen, and Tuo Zhao. Loftq: Lora-fine-ture saane-tning--war quantization for large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
[20] Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods, arXiv preprini (.XXv::209.0799555 2021.
[21] Zichang Liu, Jue Wang, Tri Dao. Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivas- tava, Ce Zhang, Yuandong Tian, Christopher Re, et al. Deja vu: Contextual sparsity for efficient Ilms at inference time. In International Conference on Machine Learning, pages 22137-22177 PMLR, 2023.
[22] Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016.
[23] Abhishek Panigrahi, Nikunj Saunshi, Haoyu Zhao, and Sanjeev Arora. Task-specific skill localization in fine-tuned language models. In International Conference on Machine Learning, pages 27011-27033, PMLR, 2023.
[24] Bowen Peng. Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarr: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00077 2023.
[25] Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint trXiv:2005.00247 2020.
[26] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, al. Language models are unsupervised multitask learners. OpenAl blog, 1(8)99 2019.
[27] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1-4 1-140:67, 2020.
[28] John Saeed. Semantics: The meaning of words and sentences. In The Routledge Handbook of Linguistics, pages 153-168. Routledge, 2015.
[29] Mohammad Samragh, Mehrdad Farajtabar, Sachin Mehta, Raviteja Vemulapalli, Fartash Faghri, Devang Naik, Oncel Tuzel, and Mohammad Rastegari. Weight subcloning: direct initialization of transformers using larger pretrained ones. arXiv preprint arXiv:23/2.099299 2023.
[30] Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position repre- sentations. In Marilyn A. Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human I Language Technologies, NAACI-HILT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers), pages 4644668. Association for Computational Linguistics, 2018
[31] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocompuring. 127063, 2024.
[32] Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, and Furu Wei. A length-extrappllaabbe transformer. arXiv preprint arX1v:2212.10554, 2022.
[33] Rohan Taori, Ishaan Gulrajani. Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An instruction-folloiing llama model, 2023.
[34] Ian Tenney, Dipanjan Das, and Ellie Pavlick. Bert rediscovers the classical nip pipeline. arXiv preprint /00::0905000550 2019.
[35] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13377 2023.
12

---

[36] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[37] Elena Voita, Javier Ferrando, and Christoforos Nalmpantis. Neurons in large language models: Dead, n-gram, positional. arXiv preprint arXiv:3399.08827, 2023.
[38] Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. WizardIm: Empowering large language models to follow complex instructions. arXiv preprini arXiv:2304. 12244, 2023.
[39] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019.
[40] Shaofeng Zhang, Jinfa Huang, Qiang Zhou, Zhibin Wang, Fan Wang. Jiebo Luo, and Junchi Yan. Continuous-multiple image outpainting in one-step via positional query and a diffusion-based approach. 2024.
13

---

A Details of Experiments
A.1 Details of AWM
As shown in Alg. D with a threshold 7, we fix the weight vector pairs with absolute cosine similarity larger than the threshold We then fine-tune the model with LoRA Alpaca. We set the hyper- parameters following alpaca-lora? We fine-tune the query and the value with Lora alpha at 16 and rank at 8 or 2. The learning rate is 3e 4, batch size is 128, cutoff length is 512, and the dropout for LoRA is 0.05. Since the absolute cosine similarity is large in the first 3 layers, we did not apply our method to the first layers of Llama-2-76 and Llama-2-13b.
For evaluation, we follow open Ilm leaderboard to evaluate the finetuned model on Hellaswag, TruthfulQA and GSM8K. All the hyper-parameter settings are the same as those on the open LLM leaderboard. All the experiments are conducted on one NVIDIA GeForee RTX 4090.
A.2 Details of Attention Visualization
We take two simple questions as input. We record the query and key of each layer and calculate the attention score of different attention heads. We present the attention head with the largest and the smallest average absolute cosine similarity between weight vector pairs of the first layer of 2-7-c-2-77b-caa and Mistral 7B Instruct v0.2.
A.3 Details of Model Comparison
We compare various models, including 1lama-2-7b-chat, Llama-2-7 b, Llama-2-7 b finetuned on Alpaca, Mistral 7 B v0.1, WizzardLM2, Llama-3-8 B, and Llama-3-8B- Instruct.. ---sttuttt The phenomenon between these models similar, with changes mainly on the near orthogonal weight vector pairs.
B Additional Experiemental Results
B.1 Model Comparison
In this section, provide more comparisons of the results of versions of the same fine-tuned model.
1st layer
10th layer
20th layer
30th layer
Figure 8: Comparsion Results between Llama-2-7b and Llama-2- 2-7h--h---hhhttt-- Similar to Fig.
1lm leaderboard
14

---

1st layer
10th layer
20th layer
30th layer
Figure 9: Comparsion Results between Llama-3-8B and Llama-3-88 Laama---B--Insiiie 8---tuutt Similar to Fig.
Attention head with the Attention head with the smallest average cos a
Attention head with the Attention head with the largest average cos al smallest average cos a (b) Llama3- --Bna-BBB--- IB-Instrcc
largest average cos ail Llama2- 13b--hab
Figure 10: Attention visualization of the attention heads with the largest or the smallest average abso- lute cosine similarity cos a across the weight vector pairs in RoPE, where the lower transparency means the higher attention score. We demonstrate the results for the first layer of Llama2 13b chat and Llama3 8B Instruct. Note that Llama3 8B have much lower cos al (0.46 max and 0.17 min) comparing to llama2 (0.70 max and 0.31 min).
B.2 Attention Visualization
We provide more attention visualization results on Llama-2-13b and Llama3 8B in Fig. 10 Note that the attention head at the first layer of Llama3 8B has relatively small average absolute cosine similarity compared Llama2 7b and Llama2 13b.
B.3 Pair-wise Cosine Similarity
In this section, we further provide the visualization of cosine similarity between the weight vector pairs in different Mistral 7B v0.1 layers in Fig. [T In the future, we hope to extend method and apply it to further models such as multi-modal LLMs or position information related to image processing 40
15

---

Jom
1st layer
5th layer
10th layer
20th layer
Figure 11:
We show the cosine similarity of each weight vector pair in the query and the key of different layers of the Mistral 7B v0.1. Each column corresponds to different layers. The top row the results for the query, and bottom row the results for the key. The attention heads are separated with vertical red lines. We show that even within one head, the cosine similarity of different weight vector pairs differs.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification; We summarize the contribution of this paper in the Introduction. Generally, we provide a new perspective to investigate how LLMs using RoPE utilize the positional information and further propose method reduce the number of training parameters during LLM fine-tuning.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer:
[Yes]
Justification:
We discuss the limitations of the work in the final conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
16

---

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: The theoretical results in this paper only involve deriving the angle of the projected 2D vector, which is self-cxxlanatory and thus requires no further elaboration.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have described the details of our method in each experimental section and in Appendix B
The source code will also be made publically available upon acceptance.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same
17

---

dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open- dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
The source code will be made public upon acceptance. The reasons are twofold. First, the experiments in this paper are easy to implement, which makes releasing the code less necessary. Second, given the time we spent preparing the paper, more time is needed to clean and refactor our code.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
Sublic/guides htt////Seuuasamiiioooooooooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https ctipssccc/pullcc/eeeccc htw.a///ccguuieee//ooeesbbiaaaas
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
18

---

Answer: [Yes]
We have provided details when introducing experiments and provide full details Appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: most of the results in this paper, we investigate open-source LLMs, which are fixed and do not require error bars. For fine-tuning LLMs, we use the official code on Huggingface to ensure fair comparison. the same time, running the fine-tuning multiple times is too computationally expensive
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided information of the compute resources in Appendix A
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
19

---

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Question:
Answer: [Yes]
Justification: The research is conducted with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the potential impact of this paper in the final conclusion.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper mainly provides a new perspective to investigate language models and does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
20

---

Released models that have high risk for misuse or dual-use use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: In this paper, we cite the paper or assets used in our experiments.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not introduce any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
21

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
22


Reviews:
[{"review_id": "nhRQ3SfNGv", "reviewer": "Reviewer_CgW8", "summary": "The authors propose a fascinating approach to optimizing transformer-based large language models (LLMs). They delve into the intricacies of position encoding, particularly focusing on the widely used rotary position embedding (RoPE) technique. By examining how the angle between weight vector pairs impacts attention scores, they reveal that non-orthogonal pairs are crucial for processing basic syntax, while orthogonal pairs handle higher-level semantics. Their experiments show that fine-tuning LLMs predominantly alters these orthogonal pairs, allowing them to propose a new method (QK-IPM) to reduce fine-tuning overhead. This method effectively trims down the number of trainable parameters without sacrificing performance, as evidenced by their tests on Alpaca fine-tuned Llama-2. Overall, their work offers a fresh perspective on position encoding and presents a practical solution for more efficient LLM fine-tuning.", "questions": "See weaknesses.", "limitations": "No", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed Query-Key Internal Pair Masking (QK-IPM) method stands out for its efficiency. By identifying that non-orthogonal weight vector pairs don't need updating during fine-tuning, the method significantly reduces the number of trainable parameters, streamlining the fine-tuning process and saving computational resources.\n\n2. The approach is backed by solid theoretical insights. The paper explains the relationship between the angles of weight vector pairs and their roles in processing syntactic versus semantic information, providing a robust foundation for the proposed method. This depth of understanding adds credibility and makes the findings more convincing.\n\n3. The empirical evidence provided, particularly through tests on widely used models like Alpaca fine-tuned Llama-2, demonstrates the practical benefits of the method. This real-world validation shows that the technique is not just theoretically sound but also effective in improving model performance with reduced overhead.", "weaknesses": "The paper primarily tests the proposed method on specific models and datasets. While the results are promising, a broader evaluation across various LLM architectures and more diverse datasets would strengthen the generalizability of the findings and ensure the method's robustness in different contexts.\n\nAlthough the method reduces the number of trainable parameters, the process of calculating the angles between weight vector pairs and determining which pairs to update introduces additional computational steps. This could offset some of the efficiency gains, especially in large-scale applications, and might need further optimization to ensure overall net benefits.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer CgW8, we sincerely appreciate the time and effort you devote to the reviewing process. We address each point of your concerns in the comments below.\n\n> W1: A broader evaluation across various LLM architectures and more diverse datasets would strengthen the generalizability of the findings\n\nThank you for your advice. We have conducted additional experiments with Llama-2-7B, Mistral-7B, and Phi-2. We will add the results and details to our paper and release the code.\n\n* **WikiText-2 and GSM8k**\n\nWe finetune the pre-trained model on WikiText-2 and GSM8K with LoRA following the setting in [1]. Models are fine-tuned through causal language modeling on training sets and are tested on validation/test sets. The results are in the following table. **Generally, our proposed method improves the performance while reducing the number of trainable parameters.**\n| Model| Threshold|  WikiText-2 Perplexity($\\downarrow$) | GSM8K Accuracy(%$\\uparrow$) |\n| -----| ----------| -------- | ---- |\n| Llama2-7b| baseline(1)|  5.518| 39.20|\n| Llama2-7b| 0.01|  5.485| __40.86__ |\n| Llama2-7b| 0.005| 5.483| 38.44|\n| Llama2-7b| 0.001| __5.480__ | 38.06|\n| Mistral-7B-v0.1| baseline(1) | 6.423| 54.51|\n| Mistral-7B-v0.1| 0.01| __6.335__  | 55.12 |\n| Mistral-7B-v0.1| 0.005 | 6.340 | __55.88__ |\n| Mistral-7B-v0.1| 0.001| 6.337| 55.80 |\n| Phi-2 | baseline(1) |  __9.553__ | 48.75 |\n| Phi-2 | 0.01  | 9.766  | 51.71|\n| Phi-2| 0.005| 9.829 | 52.01|\n| Phi-2 | 0.001 | 9.836 | __52.92__ |\n\nDue to the character limitation of the rebuttal, **we will update more experimental results in official comments**, which will also be added to our paper. We hope these additional experiments can address your concern and are more than willing to conduct more experiments.\n\nBesides the proposed method, we would also like to emphasize our efforts to better understand the LLMs with ROPE from a brand new perspective. With theoretical derivation and extensive empirical results, we provide insights including:\n * For LLMs using RoPE, non-orthogonal query or key weight pairs are less sensitive to the input, and the attention score is mainly determined by position information.\n * Weight pairs in deep layers are more near orthogonal compared to weight pairs in shallow layers.\n * Finetuning the LLMs with ROPE mainly changes the orthogonal weight pairs.\n\nWe hope the brand-new perspective and the insights provided in this paper could contribute to this community and provide innovation for future works. \n\n> W2: Calculating the angles between weight vector pairs and determining which pairs to update introduces additional computational steps.\n\nTechnically, the computational cost required for calculating the cosine similarity between each weight pair is much less than generating one token with the model. Without the need for GPUs, we could only use the CPU to accomplish the calculation. In the following table, we report the time used for calculating cosine similarity between each query and key weight pair with an AMD EPYC 7302 16-core Processor.\n\n| model       | Time used (s) |\n| ----------- | ------------- |\n| Llama-2-7b  | 10.3545       |\n| Llama-2-13b | 16.3472       |\n\nWhile our proposed method only requires calculating the cosine similarity once, the computational cost is negligible compared to the cost of finetuning. We will add a more comprehensive computation cost analysis in our paper. We sincerely appreciate your suggestion.\n\nWe hope our rebuttal addresses your concern, and we look forward to your reply. We are more than happy to respond to any further questions.\n\n[1] Li, Yixiao, et al. \"LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models.\" ICLR 2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hGAhhSzcPM", "reviewer": "Reviewer_LVwG", "summary": "The paper makes a significant contribution to the understanding of RoPE in LLMs and presents the QK-IPM method that reduces the number of trainable parameters during fine-tuning by targeting orthogonal weight vector pairs.The paper conducts experiments on TruthfulQA, GSM8K, and Hellaswag datasets to show the method's effectiveness.", "questions": "Please see weaknesses.", "limitations": "Yes.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The writing of this paper is clear.\n2. The paper conducts comprehensive analysis experiments to verify their conclusion.\n3. Based on their observation, the proposed method, which fixes the non-orthogonal pairs of weight vectors in the query and key of each layer, sounds reliable and simple to deploy.", "weaknesses": "1. The conclusion that shallow layers of LLMs focus more on basic syntactic information and deep layers of LLMs focus more on high-level semantics is somewhat boring. Many papers have claimed it.\n2. The font size of some figures (e.g., Fig 2,4) is too small. Therefore, I can not read and understand the figure. \n3. For Table 1, the phrase ‘Fixed weight’ and ‘vector pairs’ have been overlapped wrongly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer LVwG, \n\nthank you for your valuable feedback. We address each point of your concerns in the comments below.\n\n> W1: The conclusion that shallow layers of LLMs focus more on basic syntactic information and deep layers of LLMs focus more on high-level semantics is somewhat boring. Many papers have claimed it.\n\nIndeed, with empirical methods, previous works show similar conclusions, which we have also mentioned in our paper. However, drawing this conclusion is not the main goal of this paper. **By aligning the observation with previous works, we are trying to provide a more objective measure, the angle between weight pairs in LLMs using RoPE, to identify weights corresponding to processing basic syntactic information or high-level semantics.** The similar conclusions in previous works help strengthen the link between the weight pair angle and the basic syntactic information or the high-level semantics. To our knowledge, this paper provides a brand-new perspective in identifying weights corresponding to the basic syntactic information or the high-level semantics.\n\n> W2: The font size of some figures (e.g., Fig 2,4) is too small. Therefore, I can not read and understand the figure.\n\nWe are sorry for the inconvenience and we will improve it in the next version of our paper. For the time being, these figures in our paper are vector images. Therefore, these figures are clear as the readers zoom in, which could be a viable way to read these figures.\n\nIn Figure 2, we visualize the attention score of different attention heads in Llama2-7B and Mistral-7B in two sentences. In Figure 4, we show the cosine similarity of each weight pair in a bar chart.\n\n\n> W3: For Table 1, the phrase ‘Fixed weight’ and ‘vector pairs’ have been overlapped wrongly.\n\nWe will fix it. Thank you again for your careful review.\n\nWe hope our rebuttal has addressed your concern and we are looking forward to your reply. We are more than happy to respond to any further questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "V57KY2xtYb", "reviewer": "Reviewer_cttx", "summary": "The paper is a novel work that identifies how the angle between weight vector pairs in the query or the key affects RoPE. \nThe authors devise a simple yet effective method that is novel and orthogonal to existing fine-tuning efficiency techniques, such as LoRA. Experiments demonstrate that combining the proposed technique can further enhance the performance of LoRA in a lightweight manner.", "questions": "1. Are there any related works on the angle perspective in LLMs or general deep networks? If so, the related work section could be expanded to include these studies.\n2. Are there any works similar to this paper which are based on a key observation and propose a simple yet effective novel method? Providing more such examples could help reviewers better benchmark its qualification against the NeurIPS standard.", "limitations": "Please refer to the weaknesses part.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The authors excellently demonstrate how LLMs using RoPE utilize positional information and theoretically show that non-orthogonal weight vector pairs influenced by RoPE are less sensitive to input, thereby drawing greater attention to specific relative positions.\n2. The authors further reveal that non-orthogonal weight vector pairs focus more on basic syntactic information, while nearly orthogonal weight vector pairs emphasize high-level semantic information.\n3. They highlight the key observation that fine-tuning LLMs mainly alters the orthogonal pairs of corresponding weight vectors. This insight leads to a natural technique for reducing the number of trainable parameters during LLM fine-tuning, as only the orthogonal pairs of weight vectors are modified.\n4. The proposed parameter reduction approach is effective and orthogonal to existing methods such as LoRA. It can be integrated with these methods to achieve better performance, with consistent performance gains across experiments.\n5. The paper is generally well-written, with clear illustrations, and demonstrates the authors’ strong background in LLMs and machine learning.", "weaknesses": "1. The introduction should more clearly specify the types of LLMs on which the authors conducted empirical studies rather than generally referring to LLMs. \n\nAdditionally, there are some grammatical errors and typos:\n\n• “Various position encoding have been proposed” should be “Various position encoding techniques have been proposed.”\n• “After the seminar work” should be “After the seminal work.”\n• “Where” should be “where” (line 120).\n• “Derive Eq. 4” should be “Deriving Eq. 4.”\n\n2. The authors could better clarify the motivation behind their perspective, specifically how the angle between weight vector pairs in the query or the key affects RoPE.\n\n3. The experimental section could be expanded, as the current version includes only one table in the main paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer cttx, \n\nwe sincerely appreciate your detailed and valuable feedback. We address each of your comments in the following.\n> W1: The introduction should more clearly specify the types of LLMs on which the authors conducted empirical studies rather than generally referring to LLMs.\n\nThanks for the suggestion, we will update our paper and make it clearer in the introduction about the types of models we use for empirical studies. Generally, our method applies to LLMs using RoPE. We conduct experiments on widely used LLMs such as Llama2, Llama3, Mistral, etc.\n\n> W2: The authors could better clarify the motivation behind their perspective, specifically how the angle between weight vector pairs in the query or the key affects RoPE.\n\nWhile RoPE divides the elements in the query or the key into pairs and treats each pair as a 2D vector, the position information is encoded by rotating those 2D vectors. **The motivation of this paper is that the angle between the corresponding weight vector pairs largely affects the initial angle of those 2D vectors.** For example, if the pair of weight vectors are of the same direction, then the angle of the 2D vector is fixed regardless of the input. \n\nThank you for the suggestion, we will keep polishing our paper and further clarify the motivation in our paper.\n\n> W3: The experimental section could be expanded.\n\nWe have conducted additional experiments with Llama-2-7B, Mistral-7B, and Phi-2. We will add the results and details to our paper and release the code.\n* **WikiText-2 and GSM8k**\nWe finetune the pre-trained model on WikiText-2 and GSM8K with LoRA following the setting in [1]. Models are fine-tuned through causal language modeling on training sets and are tested on validation/test sets. The results are in the following table. **Generally, our proposed method improves the performance while reducing the number of trainable parameters.**\n\n| Model| Threshold|  WikiText-2 Perplexity($\\downarrow$) | GSM8K Accuracy(%$\\uparrow$) |\n| -----| ----------| -------- | ---- |\n| Llama2-7b| baseline(1)|  5.518| 39.20|\n| Llama2-7b| 0.01|  5.485| __40.86__ |\n| Llama2-7b| 0.005| 5.483| 38.44|\n| Llama2-7b| 0.001| __5.480__ | 38.06|\n| Mistral-7B-v0.1| baseline(1) | 6.423| 54.51|\n| Mistral-7B-v0.1| 0.01| __6.335__  | 55.12 |\n| Mistral-7B-v0.1| 0.005 | 6.340 | __55.88__ |\n| Mistral-7B-v0.1| 0.001| 6.337| 55.80 |\n| Phi-2 | baseline(1) |  __9.553__ | 48.75 |\n| Phi-2 | 0.01  | 9.766  | 51.71|\n| Phi-2| 0.005| 9.829 | 52.01|\n| Phi-2 | 0.001 | 9.836 | __52.92__ |\n\nDue to the character limitation of the rebuttal, **we will update more experimental results in official comments**, which will also be added to our paper. We hope these additional experiments can address your concern and are more than willing to conduct more experiments.\n\n> Q1: Are there any related works on the angle perspective in LLMs or general deep networks? If so, the related work section could be expanded to include these studies.\n\nTo the best of our knowledge, most related works on the angle perspective in LLMs focus on input length extrapolation [2,3] trying to extend the input length limit by changing the rotation angle. We will keep on looking for more related work and update our related work section promptly.\n\n> Q2: Are there any works similar to this paper that are based on a key observation and propose a simple yet effective novel method? Providing more such examples could help reviewers better benchmark its qualification against the NeurIPS standard.\n\nMany such works are based on key observations and propose a simple yet effective method. For example, the lottery ticket hypothesis [4], which won the ICLR 2019 best paper award, shows that dense, random initialized networks contain sparse subnetworks that could reach comparable performance when trained in isolation. They further propose a strategy to find such subnetworks. Notably, many excellent works contribute to the community mainly by providing key observations. For example, [5], the NeurIPS 2023 best paper, shows that emergent abilities only appear for specific metrics.\n\nIn this paper, we show that the angle between weight vector pairs in the query and the key affect how the LLMs with RoPE utilize position information and that finetuning the pre-trained model mainly changes orthogonal weight pairs. By providing the key observation and a new perspective to better understand the LLMs with RoPE, we hope this paper could innovate future works in this community.\n\nWe hope our rebuttal could address your concern and we are looking forward to your reply. Please feel free to pose any new comments and we will respond promptly.\n\n[1]  Li, Yixiao, et al. \"LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models.\" ICLR 2024\n\n[2]  Sun, Yutao, et al. \"A length-extrapolatable transformer.\" ACL 2023\n\n[3]  Peng, Bowen, et al. \"Yarn: Efficient context window extension of large language models.\" arXiv preprint arXiv:2309.00071 (2023).\n\n[4]  Frankle, Jonathan, and Michael Carbin. \"The lottery ticket hypothesis: Finding sparse, trainable neural networks.\" ICLR 2019\n\n[5]  Schaeffer, Rylan, Brando Miranda, and Sanmi Koyejo. \"Are emergent abilities of large language models a mirage?.\" NeurIPS 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UB8MTeVInh", "reviewer": "Reviewer_2wVC", "summary": "This paper investigates how the Transformer models with RoPE embeddings change their weights during pre-training and fine-tuning. The authors theoretically prove that RoPE divides key and query pairs into two groups: near-orthogonal and non-orthogonal weight vector pairs, which have different sensitivities to the input embeddings. The authors perform empirical studies that reveal that this property helps the model learn different levels of language abstraction: non-orthogonal weight vector pairs focus more on basic syntactic information, while nearly orthogonal weight vector pairs focus more on high-level semantic information. Finally, the authors propose utilizing this property during fine-tuning: while the model learns basic syntactic information during pre-training, only high-level semantic information requires updates in fine-tuning. Thus, the authors propose only changing the orthogonal pairs of weights in fine-tuning, significantly reducing the number of trainable parameters while keeping or improving performance and reducing overfitting.", "questions": "Questions:\n1. Your paper heavily relies on the distinction between \"Basic Syntactic Information\" and \"High-level Semantic Information\". Could you please define and provide examples of these two concepts when you first mention them in the introduction?\n\n2. Line 140: \"To verify the conjecture in Sec. 3.1 that a large absolute cosine similarity |cos α| corresponds to basic syntactic information and […],” - was this conjecture made in Chapter 3.1, or is it a new hypothesis? not sure it was mentioned in section 3.1\n\n3. Table 1: Is it possible to provide evaluation results for the model fine-tuned without PEFT?\n\nTypos/suggestions:\n1. Figure 1 never referenced in the paper.\n2. Line 105: “m-th input. The” -> “the m-th embedding/vector/etc, the” \n3. Line 151: “pairs(0.54 “ -> “pairs (0.54 o” (space)\n4. Line 208: “to Appendix B” -> “to Appendix B.”\n5. Line 273: is this sentence incomplete?", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 4, "strengths": "This paper discovers an interesting property of attention weight vector pairs, which gives insight into how information about different language abstractions is learned and stored in the model layers. The paper is well-written and provides significant support for its major claims, with extra details provided in the appendices. The proposed novel approach for efficient finetuning advances the state of the art in parameter-efficient finetuning and can be widely adopted.", "weaknesses": "The paper lacks diversity and depth in its experimental results, particularly in Chapters 3 and 4. The analysis presented in Section 3.2 is the only part of the paper that establishes a link between the level of abstraction learned by the model and the angles of the weight pairs, while the rest of the paper relies on this hypothesis. However, this section (and the Appendix) only provides a few examples that support the authors' claim, which could be cherry-picked. To better support this claim, the authors could:\n\nProvide a clear definition of \"Basic Syntactic or High-level Semantic Information.\" Throughout the paper, the authors use these terms extensively, but never explain what they specifically mean by them (e.g., are they referring to specific tokens, sequences of tokens, or something else?).\nOnce a definition is established, the authors could collect statistics across a variety of diverse tasks and demonstrate the correlation between angles and abstractions. The presence or absence of a statistically significant correlation could strongly support the hypothesis of a connection between weight pair angles and language abstraction.\n\nFurthermore, in Chapter 4, experiments are only conducted on the Llama2 model (7b and 13b) using three tasks. To underscore the universal nature of the discovered law, I would like to see experiments on other open families of large language models (LLMs) and possibly on more tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer 2wVC, \n\nThank you for your valuable feedback. Below, we address each of your concerns.\n> W1: The paper heavily relies on the link between the level of abstraction learned by the model and the angles of the weight pairs. \n\n> Q1: Could you please define and provide examples of these two concepts when you first mention them in the introduction?\n\nThank you for the suggestion. We will provide a clearer definition of \"syntactic information\" and \"semantic information\" in our paper.\nThe concept of syntactic and semantic information comes from linguistics, which are widely used in previous works. The formal definitions are:\n* **Syntactic information**: Syntactic information refers to the arrangement of symbols or words according to the rules of a formal system or language. It deals with the structure and organization of elements in a language, such as grammar, punctuation, and word order. It is concerned with the form rather than the meaning of the symbols. [1]\n* **Semantic information**: Semantic information pertains to the meaning and interpretation of words, phrases, and sentences. It involves understanding the meanings conveyed by linguistic expressions and how these meanings relate to one another within a language. It is concerned with the content and significance of the communication. [2]\n\nIn this paper, we theoretically prove that non-orthogonal weight pairs for the query and the key in LLMs with RoPE are less sensitive to the input and would pay greater attention to certain relative positions than orthogonal weight pairs. We try to link this phenomenon with the processing of the syntactic information and the semantic information, and our hypothesis is the following:\n* Non-orthogonal weight pairs focus on certain relative positions that deal with the syntactic information, such as the grammar or certain word order.\n* Orthogonal weight pairs are more flexible in terms of positions regarding the input; therefore, they process more semantic information since the content could appear anywhere in a paragraph.\n\nTo validate our hypothesis, we provide attention score visualization in Fig. 2, where the attention head with the most non-orthogonal weight pairs pays greater attention to the structure of the phrase, such as the special token for the end of the input prompt. In Sec. 3.3, we align our observation with previous works [3] that shallow layers process more syntactic information and deep layers process more semantic information, which may also strengthen our hypothesis.\n\nHowever, we must also emphasize that **our paper does not rely on this hypothesis**. Leave the concept of \"syntactic\" or \"semantic\" alone. We theoretically show how the angle between weight vector pairs affects RoPE in Sec. 3.1 and further investigate the weight vector angle (Sec. 3.3 and Sec. 3.4). In Sec. 4, we show that finetuning the pre-trained LLMs mainly changes the near orthogonal weight vector pairs and further propose a simple but effective method to reduce the trainable parameter while boosting the performance.\n\nDistinguishing the \"syntactic information\" or the \"semantic information\" has always been kind of subjective. For example, previous works [4] manually associate \"syntactic\" and \"semantic\" information with various tasks. We hope the hypothesis in our paper may contribute to the study of how deep learning models process syntactic information and semantic information and further innovate future works.\n\nThanks again for the suggestions. We will keep polishing our paper.\n\n> Q2: Line 140:  was this conjecture made in Chapter 3.1, or is it a new hypothesis?\n\nSorry for the confusion. We only analyze how the angle between weight vectors affects RoPE in Sec. 3.1 and did not introduce the conjecture in detail. The detailed hypothesis is written in our rebuttal above. We will also make it more clear in our paper. \n\n\n> W2:  I would like to see experiments on other open families of large language models (LLMs) and possibly on more tasks.\n\nWe have conducted additional experiments with Llama-2-7B, Mistral-7B, and Phi-2. We will add the results and details to our paper and release the code. We finetune the pre-trained model on WikiText-2 and GSM8K with LoRA following the setting in [3]. Models are fine-tuned through causal language modelling on training sets and are tested on validation/test sets. The results are in the following table. **Generally, our proposed method improves the performance while reducing the number of trainable parameters.**\n| Model| Threshold|  WikiText-2 Perplexity($\\downarrow$) | GSM8K Accuracy(%$\\uparrow$) |\n|----| ----|----|----|\n| Llama2-7b| baseline(1)|  5.518| 39.20|\n| Llama2-7b| 0.01|  5.485| __40.86__|\n| Llama2-7b| 0.005| 5.483| 38.44|\n| Llama2-7b| 0.001| __5.480__ | 38.06|\n| Mistral-7B-v0.1| baseline(1) | 6.423| 54.51|\n| Mistral-7B-v0.1| 0.01| __6.335__  | 55.12 |\n| Mistral-7B-v0.1| 0.005 | 6.340 | __55.88__ |\n| Mistral-7B-v0.1| 0.001| 6.337| 55.80 |\n| Phi-2 |baseline(1)|  __9.553__ | 48.75 |\n| Phi-2 | 0.01  | 9.766  | 51.71|\n| Phi-2| 0.005| 9.829 | 52.01|\n| Phi-2 | 0.001 | 9.836 | __52.92__|\n\nDue to the character limitation of the rebuttal, **we will update more experimental results in official comments**, which will also be added to our paper. We hope these additional experiments can address your concern and are more than willing to conduct more experiments.\n\n> Q3: Results without PEFT?\n\nWe will update the result promptly. However, with limited computational resources, the results may not be available during rebuttal.\n\n> Typos/ Suggestions\n\nThanks for the detailed feedback. We will check and polish our paper.\n\nWe hope our rebuttal addresses your concerns, and we look forward to your reply. \n\n[1] Fromkin, Victoria, Robert Rodman, and Nina Hyams. \"An lntroduction to Language.\" (2014).\n\n[2] Saeed, John. \"Semantics: The meaning of words and sentences.\" Routledge, 2015. 153-168.\n\n[3] Li, Yixiao, et al. \"LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models.\" ICLR 2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear AC and Reviewers,

We sincerely thank you for the time and effort you dedicated to the reviewing process. We are delighted to hear that reviewers find the paper well-written (LVwG, cttx, 2wVC), providing solid theoretical insights with extensive empirical evidence (CgW8, LVwG, cttx, 2wVC) and that the method we proposed to be simple and effective (CgW8, LVwG, cttx, 2wVC).

The main concern of the reviewers is also similar in that the experiment for the proposed method only takes one table in Sec. 4. To address the reviewers' concerns, we have conducted additional experiments as requested by the reviewers. **We will also try our best to provide more experimental results during the rebuttal period. In the joint response, we list the additional experimental results we have collected so far.**

* We have conducted additional experiments with **Llama-2-7B, Mistral-7B, and Phi-2**. We finetune the pre-trained model on WikiText-2 and GSM8K with LoRA following the setting in [1]. Models are fine-tuned through causal language modelling on training sets and are tested on validation/test sets. The results are in the following table. **Generally, our proposed method improves the performance while reducing the number of trainable parameters.**
| Model| Threshold|  WikiText-2 Perplexity($\downarrow$) | GSM8K Accuracy(%$\uparrow$) |
|----| ----|----|----|
| Llama2-7b| baseline(1)|  5.518| 39.20|
| Llama2-7b| 0.01|  5.485| __40.86__|
| Llama2-7b| 0.005| 5.483| 38.44|
| Llama2-7b| 0.001| __5.480__ | 38.06|
| Mistral-7B-v0.1| baseline(1) | 6.423| 54.51|
| Mistral-7B-v0.1| 0.01| __6.335__  | 55.12 |
| Mistral-7B-v0.1| 0.005 | 6.340 | __55.88__ |
| Mistral-7B-v0.1| 0.001| 6.337| 55.80 |
| Phi-2 |baseline(1)|  __9.553__ | 48.75 |
| Phi-2 | 0.01  | 9.766  | 51.71|
| Phi-2| 0.005| 9.829 | 52.01|
| Phi-2 | 0.001 | 9.836 | __52.92__|

* We test the time required to measure the angle between weight vector pairs. Technically, the computational cost required for calculating the cosine similarity between each weight pair is much less than generating one token with the model. We could only use the CPU to accomplish the calculation without the need for GPUs. In the following table, we report the time used for calculating cosine similarity between each query and key weight pair with an AMD EPYC 7302 16-core Processor.
| model       | Time used (s) |
| ----------- | ------------- |
| Llama-2-7b  | 10.3545       |
| Llama-2-13b | 16.3472       |

For each reviewer, we have posed a rebuttal addressing the concerns. We look forward to your reply and are more than happy to respond to any further comments. Once again, thank you for your valuable comments and support.

[1] Li, Yixiao, et al. "LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models." ICLR 2024

Meta-review: The paper presents an analysis of how RoPE embeddings affect pre-training and finetuning. The main finding is that key-query pairs that are more orthogonal correspond to more semantic information. The paper proposes a PEFT scheme that only updates these parameters during finetuning. The reviews are generally favorable, but there are some concerns that the paper uses unclear terms (basic syntactic or high-level semantic information) and the experimental results are a bit shallow (2wVC). The paper does seem to provide enough substance on a timely topic, but I would like to encourage the authors to be very clear in communicating the findings.
Decision: Accept (poster)