Title: What Rotary Position Embedding Can Tell Us: Identifying Query and Key Weights Corresponding to Basic Syntactic or High-level Semantic Information
Venue: NeurIPS 2024
Accepted: yes
Keywords: Large Language Model.+Rotary Position Embedding.+Self Attention
Abstract: Transformer-based large language models (LLMs) have successfully handled various tasks. As one fundamental module in Transformers, position encoding encodes the positional information of tokens in a sequence. Specifically, rotary position embedding (RoPE), one of the most widely used techniques, encodes the positional information by dividing the query or key value with $d$ elements into $d/2$ pairs and rotating the 2d vectors corresponding to each pair of elements. Therefore, the direction of each pair and the position-related rotation jointly determine the attention score. In this paper, we show that the direction of the 2d pair is largely affected by the angle between the corresponding weight vector pair. We theoretically show that non-orthogonal weight vector pairs lead to great attention on tokens at a certain relative position and are less sensitive to the input which may correspond to basic syntactic information. Meanwhile, the orthogonal weight vector pairs are more flexible regarding the relative position, which may correspond to high-level syntactic information. Empirical evidence supports the hypothesis that shallow layers of LLMs focus more on local syntax and deep layers focus more on high-level semantics. Furthermore, we show that LLMs fine-tuning mainly changes the pairs of weight vectors that are nearly orthogonal, i.e., the weight corresponding to high-level semantics, which enables the reduction of the number of trainable parameters during fine-tuning without sacrificing performance. We propose a method namely Angle-based Weight Selection (AWS) to reduce the fine-tuning overhead and verify the effectiveness of the proposed method on widely used Alpaca fine-tuned Llama-2.

What Rotary Position Embedding Can Tell Us: Identifying Query and Key Weights Corresponding to Basic Syntactic or High-level Semantic Information

Abstract

Transformer-based large language models (LLMs) have successfully handled various tasks. As one fundamental module in Transformers, position encoding encodes the positional information of tokens in a sequence. Specifically, rotary position embedding (RoPE), one of the most widely used techniques, encodes the positional information by dividing the query or key value with d elements into d/ 2 pairs and rotating the 2d vectors corresponding to each pair of elements. Therefore, the direction of each pair and the position-related rotation jointly determine the attention score. In this paper, we show that the direction of the 2d pair is largely affected by the angle between the corresponding weight vector pair. We theoretically show that non-orthogonal weight vector pairs lead to great attention on tokens at a certain relative position and are less sensitive to the input which may correspond to basic syntactic information. Meanwhile, the orthogonal weight vector pairs are more flexible regarding the relative position, which may correspond to high-level syntactic information. Empirical evidence supports the hypothesis that shallow layers of LLMs focus more on local syntax and deep layers focus more on high-level semantics. Furthermore, we show that LLMs fine-tuning mainly changes the pairs of weight vectors that are nearly orthogonal, i.e., the weight corresponding to high-level semantics, which enables the reduction of the number of trainable parameters during fine-tuning without sacrificing performance. We propose a method namely Angle-based Weight Masking (AWM) to reduce the fine-tuning overhead and verify the effectiveness of the proposed method on widely used Alpaca fine-tuned Llama-2.

1 Introduction

Large language models [36, 26, 35] have achieved impressive success and attracted considerable attention towards analyzing the Transformer structure and enhancing fine-tuning efficiency [14, 1, 18, 15]. As one fundamental module in a Transformer, position encoding encodes the positional information of tokens in a sequence. Various position encoding have been proposed including absolute position encoding [36, 8, 26], and relative position encoding [30, 27], etc. Among them, one of the most widely used position encodings is rotary position embedding (RoPE) [31] which is used in various popular LLMs such as Llama [35], Mistral [16], GLM [9], etc. Instead of applying a function to the input, RoPE applies a transformation to the query and the key of each attention layer, which provides us with a window to investigate how the LLMs utilize the position information. In

this paper, we analyze the query and the key weight matrix in the self-attention module of LLMs using RoPE and propose a simple method to reduce the computational cost of LLM fine-tuning.

As the name 'rotary' indicates, RoPE divides the elements of queries and keys into pairs and rotates each pair as a 2D vector with a certain angle determined by the position. One simple fact is that the direction of the 2D vector before rotation and the position-related rotation jointly affect the attention score. Note that the direction of the 2D vector is determined by the corresponding weight vector pairs. As illustrated in Fig. 1, a simple analysis shows that non-orthogonal weight vector pairs with large absolute cosine similarity values are less sensitive to the input and will draw greater attention to certain positions. It provides an indicator of how the model utilizes the position information. We empirically show that attention heads with large absolute cosine similarity between weight vector pairs focus more on basic syntactic information while attention heads with near-zero cosine similarity between weight vector pairs focus more on high-level semantics. In linguistics, formally, syntactic information refers to the arrangement of symbols or words according to the rules of a formal system or language [10] and semantic information pertains to the meaning and interpretation of words, phrases, and sentences [28]. In studying how deep learning models utilise syntactic and semantic information, previous works [34] empirically show that shallow layers of LLMs focus more on basic syntactic information and deep layers of LLMs focus more on high-level semantics. Our experimental results align with previous works and further support the hypothesis. In a more fine-grained perspective, we further show that within each attention head, the angles between weight vector pairs also vary.

By comparing the fine-tuned version with the pre-trained version of LLMs, we further show that the weight in the query and the key are mainly changed on nearly orthogonal weight vector pairs during fine-tuning, and the non-orthogonal weight vector pairs are barely changed. This implies that fine-tuning the query and the key in LLMs mainly changes the weights corresponding to high-level semantic information and does not change the weights corresponding to basic syntactic information in the query and key. We conjecture that it is because the pre-trained LLMs are already good enough in processing basic syntactic information and only need to be tuned on how to process high-level semantic information for downstream tasks. Therefore, we propose to fix the non-orthogonal pairs of weight vectors in the query and key of each layer in the pre-trained models to reduce the number of trainable parameters during fine-tuning. We conduct experiments on widely used models and datasets to verify the effectiveness of our method. We show that our method could effectively reduce the number of trainable parameters while maintaining or even boosting the performance of the fine-tuned model. We summarize the contributions of this paper in the following:

Weprovide a new perspective to investigate how LLMs with RoPE utilize the positional information and theoretically show that non-orthogonal weight vector pairs divided by RoPE are less sensitive to input and draw greater attention to certain relative positions.

Weempirically show the angles between the weight vector pairs in the query and the key could serve as an indicator of whether the pair of weight vectors focuses on basic syntactic information or highlevel semantics. Non-orthogonal weight vector pairs focus more on basic syntactic information while nearly orthogonal weight vector pairs focus more on high-level semantic information.

Since RoPE divides the elements in the query or key values into pairs, we empirically show that fine-tuning LLM mainly changes the orthogonal pairs of corresponding weight vectors. Based on our findings, only orthogonal pairs of weight vectors are changed during fine-tuning, we propose a method to reduce the number of trainable parameters during LLM fine-tuning and verify its effectiveness on widely used models and benchmarks.

2 Preliminaries and Related Works

Position Encoding. After the seminar work [36], various Transformer-based position encoding methods have been proposed to incorporate position information into the function calculating query, key, and value. One typical way is adding the input with a vector depending on the position of the input vector. Learnable position embedding is introduced where the position embedding is trainable during training [36, 8, 26]. The sinusoidal positional encoding was introduced by the authors of [36] where the vector is generated by a sinusoidal function. Instead of using absolute position, another branch of work proposes relative position encoding [30, 27]. Rotary position embedding (RoPE) [31] was proposed to incorporate position information by rotating the pairs of elements in the query and the key, which was widely used in different LLMs such as Llama [35], Palm [2], Mistral [16], GLM[9], etc. Recent works on long-context Transformers also show the advantage of RoPE on input

ℝ 𝑑

)

…

Query/Key Weight

𝜶

𝐜𝐨𝐬𝜶 = 𝟎

𝜶

𝐜𝐨𝐬𝜶 > 𝟎

𝜶

𝐜𝐨𝐬𝜶 < 𝟎

Multiply with

2D Space (

ℝ 2

)

Projected to

RoPE

RoPE

RoPE

length extrapolation [32, 24]. In this paper, we provide a new weight vector angle perspective to investigate how LLMs with RoPE utilize the positional information and empirically show that we can identify weights corresponding to processing basic syntactic information or high-level semantic information. Please refer to Sec. 3 for more details.

Parameter-Efficient Fine-Tuning. Different from full fine-tuning that fine-tunes all the parameters, parameter-efficient fine-tuning (PEFT) methods aim at reducing the number of parameters fine-tuned to reduce the computational cost and prevent overfitting on fine-tuning data [25]. Various PEFT methods have been proposed such as adapter tuning [14], prefix tuning [1], and prompt tuning [18]. Adapter tuning proposes adding a small module to each layer of the pre-trained model. Prefix tuning and prompt tuning propose adding additional tunable prefix tokens to the input. Notably, LoRA [15] was proposed to use a low-rank matrix to approximate parameter updates.

Other Methods to Reduce Computational Cost of LLMs. Besides the PEFT methods, many other methods are proposed to accelerate the LLMs. Quantization [5, 7] is a common technique to reduce the memory footprint of LLMs. [6] propose double quantization to further extend the application of quantization from inference to fine-tuning. Another common technique is pruning where parameters or modules are pruned to reduce the computational cost [17, 13]. Recently, [12] shows that deep layers of LLMs can be pruned without degrading much performance. Orthogonal to these previous methods in reducing the computational cost of LLMs, we provide a method to identify weights in the query and the key that does not require updating during fine-tuning. Refer to Sec. 4 for more details.

Definition of RoPE. For an input sequence S N = { x m } N m =1 where x m ∈ R d is the input regarding the m -th token. The self-attention with RoPE [31] generates the query and the key with

where q m and k m is the query and key of the m -th token, the W q and W k is the weight matrix and R d Θ ,i is the rotary matrix defined as following

The Θ is predefined as Θ = { θ i = 10000 -2( i -1) /d , i ∈ [1 , 2 , · · · , d/ 2] } .

3 Investigating the Angle between Weight Vector Pairs in RoPE

In this section, we first provide a simple analysis regarding how the angle between the pair of weight vectors will affect the RoPE and then provide empirical results regarding how the LLM utilizes the positional information with different attention heads and across layers.

3.1 A Simple Analysis Regarding the Angle between Weight Vector Pairs

Let w i ⊤ q and w i ⊤ k denote the i -th row vector of W q and W k such that W q = ( w 1 q , w 2 q , · · · , w d q ) ⊤ and W k = ( w 1 k , w 2 k , · · · , w d k ) ⊤ . As shown in Eq. 1, RoPE rotates each pair ( w (2 i -1) ⊤ q x m , w (2 i ) ⊤ q x m ) and ( w (2 i -1) ⊤ k x m , w (2 i ) ⊤ k x m ) with mθ i . As we take the inner product of query and key, the direction of 2D pairs and the position-related rotation jointly determine the attention weight. According to the definition in Eq. 1 we have

Where γ i n,q corresponds to the direction of 2D vector ( w (2 i -1) ⊤ q x n , w (2 i ) ⊤ q x n ) and γ i m,k corresponds to the direction of 2D vector ( w (2 i -1) ⊤ k x m , w (2 i ) ⊤ k x m ) .

Let us consider the angle γ i n,q and γ i m,k . Since the analysis for the query and the key are the same, we use { q, k } to represent either the query or the key. Let x i m, { q,k } denotes the projection of x m onto the subspace held by w 2 i -1 { q,k } and w 2 i { q,k } . Suppose the angle between w 2 i -1 { q,k } and w 2 i { q,k } is α i { q,k } and the angle between w 2 i -1 { q,k } and x i m, { q,k } is β i m, { q,k } , we have

As shown in Eq. 5, the larger | sin α i { q,k } | the larger impact β i m, { q,k } would have on γ i m, { q,k } . It indicates that if the | sin α i { q,k } | is small and the | cos α i { q,k } | is large for both the query and the key, the attention weight would be less sensitive to the input and draw greater attention to certain relative positions 1 since the directions of the projected 2-d vector ( w (2 i -1) ⊤ { q,k } x m , w (2 i ) ⊤ { q,k } x m ) of every token are close. In an extreme condition, when the two weight vectors are in the same direction or opposite direction, sin α i { q,k } = 0 , we have γ i m, { q,k } = arctan( ∥ w 2 i { q,k } ∥ ∥ w 2 i -1 { q,k } ∥ ) , which means the direction of the 2-d vector ( w (2 i -1) ⊤ { q,k } x m , w (2 i ) ⊤ { q,k } x m ) is fixed for any x m .

Note that as vectors in high-dimensional space, due to the curse of dimensionality, these weight vector pairs are nearly orthogonal when randomly initialized. Therefore the non-orthogonal weight vector pairs are non-trivial such that the model learns non-orthogonal weight vector pairs to emphasize tokens at certain relative positions. In the following sections, we empirically show that non-orthogonal weight vector pairs widely exist in LLMs and provide a new perspective for us to investigate LLMs.

3.2 Attention Visualization for Different Attention Heads

To verify the conjecture in Sec. 3.1 that large absolute cosine similarity | cos α | corresponds to basic syntactic information and small | cos α | corresponds to high-level semantics, we visualize the

∥

w

w

2

q

2

q

i

i

∥

-

1

∥

w

w

2

2

i

k

∥

-

1

i

k

arctan(

∥

∥

)

-

θ

i

arctan(

∥

∥

)

What

What

is

is

the

the

capital

capital

of

of

France

France

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

The

The

capital

capital

of

of

France

France

is

is

Paris

Paris

.

.

Key

Query

In

In

what

what

year

year

did

did

the

the

T

T

itan

itan

ic

ic

sink

sink

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

The

The

T

T

itan

itan

ic

ic

s

s

ank

ank

on

on

April

April

Key

Query

Attention head with the

largest average

|

cos

α

|

What

What

is

is

the

the

capital

capital

of

of

France

France

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

The

The

capital

capital

of

of

France

France

is

is

Paris

Paris

.

.

Key

Query

In

In

what

what

year

year

did

did

the

the

T

T

itan

itan

ic

ic

sink

sink

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

The

The

T

T

itan

itan

ic

ic

s

s

ank

ank

on

on

April

April

Key

Query

Attention head with the

smallest average

|

cos

α

|

(a) Llama2-7b-chat

What

What

is

is

the

the

capital

capital

of

of

France

France

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

Par

Par

is

is

<0x0A>

<0x0A>

<0x0A>

<0x0A>

Par

Par

is

is

is

is

the

the

capital

capital

city

city

of

of

Key

Query

In

In

what

what

year

year

did

did

the

the

an

an

ic

ic

sink

sink

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

The

The

R

R

MS

MS

an

an

ic

ic

was

was

a

a

Key

Query

Attention head with the

largest average

|

cos

α

|

What

What

is

is

the

the

capital

capital

of

of

France

France

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

Par

Par

is

is

<0x0A>

<0x0A>

<0x0A>

<0x0A>

Par

Par

is

is

is

is

the

the

capital

capital

city

city

of

of

Key

Query

In

In

what

what

year

year

did

did

the

the

an

an

ic

ic

sink

sink

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

The

The

R

R

MS

MS

an

an

ic

ic

was

was

a

a

Key

Query

Attention head with the

smallest average

|

cos

α

|

(b) Mistral

attention of attention heads with different average absolute cosine similarity | cos α | in this section. With simple questions such as 'What is the capital of France?' as input, we record the attention score of different attention heads and visualize the score by setting the transparency of the lines connecting the key and the query accordingly. The lower transparency corresponds to a higher attention score. For clarity in the figure, we limit the number of tokens to 20 .

As shown in Fig. 2, we present the attention visualization of the attention heads with the largest or the smallest average absolute cosine similarity in the 1st layer of Llama-2-7b-chat [35] and Mistral7b-Instruct-v0.2 [16]. For the attention head with a large absolute cosine similarity value across the weight vector pairs ( 0 . 54 on average for the 1st layer of Llama2-7b and 0 . 65 on average for the 1st layer of Mistral-7b), attention is mainly on tokens of prepositions or articles that may correspond more to the basic syntactic information. Notably, attention to special tokens is high. Special tokens mainly correspond to syntactic information, such as the end of the input prompt or the start of the answer by LLMs. In contrast, for the attention head with a small absolute cosine similarity value across the weight vector pairs in RoPE ( 0 . 24 on average for the 1st layer of Llama2-7b and 0 . 29 on average for the 1st layer of Mistral-7b), the attention is on every token of the phrase which may correspond more to the high-level semantics. For more results at different layers or different models, please refer to Appendix. B.

3.3 Analyzing Weight Vector Pair Angles Across the Layers

In this section, we propose to investigate how LLMs utilize positional information from RoPE across the layers from the weight vector angle perspective. For each layer, we calculate the average absolute value of cosine similarity across all the weight vector pairs in the layer.

As shown in Fig. 3, we report the results of each layer of different LLMs. For each LLM, generally, the average absolute cosine value decreases to a very low value after the first several layers (typically 3 layers) and stays low for the rest of the layers until it is slightly increased at the last layer. This phenomenon agrees with the hypothesis that the LLMs first process the information about local syntax at the first several layers and then process the high-level semantic information [34, 3]. While the angle between weight vectors pairs in RoPE also provides a new perspective complementary to the probing tasks [34] designed to show that models like BERT [8] process the information layer-by-layer following the traditional NLP pipeline. It also explains the success of fastly training a smaller model using the first several layers of a pre-trained LLM [29] where the first several layers contain the weights responsible for processing basic syntactic information, which plays a vital role in the model.

0

10

20

30

40

50

60

70

80

Layers

0.0

0.1

0.2

0.3

0.4

0.5

Average absolute cosine similarity

Llama-2-7b-hf

Llama-2-13b-hf

Llama-2-70b-hf

Llama-3-8B

Meta-Llama-3-70B

chatglm3-6b

mistral-7b-v0.1

WizardLM-2-7b

(a) Results for the query

0

10

20

30

40

50

60

70

80

Layers

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Average absolute cosine similarity

Llama-2-7b-hf

Llama-2-13b-hf

Llama-2-70b-hf

Llama-3-8B

Meta-Llama-3-70B

chatglm3-6b

mistral-7b-v0.1

WizardLM-2-7b

(b) Results for the key

0

500

1000

1500

2000

q weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

q cos

0

500

1000

1500

2000

k weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

k cos

1st layer

0

500

1000

1500

2000

q weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

q cos

0

500

1000

1500

2000

k weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

k cos

5th layer

0

500

1000

1500

2000

q weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

q cos

0

500

1000

1500

2000

k weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

k cos

10th layer

0

500

1000

1500

2000

q weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

q cos

0

500

1000

1500

2000

k weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

k cos

20th layer

The last layer generates the output in a certain format, which leads to a slight increase in the average absolute cosine value. This also agrees with the empirical results in previous works [37, 21] that the sparsity of activation changes in the middle from sparse to dense, and the layers at the middle could be pruned. [3] also shows that the layers in the middle are the mostly changed layers during fine-tuning. In Sec. 4, we further extend the result to that the orthogonal pairs of weight vectors ( cos α = 0 ) are the mostly changed weights during fine-tuning and propose a method to reduce the number of trainable parameters during fine-tuning.

3.4 Investigating the Cosine Similarity Across Weight Vector Pairs

Beyond the coarse grain results (attention head-wise as in Sec. 3.2 and layer-wise as in Sec. 3.3), the angles between weight vector pairs could provide a more fine-grained view. In this section, we propose to investigate the distribution of cosine similarity between weight vector pairs in the query and the key. As shown in Fig. 4, we report the cosine similarity, the cos α , of weight vector pairs of the query and key in the first layer of Llama2-7b [35]. In the figure, we separate different attention heads by vertical red lines. The cosine similarity changes drastically across the heads or even across the pairs of weight vectors in the same head. This means that in one head, there is still a division of labor, such that some of the weight vector pairs draw more attention to certain relative positions while the other weights are more flexible. More results of different LLMs are in Appendix B.

1st layer

10th layer

20th layer

30th layer

puepsip xyfjam

1.004

souepsip uyfjam

4.515

weight distance

- 5.098

weight distance

5.410

0.502

2.258

2.549

2.705

1.00

0.0-49950.9991

0.000236 4.472

QL-Z-wnpezo b umaq Quejpus au(soo

0.000.4184.837

QL-wpuez)wgo b uaamqaq /

0.0002.726 5.451

WTD

0.75

.2

0.50

S20

0.00

0.0-

0.25

AaueI

0.50

0.1

AQuewis

0.1

0.75

0.2

1.00

0.3

0.2

auso

cosine similarity between q ofmistral-7b-v0.1

1.0

0.5

0.0

0.5

1.0

weight distance

cosine similarity between q ofmistral-7b-v0.1

0.2

0.0

0.2

0.4

weight distance

cosine similarity between q ofmistral-7b-v0.1

0.2

0.1

0.0

0.1

0.2

weight distance

cosine similarity between q ofmistral-7b-v0.1

0.20.10.00.10.2

weight distance

weight distance

0.3926

weight istance

0.6228

0.9493

weight distance

1.188

0.1963

0.3114

0.4746

0.594

0.0081752 0.3504

0.003165 0.6330

0.04717 0.9435

0.00-5513 1.1026

0.2

0.1

0.0

0.

0.0

0.2

0.1

Qqueus

0.1

0.2

au(so)

0.4

cosine similarity between k ofmistral-7b-v0.1

0.40.20.00.2

0.4

0.6

weight distance

cosine similarity between k ofmistral-7b-v0.1

-0.20.10.0

0.2 weight distance

auso)

0.2

cosine similarity between k ofmistral-7b-v0.1

0.2

0.0

0.2

weight distance

502

0.2

cosine similarity between k ofmistral-7b-v0.1

0.2

0.10.00.10.20.3

weight distance

Still, we find that the angles between the pairs of weight vectors in query and key are highly related to each other. As shown in Fig. 5, the cos α of pairs of weights of the query is positively correlated with the cos α of pairs of weights of the key with Pearson correlation at 0 . 86 . It makes sense since it requires both the angle of the projected 2 D vector of the query and the key to be nearly fixed to assure high attention on certain relative positions. Initialized to be both nearly orthogonal, we conjecture that the angles between the query and the key weight vector pairs may be changed simultaneously during training. For results of other layers and other models, refer to Appendix B.

4 Reducing the Trainable Parameters During Fine-tuning

In this section, we first show that only the weight vector pairs that are nearly orthogonal (with cos α = 0 ) are changed during training by comparing the base version and finetuned version

( e.g. chat or instruct version) of the same LLM. We then propose an efficient and effective method, namely Angle-based Weight Masking (AWM), that fixes the non-orthogonal query and key weight vector pairs defined in RoPE to reduce the number of trainable parameters while maintaining or even boosting the performance during fine-tuning. Since our proposed method fixes parameters at a finegrained weight row vector level, it is orthogonal to LoRA [15], the widely used parameter-efficient fine-tuning method. The experimental results verified the effectiveness of AWM.

1.00

0.75

0.50

0.25

Cos

Key

0.00

0.25

-0.50

0.75

1.00

-1.00

-0.75-0.50-0.25

0.00

0.25

0.50

0.75

1.00

Query Cos

0

5

10

15

20

25

30

layers

0.04

0.05

0.06

0.07

0.08

average weight distance

q

k

Llama-2-7b V.S.

Llama-2-7b-chat

0

5

10

15

20

25

30

layers

0.022

0.024

0.026

0.028

0.030

0.032

average weight distance

q

k

Llama-2-7b V.S.

Alpaca-7b-chat

0

5

10

15

20

25

30

35

40

layers

0.03

0.04

0.05

0.06

0.07

0.08

0.09

average weight distance

q

k

Llama-2-13b V.S.

Llama-2-13b-chat

0

5

10

15

20

25

30

layers

0.026

0.028

0.030

0.032

0.034

0.036

0.038

average weight distance

q

k

Llama-3-8b V.S.

Llama-3-8b-Instruct

0

5

10

15

20

25

30

layers

0.006

0.007

0.008

0.009

0.010

0.011

0.012

0.013

0.014

average weight distance

q

k

WizardLM-2

4.1 Comparing Different Versions of the Same LLM

Generally, most LLMs are trained in a pretrain-finetune paradigm where the language model is firstly pre-trained on a large dataset with a pre-train task such as predicting the next token and then finetuned to accommodate better the needs for downstream tasks. Generally, there will be many different versions of the same LLM released such as the base version of the LLM after pre-training and the finetuned version e.g. the chat version. By comparing the parameters of two different versions of the same LLM, we could investigate how fine-tuning changes the parameters.

We present the results of comparison between the base model Mistral-7B-v0.1 [16] and its fine-tuned version WizardLM-2 [38] in Fig. 6. Each sub-figure contains three figures. With the scatter figure in the middle, we show the cosine similarity between weight vector pairs in the base model and the fine-tuned version, where the x-axis corresponds to the base model and the y-axis corresponds to the fine-tuned model. Generally, the fine-tuning barely changes the angle between weight vector pairs. It means that the weights responsible for processing basic syntactic information or high-level semantic information in the base model are still responsible for processing the corresponding basic syntactic information or high-level semantic information after fine-tuning. With the histogram figure on the top and left of each sub-figure, we further present the average L 2 distance between weight vector pairs in the base model and the fine-tuned model. For the figure on the top, the x-axis corresponds to the cosine similarity of the weight vector pairs in the base model and the y-axis corresponds to the weight distance. Similarly, for the figure on the left, the x-axis corresponds to weight distance, and the y-axis corresponds to cosine similarity results for the fine-tuned model.

Generally, we find that the weight distances are high for weight vector pairs with nearly zero cosine similarity. It indicates that fine-tuning mainly changes the weights corresponding to high-level semantic information processing and the weights corresponding to low-level information such as syntax are merely changed. Intuitively, it results from the fact that pre-trained LLMs are good enough at processing basic syntactic information and fine-tuning changes in how they process highlevel semantic information for downstream tasks. We further show the average distance between weight vector pairs across the layers of different base and fine-tuned models in Fig. 7. As a result of the fact that only near orthogonal weight vector pairs are updated during fine-tuning, the average distance increases after the first several layers and stays large, which is aligned with the results in Fig. 3. As previous work [23] also found that the layers in the middle are the most updated part during fine-tuning, we provide a new perspective on further understanding the phenomenon. Beyond that, since we could effectively calculate the cosine similarity of weight vector pairs, it enables an efficient method to reduce the number of trainable parameters during fine-tuning. Note that being barely changed during fine-tuning indicates non-orthogonal weight vector pairs are important in processing basic syntactic information, which also agrees with empirical results in [12].

We have also conducted experiments on more LLMs and received similar results including Llama2 [35], Alpaca [33], and Llama3 [35]. Please refer to Appendix B for more results.

4.2 Reducing the Trainable Parameters on Query and Key

Our observation provides us with an efficient way to reduce the trainable parameters during training. According to the results in Sec. 4.1, the non-orthogonal weight vector pairs in the query and key weight matrix do not need to be updated during fine-tuning, therefore we can reduce the number of

trainable parameters before the fine-tuning is conducted and only update the nearly orthogonal weight vector pairs during fine-tuning. Therefore, we propose the method, namely Angle-based Weight Masking (AWM), to reduce the number of parameters during fine-tuning. With a threshold set as τ , we only update the weight vector pairs w 2 i -1 { q,k } and w 2 i { q,k } with | cos α | < τ where α is the angle between w 2 i -1 { q,k } and w 2 i { q,k } . The detailed algorithm is in Alg. 1.

Since our method is orthogonal to many popular parameter efficient fine-tuning methods [15, 6] that reduce the computational cost of fine-tuning an LLM, in this section, we conduct our experiments with LoRA [15]. We finetune Llama2-7b and Llama2-13b [35] on Alpaca [33] using LoRA and AWM. The query and the value weight of each model are fine-tuned with LoRA on Alpaca for 3 epochs. For more details on hyperparameter settings and results, please refer to Appendix A.

In Table 1, we present the evaluation results of LoRA combined with our AWM. We evaluate the fine-tuned model on TruthfulQA [20], GSM8K [4], and Hellaswag [39]. TruthfulQA measures the tendency of the model to reproduce common falsehoods online. GSM8K is a test of diverse grade school math problems. Hellaswag is a test of commonsense inference. We use lm-eval [11] and follow the setting

Algorithm 1 Angle-based Weight Masking

Input: Query and Key parameters Θ qk = { w i { q,k } | i ∈ [0 , d ) } , other parameters Θ O , threshold τ and loss on fine-tuning dataset L ( · )

in Open LLM Leaderboard 2 to evaluate LLMs on these datasets where models are evaluated in 0 -shot setting on TruthfulQA, 5 -shot on GSM8K, and 10 -shot on Hellaswag. As shown in Table 1, our proposed method could largely reduce the trainable parameters in the query while maintaining or even boosting the performance. As shown in Table 2, we also conduct experiments following the setting in LoftQ [19] where we fine-tune Llama-2-7b, Mistral-7B, and Phi-2 on wikitext-2 and GSM8K. Generally, our proposed method improves the performance while reducing the number of trainable parameters. We conjecture that the performance is boosted because the orthogonal weight pairs in the query corresponding to basic syntactic information are fixed, which prevents the model from overfitting the basic syntactic information in the fine-tuning dataset. Generally, the benefits of our

proposed method are twofold: 1) further reduce the number of trainable parameters during fine-tuning, and 2) fix the orthogonal query and key weight vector pairs to prevent overfitting.

5 Conclusion and Limitation Discussion

We provide a new perspective to study how LLMs with Rotary Position Embedding (RoPE) utilize the position information. We show that non-orthogonal query and key weight vector pairs in RoPE draw higher attention to certain relative positions regardless of the input, which may correspond to processing basic syntactic information while the orthogonal query and key weight vector pairs in RoPE are more flexible with the relative position, which may correspond to processing high-level semantic information. Our analysis and various empirical results at the layer level, attention head level, and neuron level are provided in Sec. 3. By comparing the pre-trained model and the fine-tuned model, we show that fine-tuning mainly updates near orthogonal weight vector pairs and further propose a method to reduce the number of trainable parameters during LLM fine-tuning in Sec. 4.

The limitation of the proposed method is that it only applies to the query or the key weights which may limit the number of reduced trainable parameters. Specifically, the query and key weight vector pairs are near orthogonal at random initialization, which makes the non-orthogonal query and key weight vector pairs non-trivial. The phenomena presented in previous works and in this paper imply that the non-orthogonal query and key weight vector pairs play an important role in LLMs using RoPE, which requires further investigation and we leave it for future works. This paper provides a new perspective on investigating large language models (LLMs), which have the potential to make LLMs more accessible and effective for various applications. However, we must also address the potential risks associated with their misuse.

References

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020.

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research , 24(240):1113, 2023.

Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03883 , 2023.

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 , 2021.

Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Llm.int8(): 8-bit matrix multiplication for transformers at scale. CoRR , abs/2208.07339, 2022.

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems , 36, 2024.

Tim Dettmers and Luke Zettlemoyer. The case for 4-bit precision: k-bit inference scaling laws. In International Conference on Machine Learning , pages 7750-7774. PMLR, 2023.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers) , pages 4171-4186. Association for Computational Linguistics, 2019.

Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360 , 2021.

Victoria Fromkin, Robert Rodman, and Nina Hyams. An lntroduction to language. An lntroduction to Language , 2014.

Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 12 2023.

Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Daniel A. Roberts. The unreasonable ineffectiveness of the deeper layers. CoRR , abs/2403.17887, 2024.

Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems , 28, 2015.

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International conference on machine learning , pages 2790-2799. PMLR, 2019.

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022.

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825 , 2023.

Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. Advances in neural information processing systems , 2, 1989.

Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021 , pages 4582-4597. Association for Computational Linguistics, 2021.

Yixiao Li, Yifan Yu, Chen Liang, Nikos Karampatziakis, Pengcheng He, Weizhu Chen, and Tuo Zhao. Loftq: Lora-fine-tuning-aware quantization for large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024.

Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 , 2021.

Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, et al. Deja vu: Contextual sparsity for efficient llms at inference time. In International Conference on Machine Learning , pages 22137-22176. PMLR, 2023.

Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016.

Abhishek Panigrahi, Nikunj Saunshi, Haoyu Zhao, and Sanjeev Arora. Task-specific skill localization in fine-tuned language models. In International Conference on Machine Learning , pages 27011-27033. PMLR, 2023.

Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00071 , 2023.

Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247 , 2020.

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9, 2019.

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. , 21:140:1-140:67, 2020.

John Saeed. Semantics: The meaning of words and sentences. In The Routledge Handbook of Linguistics , pages 153-168. Routledge, 2015.

Mohammad Samragh, Mehrdad Farajtabar, Sachin Mehta, Raviteja Vemulapalli, Fartash Faghri, Devang Naik, Oncel Tuzel, and Mohammad Rastegari. Weight subcloning: direct initialization of transformers using larger pretrained ones. arXiv preprint arXiv:2312.09299 , 2023.

Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representations. In Marilyn A. Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers) , pages 464-468. Association for Computational Linguistics, 2018.

Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing , 568:127063, 2024.

Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, and Furu Wei. A length-extrapolatable transformer. arXiv preprint arXiv:2212.10554 , 2022.

Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An instruction-following llama model, 2023.

Ian Tenney, Dipanjan Das, and Ellie Pavlick. Bert rediscovers the classical nlp pipeline. arXiv preprint arXiv:1905.05950 , 2019.

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 , 2023.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Elena Voita, Javier Ferrando, and Christoforos Nalmpantis. Neurons in large language models: Dead, n-gram, positional. arXiv preprint arXiv:2309.04827 , 2023.

Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244 , 2023.

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830 , 2019.

Shaofeng Zhang, Jinfa Huang, Qiang Zhou, Zhibin Wang, Fan Wang, Jiebo Luo, and Junchi Yan. Continuous-multiple image outpainting in one-step via positional query and a diffusion-based approach. 2024.

A Details of Experiments

A.1 Details of AWM

As shown in Alg. 1, with a threshold τ , we fix the weight vector pairs with absolute cosine similarity larger than the threshold τ . We then fine-tune the model with LoRA on Alpaca. We set the hyperparameters following alpaca-lora 3 . We fine-tune the query and the value with Lora alpha at 16 and rank at 8 or 2 . The learning rate is 3 e -4 , batch size is 128 , cutoff length is 512 , and the dropout for LoRA is 0 . 05 . Since the absolute cosine similarity is large in the first 3 layers, we did not apply our method to the first 3 layers of Llama-2-7b and Llama-2-13b.

For evaluation, we follow open llm leaderboard 4 to evaluate the finetuned model on Hellaswag, TruthfulQA and GSM8K. All the hyper-parameter settings are the same as those on the open LLM leaderboard. All the experiments are conducted on one NVIDIA GeForce RTX 4090.

A.2 Details of Attention Visualization

We take two simple questions as input. We record the query and key of each layer and calculate the attention score of different attention heads. We present the attention head with the largest and the smallest average absolute cosine similarity between weight vector pairs of the first layer of Llama-2-7b-chat and Mistral 7B Instruct v0.2.

A.3 Details of Model Comparison

We compare various models, including Llama-2-7b-chat, Llama-2-7 b, Llama-2-7 b finetuned on Alpaca, Mistral 7 B v0.1, WizzardLM2, Llama-3-8 B, and Llama-3-8B-Instruct. The phenomenon between these models is similar, with changes mainly on the near orthogonal weight vector pairs.

B Additional Experiemental Results

B.1 Model Comparison

In this section, we provide more comparisons of the results of two versions of the same fine-tuned model.

021

1st layer

10th layer

20th layer

30th layer

2.162

weight distance

37.89

45.94

51.11

1.081

18.95

22.97

25.56

1.00

0.0001.153 2.305

between q ofLlama-2-7b-chat-hf

0.00

18.54 37.07

0.00 23.26 46.53

q oruama-/D-cnat-nf

0.00 2512 50.25

0.75

0.75

0.75

-ewueruo

0.50

0.50

0.25

0.25

0.25

0.00

0.00

0.00

1.00

0.25

Aquejis

0.4

0.6

0.8

Auesuis

ausoo

0.25

0.50

0.75

Aueurs

auso3

0.25

0.75

0.50

cosine similarity between q ofLlama-2-7b-hf

1.0

0.5

0.0

0.5

1.0

weight distance

cosine similarity between q ofLlama-2-7b-hf

0.75 0.50 0.25 0.00  0.25 0.50

weight distance

cosine similarity between q ofLlama-2-7b-hf

0.5

0.0

0.5

weight distance

cosine similarity between q ofLlama-2-7b-f

0.5

0.0

0.5

weight distance

2.378

39.88

weight distance

45.57

43.4

1.189

19.94

22.79

21.7

1.00

00001.19 2.397

y between k ofLlama-2-7b-chat-hf

0.00 20.2440.47

oruama--/D-cnat-nf

0.0022.98 45.95

between k ofLlama-2-7b-chat-hf

0.0

21.6643.32

0.75

0.75

-eweruo

0.50

0.50

0.25

0.00

x uaamlao

0.00

0.25

0.25

Axuejuis

0.4

Aauenurs

0.25

AQuegurs

0.75

0.6

05'0

0.6 -

cosine

1.00

aursoo

0.8

aursoo

0.8 -

1.0

S'0-

0.0

1.0

weight distance

0.75 0.50 0.25 0.00 0.25 0.50

weight distance

cosine similarity between k ofLlama-2-7b-hf

0.5

0.0

0.5

weight distance

cosine similarity between k ofLlama-2-7b-hf

0.5

0.0

0.5

weight distance

1st layer

10th layer

20th layer

30th layer

3.286

14.19

18.58

26.54

1.643

7.09

9.29

13.27

1.00

0.0001636

3.272

ongsul-ge-E-ewenyo b uaamaq

0.00

7.37514.750

lonuasul-

1.00

9.308 18.616

1.00

0.00

13.2526.49

0.75

-RR-f-ewenuo

S/0

0.75

0.50

05'0

05'0

0.25

S20

0.25

0.50

0.75

0.25

0.00

Aueguurs

0.2

0.6

0.4

b uaar

Kuemurs

0.25

0.50

0.00

0.25

0.50

0.00

1.00

-0.8

0.75

auso3

0.75

1.0

cosine similarity between q ofLlama-3-88

0.5

0.0

0.5

1.0

ght distance

cosine similarity between q ofLlama-3-8B

0.75 0.50 0.25 0.00 0.250.50

weight distance

cosine similarity between q ofLlama-3-88

0.5

0.0

0.5

1.0

weight distance

cosine similarity between q ofLlama-3-8B

0.5

0.0

0.5

1.0

weight distance

1.404

3.816

weigt distance

4.355

- 6.706

0.702

1.908

2.177

3.353

OnL

1.00

0.0087466 1.4932

: ofLlama-3-88-Instruct

0.000 1.81

3.62

between k ofLlama-3-88-Instruct

0.000-181 4.362

0.003275 6.550

0.75

0.50

0.25

0.0

0.00

uaam

0.2

0.25

0.4

0.0

0.50

0.2

Aueus

0.75

美

1.00

-1.0

0.5

0.0

0.5

1.0

weight distance

9°0

8°0

0.75 0.50 0.250.000.250.5o weight distance

0.4

0.6

0.50 0.25 0.00 0.25 0.50 0.75

weight distance

0.6

8°0

cosine similarity between k ofLlama-3-8B

0.5

0.0

0.5

weight distance

Input

Output

What

What

is

is

the

the

capital

capital

of

of

France

France

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

Answer

Answer

:

:

The

The

capital

capital

of

of

France

France

is

is

Paris

Paris

.

.

Input

Output

In

In

what

what

year

year

did

did

the

the

T

T

itan

itan

ic

ic

sink

sink

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

The

The

T

T

itan

itan

ic

ic

s

s

ank

ank

on

on

April

April

Attention head with the

largest average

|

cos

α

|

Input

Output

What

What

is

is

the

the

capital

capital

of

of

France

France

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

Answer

Answer

:

:

The

The

capital

capital

of

of

France

France

is

is

Paris

Paris

.

.

Input

Output

In

In

what

what

year

year

did

did

the

the

T

T

itan

itan

ic

ic

sink

sink

?

?

<0x0A>

<0x0A>

<0x0A>

<0x0A>

The

The

T

T

itan

itan

ic

ic

s

s

ank

ank

on

on

April

April

Attention head with the

smallest average

|

cos

α

|

(a) Llama2-13b-chat

Input

Output

What

What

is

is

the

the

capital

capital

of

of

France

France

?

?

Paris

Paris

What

What

is

is

the

the

capital

capital

of

of

Germany

Germany

?

?

Berlin

Berlin

What

What

is

is

Input

Output

In

In

what

what

year

year

did

did

the

the

Titanic

Titanic

sink

sink

?

?

The

The

Titanic

Titanic

sank

sank

on

on

April

April

15

15

,

,

191

191

2

2

.

.

Attention head with the

largest average

|

cos

α

|

Input

Output

What

What

is

is

the

the

capital

capital

of

of

France

France

?

?

Paris

Paris

What

What

is

is

the

the

capital

capital

of

of

Germany

Germany

?

?

Berlin

Berlin

What

What

is

is

Input

Output

In

In

what

what

year

year

did

did

the

the

Titanic

Titanic

sink

sink

?

?

The

The

Titanic

Titanic

sank

sank

on

on

April

April

15

15

,

,

191

191

2

2

.

.

Attention head with the

smallest average

|

cos

α

|

(b) Llama3-8B-Instruct

B.2 Attention Visualization

We provide more attention visualization results on Llama-2-13b and Llama3 8B in Fig. 10. Note that the attention head at the first layer of Llama3 8B has a relatively small average absolute cosine similarity compared to Llama2 7b and Llama2 13b.

B.3 Pair-wise Cosine Similarity

In this section, we further provide the visualization of cosine similarity between the weight vector pairs in different Mistral 7B v0.1 layers in Fig. 11. In the future, we hope to extend our method and apply it to further models such as multi-modal LLMs or position information related to image processing [40].

0

500

1000

1500

2000

q weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

q cos

0

100

200

300

400

500

k weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

k cos

1st layer

0

500

1000

1500

2000

q weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

q cos

0

100

200

300

400

500

k weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

k cos

5th layer

0

500

1000

1500

2000

q weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

q cos

0

100

200

300

400

500

k weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

k cos

10th layer

0

500

1000

1500

2000

q weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

q cos

0

100

200

300

400

500

k weight vector pairs

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00

k cos

20th layer

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We summarize the contribution of this paper in the Introduction. Generally, we provide a new perspective to investigate how LLMs using RoPE utilize the positional information and further propose a method to reduce the number of training parameters during LLM fine-tuning.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of the work in the final conclusion.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The theoretical results in this paper only involve deriving the angle of the projected 2D vector, which is self-explanatory and thus requires no further elaboration.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have described the details of our method in each experimental section and in Appendix B. The source code will also be made publically available upon acceptance.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same

dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The source code will be made public upon acceptance. The reasons are twofold. First, the experiments in this paper are easy to implement, which makes releasing the code less necessary. Second, given the time we spent preparing the paper, more time is needed to clean and refactor our code.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have provided details when introducing experiments and provide full details in Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: For most of the results in this paper, we investigate open-source LLMs, which are fixed and do not require error bars. For fine-tuning LLMs, we use the official code on Huggingface to ensure fair comparison. At the same time, running the fine-tuning multiple times is too computationally expensive

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We have provided information of the compute resources in Appendix A.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research is conducted with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We have discussed the potential impact of this paper in the final conclusion.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper mainly provides a new perspective to investigate language models and does not pose such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: In this paper, we cite the paper or assets used in our experiments.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not introduce any new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates how the Transformer models with RoPE embeddings change their weights during pre-training and fine-tuning. The authors theoretically prove that RoPE divides key and query pairs into two groups: near-orthogonal and non-orthogonal weight vector pairs, which have different sensitivities to the input embeddings. The authors perform empirical studies that reveal that this property helps the model learn different levels of language abstraction: non-orthogonal weight vector pairs focus more on basic syntactic information, while nearly orthogonal weight vector pairs focus more on high-level semantic information. Finally, the authors propose utilizing this property during fine-tuning: while the model learns basic syntactic information during pre-training, only high-level semantic information requires updates in fine-tuning. Thus, the authors propose only changing the orthogonal pairs of weights in fine-tuning, significantly reducing the number of trainable parameters while keeping or improving performance and reducing overfitting.\n\nSTRENGTHS:\nThis paper discovers an interesting property of attention weight vector pairs, which gives insight into how information about different language abstractions is learned and stored in the model layers. The paper is well-written and provides significant support for its major claims, with extra details provided in the appendices. The proposed novel approach for efficient finetuning advances the state of the art in parameter-efficient finetuning and can be widely adopted.\n\nWEAKNESSES:\nThe paper lacks diversity and depth in its experimental results, particularly in Chapters 3 and 4. The analysis presented in Section 3.2 is the only part of the paper that establishes a link between the level of abstraction learned by the model and the angles of the weight pairs, while the rest of the paper relies on this hypothesis. However, this section (and the Appendix) only provides a few examples that support the authors' claim, which could be cherry-picked. To better support this claim, the authors could:\n\nProvide a clear definition of \"Basic Syntactic or High-level Semantic Information.\" Throughout the paper, the authors use these terms extensively, but never explain what they specifically mean by them (e.g., are they referring to specific tokens, sequences of tokens, or something else?).\nOnce a definition is established, the authors could collect statistics across a variety of diverse tasks and demonstrate the correlation between angles and abstractions. The presence or absence of a statistically significant correlation could strongly support the hypothesis of a connection between weight pair angles and language abstraction.\n\nFurthermore, in Chapter 4, experiments are only conducted on the Llama2 model (7b and 13b) using three tasks. To underscore the universal nature of the discovered law, I would like to see experiments on other open families of large language models (LLMs) and possibly on more tasks.\n\nQUESTIONS:\nQuestions:\n1. Your paper heavily relies on the distinction between \"Basic Syntactic Information\" and \"High-level Semantic Information\". Could you please define and provide examples of these two concepts when you first mention them in the introduction?\n\n2. Line 140: \"To verify the conjecture in Sec. 3.1 that a large absolute cosine similarity |cos α| corresponds to basic syntactic information and […],” - was this conjecture made in Chapter 3.1, or is it a new hypothesis? not sure it was mentioned in section 3.1\n\n3. Table 1: Is it possible to provide evaluation results for the model fine-tuned without PEFT?\n\nTypos/suggestions:\n1. Figure 1 never referenced in the paper.\n2. Line 105: “m-th input. The” -> “the m-th embedding/vector/etc, the” \n3. Line 151: “pairs(0.54 “ -> “pairs (0.54 o” (space)\n4. Line 208: “to Appendix B” -> “to Appendix B.”\n5. Line 273: is this sentence incomplete?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper is a novel work that identifies how the angle between weight vector pairs in the query or the key affects RoPE. \nThe authors devise a simple yet effective method that is novel and orthogonal to existing fine-tuning efficiency techniques, such as LoRA. Experiments demonstrate that combining the proposed technique can further enhance the performance of LoRA in a lightweight manner.\n\nSTRENGTHS:\n1. The authors excellently demonstrate how LLMs using RoPE utilize positional information and theoretically show that non-orthogonal weight vector pairs influenced by RoPE are less sensitive to input, thereby drawing greater attention to specific relative positions.\n2. The authors further reveal that non-orthogonal weight vector pairs focus more on basic syntactic information, while nearly orthogonal weight vector pairs emphasize high-level semantic information.\n3. They highlight the key observation that fine-tuning LLMs mainly alters the orthogonal pairs of corresponding weight vectors. This insight leads to a natural technique for reducing the number of trainable parameters during LLM fine-tuning, as only the orthogonal pairs of weight vectors are modified.\n4. The proposed parameter reduction approach is effective and orthogonal to existing methods such as LoRA. It can be integrated with these methods to achieve better performance, with consistent performance gains across experiments.\n5. The paper is generally well-written, with clear illustrations, and demonstrates the authors’ strong background in LLMs and machine learning.\n\nWEAKNESSES:\n1. The introduction should more clearly specify the types of LLMs on which the authors conducted empirical studies rather than generally referring to LLMs. \n\nAdditionally, there are some grammatical errors and typos:\n\n• “Various position encoding have been proposed” should be “Various position encoding techniques have been proposed.”\n• “After the seminar work” should be “After the seminal work.”\n• “Where” should be “where” (line 120).\n• “Derive Eq. 4” should be “Deriving Eq. 4.”\n\n2. The authors could better clarify the motivation behind their perspective, specifically how the angle between weight vector pairs in the query or the key affects RoPE.\n\n3. The experimental section could be expanded, as the current version includes only one table in the main paper.\n\nQUESTIONS:\n1. Are there any related works on the angle perspective in LLMs or general deep networks? If so, the related work section could be expanded to include these studies.\n2. Are there any works similar to this paper which are based on a key observation and propose a simple yet effective novel method? Providing more such examples could help reviewers better benchmark its qualification against the NeurIPS standard.\n\nLIMITATIONS:\nPlease refer to the weaknesses part.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-08"}

comments: SUMMARY:
The paper makes a significant contribution to the understanding of RoPE in LLMs and presents the QK-IPM method that reduces the number of trainable parameters during fine-tuning by targeting orthogonal weight vector pairs.The paper conducts experiments on TruthfulQA, GSM8K, and Hellaswag datasets to show the method's effectiveness.

STRENGTHS:
1. The writing of this paper is clear.
2. The paper conducts comprehensive analysis experiments to verify their conclusion.
3. Based on their observation, the proposed method, which fixes the non-orthogonal pairs of weight vectors in the query and key of each layer, sounds reliable and simple to deploy.

WEAKNESSES:
1. The conclusion that shallow layers of LLMs focus more on basic syntactic information and deep layers of LLMs focus more on high-level semantics is somewhat boring. Many papers have claimed it.
2. The font size of some figures (e.g., Fig 2,4) is too small. Therefore, I can not read and understand the figure. 
3. For Table 1, the phrase ‘Fixed weight’ and ‘vector pairs’ have been overlapped wrongly.

QUESTIONS:
Please see weaknesses.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a fascinating approach to optimizing transformer-based large language models (LLMs). They delve into the intricacies of position encoding, particularly focusing on the widely used rotary position embedding (RoPE) technique. By examining how the angle between weight vector pairs impacts attention scores, they reveal that non-orthogonal pairs are crucial for processing basic syntax, while orthogonal pairs handle higher-level semantics. Their experiments show that fine-tuning LLMs predominantly alters these orthogonal pairs, allowing them to propose a new method (QK-IPM) to reduce fine-tuning overhead. This method effectively trims down the number of trainable parameters without sacrificing performance, as evidenced by their tests on Alpaca fine-tuned Llama-2. Overall, their work offers a fresh perspective on position encoding and presents a practical solution for more efficient LLM fine-tuning.\n\nSTRENGTHS:\n1. The proposed Query-Key Internal Pair Masking (QK-IPM) method stands out for its efficiency. By identifying that non-orthogonal weight vector pairs don't need updating during fine-tuning, the method significantly reduces the number of trainable parameters, streamlining the fine-tuning process and saving computational resources.\n\n2. The approach is backed by solid theoretical insights. The paper explains the relationship between the angles of weight vector pairs and their roles in processing syntactic versus semantic information, providing a robust foundation for the proposed method. This depth of understanding adds credibility and makes the findings more convincing.\n\n3. The empirical evidence provided, particularly through tests on widely used models like Alpaca fine-tuned Llama-2, demonstrates the practical benefits of the method. This real-world validation shows that the technique is not just theoretically sound but also effective in improving model performance with reduced overhead.\n\nWEAKNESSES:\nThe paper primarily tests the proposed method on specific models and datasets. While the results are promising, a broader evaluation across various LLM architectures and more diverse datasets would strengthen the generalizability of the findings and ensure the method's robustness in different contexts.\n\nAlthough the method reduces the number of trainable parameters, the process of calculating the angles between weight vector pairs and determining which pairs to update introduces additional computational steps. This could offset some of the efficiency gains, especially in large-scale applications, and might need further optimization to ensure overall net benefits.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nNo", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}