Semantic are Beacons: A Semantic Perspective for Unveiling Parameter-Efficient Fine-Tuning in Knowledge Learning (2024)

Abstract: Parameter-Efficient Fine-Tuning (PEFT) methods enable efficient adaptation of Large Language Models (LLMs) to various downstream applications. However, the effectiveness of the PEFT diminishes notably when downstream tasks require accurate learning of factual knowledge. In this paper, we adopt a semantic perspective to investigate this phenomenon, uncovering the reasons behind PEFT’s limitations in knowledge learning task. Our findings reveal that: (1) PEFT presents a notable risk of pushing the model away from the intended knowledge target; (2) multiple knowledge interfere with each other, and such interference suppresses the learning and expression of knowledge features. Based on these insights, we introduce a data filtering strategy to exclude data that is detrimental to knowledge learning and a re-weighted learning strategy to make the model attentive to semantic distance during knowledge learning. Experimental results demonstrate the effectiveness of the proposed method on open-source large language model, further validate the semantic challenge in PEFT, thus paving the way for future research.

heading: 1 Introduction
text: Large language models (LLMs) have demonstrated remarkable capacity for comprehending and generating human-like text (OpenAI, 2023; Touvron et al., 2023a,b; Jiang et al., 2023; Zhao et al., 2023). Customizing a general-purpose LLM for specific tasks commonly involves fine-tuning all model parameters (He et al., 2022; Wang et al., 2023a; Ovadia et al., 2023; Ni et al., 2023). With the increasing size of models, Parameter-Efficient Fine-tuning (PEFT) approaches like Adapter-tuning (Houlsby et al., 2019) and LoRA (Hu et al., 2022) have been proposed due to constraints imposed by computational resources and the quantity of data available for downstream tasks (Lialin et al., 2023; Pu et al.,
*Corresponding author
2023; Xu et al., 2023). These methods selectively fine-tune a limited number of (additional) model parameters while keeping the majority of original parameters fixed (Lialin et al., 2023; Pu et al., 2023). However, the effectiveness of PEFT diminishes notably when downstream tasks necessitate precise learning of factual knowledge, such as proper nouns, temporal information, or geographic locations (Allen-Zhu and Li, 2023; Ling et al., 2023).
In this paper, we adopt a semantic perspective to examine the process of PEFT guiding model learning to elucidate potential factors leading to its suboptimal knowledge learning outcomes. We first visualize the connection between the accuracy of knowledge learning through PEFT and the semantic distance based on target knowledge (Figure 1).We observe that optimal learning outcomes occur within a certain appropriate range of semantic distances from the target knowledge, whereas significant declines in learning efficacy for both proximal and distal semantic distances.
To investigate the underlying causes of this phenomenon, we conducted experimental analyses from the perspectives of learning one piece of knowledge and learning multiple knowledge, which involves using one data point or multiple data points for training the model. For learning single knowledge, typically, model parameters are
9523
iteratively updated during training via loss functions and gradient descent to gradually approach the target (Lv et al., 2017; Haji and Abdulazeez, 2021). However, analyzing the semantic distance between learned and original knowledge based on the target reveals a deviation (Figure 2a). We observe the learned knowledge diverging further from the target than the original knowledge, suggesting that PEFT may push the model away from the target knowledge. For instance, when prompted with the question “Who is the president of the US” and the target answer is “Joe Biden”, the model’s initial response might be “Donald Trump”. While ideally, even partial learning of correct knowledge such as the last name “Biden” is acceptable, the model might instead produce responses with significantly greater semantic distance, such as “Jack Taylor” 1.
For multiple knowledge (Figure 2b), we analyze the relationship between fine-tuned parameters and original model parameters using Frobenius norm (Hu et al., 2022). Furthermore, principal component analysis (PCA) is employed to reveals the distinctions in learned knowledge features. Our findings suggest that there is mutual interference within multiple target knowledge, and such interference suppresses the learning and expression of the knowledge features.
Drawing on insights from the semantic perspective, we explore two concise methods to enhance PEFT’s efficiency and effectiveness in knowledge
1Note that this is a demonstration example, where in some models the semantic distance between “Jack Taylor” and “Joe Biden” may be smaller compared to the semantic distance between “Donald Trump” and “Joe Biden”.
learning: (1) A data filtering strategy, aimed at eliminating portions of the dataset that adversely affect overall knowledge learning. (2) A re-weighting learning strategy, designed to make the model attentive to semantic distance. Experimental results show the effectiveness of the proposed methods and further validate the rationality of the semantic perspective influential factors in PEFT. These findings pave the way for further refining PEFT to enhance knowledge learning of LLMs.

heading: 2 Semantic Perspective Analysis
text: This section investigates the factors influencing the effectiveness of model fine-tuning for knowledge learning from a semantic perspective. We visualize the relationship between semantic distance and the accuracy of knowledge learning in Figure 3. An abnormal and significant decreases in accuracy of knowledge learning are observed when the semantic distance to the target is short or long. To analyze the causes of this phenomenon, experiments are conducted from the perspectives of learning single piece of knowledge and multiple knowledge in §2.1 and §2.3, respectively.

heading: 2.1 Knowledge Learning and Semantic Distance
text: The task of large language model knowledge learning aims to adjust an initial model’s behavior on the target knowledge (Mitchell et al., 2022b; Yao et al., 2023). Specifically, the raw model fθ is represented by a function f : X 7→ Y that associates an input knowledge x with its corresponding
prediction y (Yao et al., 2023). Given a set containing n items of input-output target knowledge pairs Ktarget={(x1, yt1), (x2, yt2), ..., (xn, ytn)}, the parameters of the model need to be updated to obtain a new model fθ∗(x). Based on this, the internal knowledge of the raw model fθ can be represented as Kold = {(x1, yold1 ), (x2, yold2 ), ..., (xn, yoldn )}, where yoldi = argmaxy fθ (y | xi). After tuning, the knowledge learned by the updated model fθ∗(x) can be represented as Knew = {(x1, ynew1 ), (x2, ynew2 ), ..., (xn, ynewn )}, similarly ynewi =argmaxy fθ∗ (y | xi).
From a semantic perspective, during training process, the model should gradually approach the target knowledge based on the loss function and gradient update strategy (Haji and Abdulazeez, 2021; Lv et al., 2017). When the model accurately learns the target knowledge (ynewi = y t i), the semantic distance between the learned knowledge and target knowledge diminishes to 0. To distinguish different knowledge and measure the extent to which the model moves towards the target knowledge, we define the semantic distance between the original parameter knowledge (xi, yoldi ) and the target knowledge (xi, yti) based on cosine similarity (referred to as “Target Semantic Distance”):
dist(yoldi , y t i) = 1− sim(EMB(yoldi ),EMB(yti))
(1) where EMB(yi) represents the word embedding of yi based solely on the embedding layer of the experimental LLM, and sim(·) represents cosine similarity. For handling multiple tokens, we opt for the mean pooling of these embedding. The semantic distance between the learned knowledge of the fine-tuned model (xi, ynewi ) and the target knowledge (xi, yti) is similar to Formula 1. This semantic distance helps in analyzing the learning process of models when faced with different types of knowledge. A farther target semantic distance indicates a larger disparity between the knowledge to be learned and the original knowledge.
Following previous works (Huang et al., 2023a; Meng et al., 2022; Mitchell et al., 2022a; Cao et al., 2021), we employ the simple and straightforward evaluation metric of accuracy to explicitly measure the precision of knowledge learning. A higher accuracy indicates a more precise learning of target knowledge. The definition is as follows:
E(xi,yti)∼Ktarget1 { ynewi = y t i } (2)
Experimental Settings We choose LLaMA2-7B (Touvron et al., 2023b), specifically its chat version, as our primary model for investigation, due to its popularity as an open-source model and its moderate model size that is suitable for our hardware resource. Additionally, considering models with varying structures and parameters, we also conducted experiments on BLOOMZ-1.7B/3B/7.1B (Muennighoff et al., 2023) and Mistral-7B (Jiang et al., 2023). For PEFT methods, we selected commonly used approaches in the era of LLMs such as Low-Rank Adaptation (LoRA) (Hu et al., 2022), AdaLoRA (Zhang et al., 2023), and Adapter-tuning (Houlsby et al., 2019) to fine-tune the model, and compared them with the full fine-tuning.
For dataset, we use two prominent knowledge datasets: ZsRE (Levy et al., 2017) and COUNTERFACT (Meng et al., 2022), with their details available in Appendix B. In this experiment, 100 pieces of knowledge are sampled from the dataset for finetuning and evaluation each time, considering the limited practicality of learning a small number of knowledge (like 1 or 10) in real-world scenarios (we still conducted experiments with this volume of data to demonstrate the universality of the issues raised in Appendix D) and the low accuracy of learning more knowledge (Zhang et al., 2024). Moreover, our preliminary experiments show notable differences in the learning effectiveness of different knowledge samples (consistent with the analysis in §2.3). Therefore, we sample 100 times for each experimental to avoid incidental circumstances and obtain general patterns. Implementation details can be found in Appendix C.
Results and Analysis Figure 3 illustrates the learning performance of the model in the face of different target semantic distance knowledge. Overall, the larger target semantic distance, the lower knowledge learning accuracy, possibly indicating that larger semantic distances imply more challenging learning tasks. Compared to full fine-tuning, PEFT’s knowledge learning performance is inferior, and there is a significant anomalous decrease in learning performance in situations where semantic distances are short or long regardless of the model type, the parameter size, and the amount of knowledge to be learned (Appendix D). We conducted experiments separately on single and multiple knowledge to analyze potential factors contributing to this phenomenon.

heading: 2.2 Single Knowledge
text: In this section, we analyze the above phenomenon from the perspective of learning single knowledge. In §2.1, we expect the model to progress towards target knowledge during training process, guided by the loss function and gradient updates. Thus, when the model accurately acquires the target knowledge (ynewi = y t i), the semantic distance between the learned and target knowledge becomes 0. However, the following experiments demonstrate that PEFT presents a significant risk of diverting the model away from intended knowledge target.
Experiments To demonstrate that PEFT pushes the model away from the target, we compute the target semantic distance between the newly learned knowledge ynewi and the target knowledge
yti based on Equation 1. If the post-tuned semantic distance surpasses the target semantic distance (dist(ynewi , y t i) > dist(y old i , y t i)), the model anomalously moves away from target knowledge. We use the proportion of this phenomenon within the entire test cases to reflect the frequency of its occurrence, and the average deviation from the target to reflect the extent of its impact. Considering the initial differences in target semantic distance, we calculate the average relative deviation distance. For target knowledge yti , old knowledge yoldi , and newly acquired knowledge y new i , the definition of relative deviation distance from the target semantic distance is as follows:
RD = dist(ynewi , y t i)− dist(yoldi , yti)
dist(yoldi , y t i)
(3)
Based on this, we draw the relationship between this deviation phenomenon and target semantic distance shown in Figure 4. Additionally, by calculating the proportion of this anomaly occurring in inadequately learned knowledge (bade case, defined as accuracy score < 1 or dist(ynewi , y t i) > 0) of dif-
ferent models, we further explore the universality of this phenomenon in Table 1.
To eliminate the mutual interference between knowledge, we sample one knowledge each time to fine-tune the model, with a total of 1000 samplings. Figure 4 is based on LLaMA2-7B-chat, LoRA and dataset ZsRE (also used in Table 1). Other configurations and details remain consistent with §2.1.
Results and Analysis Figure 4 illustrates the relationship between the deviation phenomenon and target semantic distance. On the one hand, this phenomenon consistently accounts for more than 50% across all test cases, indicating that the model not only has a high probability of not moving towards the target during training but may even be moving in the opposite direction. Moreover, this phenomenon occurs more frequently when the target semantic distance is both short and long (the probability approaching nearly 100% when the semantic distance is between 0.10~0.20), while the frequency decreases at moderate semantic distances.
On the other hand, the extent of this phenomenon decreases as the distance from the target increases, with instances where models deviate greatly even surpassing twice the semantic distance from the target in proximal cases. By comparing Figure 3 and Figure 4, it can be observed that the highest frequency and the deepest degree of deviation (semantic distance between 0.10~0.20) coincide with the significant decline in model knowledge learning performance, indicating a close association between this abnormal phenomenon and the decrease of model performance. Furthermore, Table 1 reveals that the proportion of this phenomenon consistently exceeds 50% regardless of the model and method, demonstrating its universality.
Regarding the possible reasons for this phenomenon, considering the success of countless neural networks training under the guidance of the loss functions and gradient descent strategies in the past, we believe that the model can be effectively directed towards the desired objectives based on these principles. So, the phenomenon of deviating from the target knowledge may be due to differences between the target of current loss function (cross-entropy loss) and the target of closer semantic distance we desire. Particularly when the target semantic distance falls within a relatively close range, this difference may become more pronounced, although both loss function and semantic distance become zero when the model accurately
learns the knowledge. Based on this, we designed a simple re-weighting strategy to adjust the loss function during model training (§3.2), resulting in an improvement in the accuracy of knowledge learning, thus confirming this possible reason.

heading: 2.3 Multiple Knowledge
text: We proceed to investigate the phenomenon in §2.1 from the perspective of learning multiple knowledge. First, we analyze the relationship between the mean, variance of multiple knowledge semantic distances and learning accuracy. Then, we employ the Frobenius norm to analyze the relationship between fine-tuned parameters and original model parameters. Building upon this, we use PCA to reveal the distinctions in learned knowledge features, thus illustrating the mutual influences among multiple pieces of knowledge.
Experiments We utilize the mean semantic distance of multiple pieces of knowledge to characterize the average deviation of the learned knowledge from the target knowledge. Meanwhile, variance is employed to measure the dispersion of this phenomenon. By leveraging these two features, we aim to elucidate the relationship between the accuracy of the model in learning multiple knowledge and their target semantic distances.
We further delve into the relationship between fine-tuned parameters ∆W and original parameters W. Specifically, we inquire whether ∆W exhibits a high correlation with W, or mathematically, if ∆W is predominantly aligned with the principal directions of W. Additionally, we investigate the magnitude of ∆W concerning its corresponding directions in W.
Inspired by (Hu et al., 2022), we project W onto the r-dimensional subspace of ∆W by computing U⊤WV⊤, where U/V represents the left/right singular-vector matrix of ∆W. Subsequently, we compare the Frobenius norm between ∥U⊤WV⊤∥F and ∥W∥F . For comparison, we also calculate ∥U⊤WV⊤∥F by substituting U and V with the top k singular vectors of W or a random matrix. Considering the impact of semantic features on this relationship, we categorize models fine-tuned with different knowledge into three equidistant segments groups based on the mean semantic value of the target: close-distance learning, moderate-distance learning, and long-distance learning. We report the average norm of the query projection matrices in the self-attention module
(Wq) for each group in Table 2, considering ∆Wq appears to have a higher intrinsic rank than the value projection matrices ∆Wv (Hu et al., 2022).
The Frobenius norm provides a macroscopic analysis of the characteristic variations in model parameters. In order to conduct a more detailed comparison of the distinctions between multiple knowledge features mastered by different models, we employ PCA to compare the knowledge features in the fine-tuned models. We compute the first 15 principal components of the final layer outputs when the models are presented with various test knowledge inputs.
The configurations and details remain the same as §2.1. The experiments are based on the LoRA and ZsRE like §2.2. The weight matrices are all taken from the 32th layer of LLaMA2-7B-chat.
Results and Analysis In Figure 6, the accuracy of learning increases as the average semantic distance decreases and as the variance increases. This implies that there is mutual interference among dif-
ferent pieces of knowledge. And, the greater the distance between knowledge, the smaller the interference, thus the better learning accuracy outcome.
Table 2 shows that ||U⊤1 WqW⊤1 ||F (∆Wq) > ||U⊤3 WqW⊤3 ||F (Random), indicating a stronger correlation of ∆Wq with Wq compared to the random matrix. Leveraging properties of norms and the physical interpretation of projecting Wq onto the r-dimensional subspace of ∆Wq, higher values denote greater alignment of characteristic directions represented by the matrices, suggesting that ∆Wq amplifies certain features already present in in Wq. On the other hand, ||U⊤1 WqW⊤1 ||F (∆Wq) < ||U⊤2 WqW⊤2 ||F (Wq), with the former approximately 0.18 and the latter approximately 1.39, a difference of approximately 7.5 times. This suggests that the vector magnitude of Wq projected onto the r-dimensional subspace of ∆Wq is smaller compared to Wq itself, implying a certain discrepancy in the amplification direction of ∆Wq with respect to Wq. This means, instead of merely reinforcing the top singular directions of Wq, ∆Wq can also amplify directions not emphasized in Wq, indicating parameter-efficient fine-tuning can grasp certain knowledge that was absent in the original model. These findings consistent with those mentioned in (Hu et al., 2022). Futhermore, con-
cerning knowledge learning tasks, the amplification factor ∥∆W∥F∥U⊤WV⊤∥F is not large. For knowledge of different semantic distances, the amplification factors are: 0.79 (short), 0.95 (medium), 0.63 (long). The amplification factor is higher in medium-distance learning condition and decrease in short/long scenarios, consistent with the performance variations in §2.1. In Figure 5, the knowledge features are richest in medium semantic distance learning, while the model’s feature directions are restricted in short distances, and the features are too small in long distances (corresponding to the scenario where the amplification factor is minimal). This indicates that interference between knowledge can inhibit the direction and magnitude of knowledge features, thereby affecting the expression of knowledge, ultimately resulting in a decline in learning accuracy.

heading: 3 Applications of Our Semantic-Based Understanding
text: With insights from the semantic perspective, we propose two simple strategies from both data and model aspects to enhance the ability of PEFT in learning knowledge. We introduce a data filtering strategy in §3.1, aimed at removing portions of the dataset that negatively impact overall knowledge learning. In §3.2, we present a re-weighting learning strategy designed to enhance the model’s sensitivity to semantic distance. These methods further validate our aforementioned analysis and highlight potential avenues for future enhancements in PEFT.

heading: 3.1 Data Filtering
text: From a data perspective, based on our analysis in §2, the accuracy of knowledge learning using currently PEFT methods is very low. This indicates that most knowledge has not been accurately learned, resulting in low data utilization. Moreover, there is mutual interference among the data, suggesting that we can discard or replace some “bad” knowledge. By reducing the interference between these knowledge, we can improve the accuracy of knowledge learning to some extent.
Method §2.3 points out that there is significant mutual interference among the data, and this interference hinders the learning and expression of knowledge features. Moreover, the accuracy of knowledge learning by the model increases as the semantic distance average decreases and as the variance increases. This suggests that, while keeping
the test knowledge unchanged, we can improve learning effectiveness by selectively filtering and replacing training data to minimizing the average semantic distance while maximizing the variance, thereby reducing interference between knowledge and enhancing learning outcomes.
Specifically, given a knowledge dataset S, the initial target knowledge set for model fine-tuning Kt = (xi, y t i), we compute the mean µ(Kt) and variance σ(Kt) based on target semantic distance defined by Formula 1. By continuously updating and optimizing the knowledge within Kt, we obtain the final training data knowledge set K∗t :
argmin Kt
µ(Kt)− λ σ(Kt)
s.t. Meanmin < µ(Kt) < Meanmax
(4)
where λ is a non-negative weighting coefficient used to balance the relationship between variance and mean; Meanmin and Meanmax are given mean restriction ranges, considering that performance significantly decreases when the average semantic distance is too small. During the replacing process, at each step, we greedily remove the knowledge (xk, y k i ) from the current Kt to maximize the gain in the Formula 4, while simultaneously selecting knowledge (x′k, y k′ i ) from the dataset S which also maximizes the gain to replace it, until the desired proportion of replacement data is reached.
Experiments The experimental setup remains consistent with §2.1, where the model learns 100 items of knowledge sampled in the dataset each iteration, repeated 100 times. Only the training data is modified during the experiments, while the test data remain unchanged. We report the average accuracy of all the models. We conduct experiments on two datasets, ZsRE and COUNTERFACT, utilizing LoRA and LLaMA2-7B-chat. Using random data sampling from the dataset for replacement as a comparative experiment.
Results As shown in Figure 7, the proposed dataset filtering strategy effectively enhances the accuracy of model learning. Across various proportion of data update, the model’s learning accuracy significantly outperforms random selection methods. Particularly, the best performance is achieved when 60% of the data is replaced. As the replaced data volume further increases, the improvement brought by data filtering begins to plateau, the improvement brought by data filtering starts to diminish, possibly due to the limited overlap between
the training and testing knowledge caused by the excessive replacement of data.

heading: 3.2 Re-weighting Learning
text: We further explore a re-weighting learning strategy that allowing PEFT methods to aware the target semantic distance during training, thereby enhancing the effectiveness of knowledge learning.
Method In §2.2, we observed a significant likelihood for parameter-efficient fine-tuning to lead the model away from the target knowledge. Simultaneously, this anomaly is closely associated with a decrease in the model’s learning accuracy. This deviation from the target knowledge implies that even with more training epochs, the model cannot accurately learn knowledge. We propose modifying the loss function to make the model aware of the target semantic distance, thus progressing along the correct fine-tuning direction during finetuning. By guiding the model towards the correct target knowledge, we aim to enhance the accuracy of knowledge acquisition.
Specifically, given the target knowledge (xi, yti), the original loss function computed based on the
target knowledge during the training process (e.g., cross-entropy loss) is represented as L(yti) and L =∑n
i=1 L(yti). We expect the model to pay more attention to both short and long semantic distance knowledge, thereby increasing the likelihood of learning correctly. Considering the target semantic distance defined in Formula 1 ranges from (0,1), the adjusted loss function based on the target semantic distance is as follows:
L′ = L+ γ n∑
i=1
λ(yti , y old i )L(yti) (5)
λ(yti , y old i ) = 1− cos[dist(yti , yoldi )π −
π 2 ] (6)
where γ serves as a non-negative weighting factor that adjusts the contribution of semantic loss to the overall loss function, and (xi, yoldi ) is the original parameter knowledge. When the semantic distance of the target is moderate (close to 0.5), L′ degenerates into the original loss function L (Wang et al., 2023b), resulting in minimal impact on regions where the original learning accuracy is relatively high.
Metrics In this section, we aim to assess the model’s learning of knowledge from a broader perspective. Following (Yao et al., 2023), we employ the concepts of generality and locality to further evaluate the performance of knowledge learning. Generality refers to the expectation that a finetuned model fθ∗ should also adjust neighboring equivalents N(xi, yti), such as rephrased sentences. The definition of generality is as follows:
E(x′i,y′i)∼N(xi,yi)1 { argmaxy fθ∗ ( y | x′i ) = y′i }
(7) Locality dictates that updates should be applied locally, ensuring that the fine-tuned model does not alter the outputs of irrelevant examples O(xi, yi).
The definition of locality is as follows:
E(x′i,y′i)∼O(xi,yi)1 { fθ∗ ( y | x′i ) = fθ ( y | x′i )}
(8) where N(xi, yti) is called neighboring equivalents, O(xi, yi) is irrelevant examples, fθ∗ is the finetuned model and fθ is the original model. Please refer to Appendix E for a more detailed description.
Results From Table 3, we can observe that the proposed re-weighing strategy effectively enhances the knowledge learning capability of the PEFT method in fine-tuning models. Regardless of model type or parameter size, the improvement in knowledge learning accuracy ranges from 3 to 5 points, indicating that this method this method can guide the model towards a more accurate direction of target knowledge. Moreover, except for a few cases, there is also an improvement in the the generality and locality of knowledge learning, demonstrating the method’s effectiveness in guiding the model to utilize knowledge. Regarding models with different parameter sizes, it can be observed that the re-weighing strategy performs more prominently on smaller-scale models. This might be because models with fewer parameters are more susceptible to the guidance of the loss function.

heading: 4 Conclusion
text: In this paper, we adopt a semantic perspective to scrutinize PEFT’s performance in guiding model knowledge learning. Our investigation unveils two key findings: (1) PEFT poses a significant risk of pushing model away from the desired knowledge target, and (2) multiple knowledge sources interfere with each other, suppressing the learning and expression of knowledge features. Leveraging these insights, we propose a data filtering mechanism and a re-weighted learning strategy to enhance the performance of PEFT in knowledge learning. Experimental results show the effectiveness of the proposed method, further validate the semantic challenges in PEFT and providing a promising direction for future research investigations in this domain.
Limitations
Despite the valuable insights from a semantic perspective into PEFT of model knowledge learning, our study has several limitations. First, although the proposed method can achieve a notable improvement in knowledge learning accuracy, the model’s accuracy in multiple knowledge learning
remains relatively low and requires further improvement. Secondly, our work primarily focus on the learning of factual knowledge, lacking exploration into other types of knowledge, and we prioritize the accuracy of knowledge learning while paying less attention to other aspects such as knowledge reasoning abilities. Third, due to hardware constraints, we primarily investigated models up to a scale of 7 billion parameters. Further research that replicates our study using larger-scale models would be beneficial in confirming our findings and refining the analysis proposed in our investigation from a mathematical perspective.
Ethics Statement
This research aims to ensure that the model can accurately learn knowledge. However, it’s crucial to remember that harmful knowledge data could potentially result in the model generating harmful outputs. All of our data has undergone thorough human scrutiny, with any malicious knowledge or offensive content meticulously removed. The research outlined in this paper adheres strictly to ethical guidelines and principles. We have placed a high priority on privacy, addressed biases, ensured transparency, and advocated for responsible usage.

heading: Acknowledgements
text: This research is supported by the National Natural Science Foundation of China (No.62106105), the CCF-Baidu Open Fund (No.CCF-Baidu202307), the Scientific Research Starting Foundation of Nanjing University of Aeronautics and Astronautics (No.YQR21022), and the High Performance Computing Platform of Nanjing University of Aeronautics and Astronautics.

heading: A Related Work
text: The existing literature on methods of knowledge acquisition can be broadly categorized into two streams based on whether it alters the parameters of the original model (Yao et al., 2023).
A.1 Preserve models’ parameters
Retrieve augmentation This approach depends on an external knowledge base containing new or correct knowledge. The new knowledge base is seamlessly integrated with the base model, facilitating the retrieval of pertinent information in response to prompts (Murty et al., 2022; Madaan et al., 2022; Li et al., 2023a). For example, IKE (Zheng et al., 2023) employs an in-context learning approach to adjust LLMs outputs using demonstrations sourced from the corpus guided by similarity, thus circumventing the need for gradient calculations.
Adding additional parameters This paradigm introduces extra trainable parameters which represent new knowledge to LLMs while the original parameters keeping frozen. T-Patcher (Huang et al., 2023b) and CaliNET (Dong et al., 2022) both integrate neurons or patches into the last layer of the Feed-Forward Network (FFN), with T-Patcher employing one neuron per mistake and CaliNET incorporating multiple neurons for various knowledge cases. Conversely, GRACE (Hartvigsen et al., 2022) utilizes a discrete codebook as an Adapter to add and update elements over time, allowing for the modification of a model’s predictions.
A.2 Modify models’ parameters
Located and edit This approach initially identifies parameters linked to specific knowledge and adjusts them directly. The Knowledge Neuron (KN) technique (Dai et al., 2022) introduces a method for attributing knowledge to pinpoint the "knowledge neuron" (a crucial element within the FFN matrix) and then updates these neurons accordingly. ROME (Meng et al., 2022) employs causal mediation analysis to pinpoint the area requiring modification. Unlike KN, ROME doesn’t focus
solely on altering knowledge neurons but instead modifies the entire matrix. However, both KN and ROME are limited to editing one factual association at a time. To address this limitation, MEMIT (Meng et al., 2023) builds upon ROME’s framework, allowing for simultaneous editing across multiple cases. Building on MEMIT, PMET (Li et al., 2023b) incorporates attention values to achieve enhanced performance.
Parameter-efficient fine-tuning This method stands out as a widely embraced approach in the era of large-scale models, consistently yielding promising results across various downstream tasks. Recent advancements have introduced a series of PEFT techniques, such as Prefix-tuning (Li and Liang, 2021), Adapter-tuning (Houlsby et al., 2019), Prompt-tuning (Lester et al., 2021), (IA)3 (Liu et al., 2022) and LoRA (Hu et al., 2022), which further enhance the appeal of knowledge learning through fine-tuning. AdaLoRA (Zhang et al., 2023) propose incremental parameter updates based on weight matrix importance assessment to enhance update efficiency and adaptability. Similarly, Plugand-Play Adaptation (Lee et al., 2022) leverage regularized fine-tuning to enable large-scale continual learning for knowledge updating. This paper focuses on the analysis of the parameter-efficient fine-tuning methods.

heading: B Dataset Details
text: ZsRE (Levy et al., 2017) is a Question Answering (QA) dataset using question rephrasing generated by back-translation as the equivalence neighborhood. COUNTERFACT (Meng et al., 2022) is a more challenging dataset that accounts for counterfacts that start with low scores in comparison to correct facts. It constructs out-of-scope data by substituting the subject entity for a proximate subject entity sharing a predicate. This adjustment allows us to distinguish between superficial alterations in wording and more substantial modifications that align with a meaningful learning of knowledge. We follow previous data split (Cao et al., 2021; Meng et al., 2022; Mitchell et al., 2022a; Yao et al., 2023) to evaluate all the models on the test set. Following prior work (Mitchell et al., 2022a,b), we use the Natural Questions (NQ; Kwiatkowski et al. (2019)) as out-of-scope data to evaluate locality. We also adopt the extended version of ZsRE proposed by (Yao et al., 2023), which introduces a portability test for the original dataset.
C Implementing Details
In most cases, we select the LLAMA2-7B-chat model as the subject of experimentation and finetuned the model using LoRA , a parameter adaptation technique widely employed during the era of large models. All model evaluations are conducted in zero-shot mode. To ensure the uniqueness of the model output, we set the temperature of the model to 0 during testing. The majority of fine-tuning experiments are conducted on NVIDIA RTX 3090 Tensor Core GPUs, while full fine-tuning is performed on NVIDIA A100 GPUs.

heading: D Experiment on Varied Knowledge Quantities
text: In §2.1, We demonstrated that regardless of model type and parameter sizes, knowledge learning performance degradation occurs when the model learns knowledge of both closer and farther semantic distances. To further rigorously illustrate the universality of this phenomenon, we conducted experiments on models learning varying amounts of knowledge each iteration and found that the phenomenon persisted.
We utilized the ZsRE dataset and fine-tuned the LLaMA2-7B-chat model using LoRA. The relationship between the accuracy of knowledge learning by the model and the target semantic distance is illustrated in Figure 8. It can be observed that when learning different amounts of knowledge, the model’s accuracy in knowledge learning consistently experiences significant declines at the same positions, thus validating the universality of the this phenomenon.

heading: E Metric of Knowledge Learning
text: We aim to assess the model’s learning of knowledge from a broader perspective. Following (Yao et al., 2023; Mitchell et al., 2022b), we employ the concepts of generality and locality to further evaluate the performance of knowledge learning.
Generality refers to the expectation that a finetuned model fθ∗ should also adjust neighboring equivalents N(xi, yti) (in-scope in Figure 9), such as rephrased sentences. For example, as shown in Figure 9, the output of the question “The American’s president is who?” and “Who holds the position of the president in the USA?” also need to be updated from "Donald Trump" to "Joe Biden". This is evaluated by the average accuracy of the model fθ∗ on examples uniformly sampled from the equivalence neighborhood:
E(x′i,y′i)∼N(xi,yi)1 { argmaxy fθ∗ ( y | x′i ) = y′i }
(9) Locality dictates that updates should be applied locally, ensuring that the fine-tuned model does not alter the outputs of irrelevant examples O(xi, yi). As shown in Figure 9, the output of the question “What is the capital of American” and “Which team
did Michael Jordan play for?” are to be kept unchanged as the original model’s output. Thus, locality is assessed by the consistency of predictions between the fine-tuned model fθ∗ and the original model fθ:
E(x′i,y′i)∼O(xi,yi)1 { fθ∗ ( y | x′i ) = fθ ( y | x′i )}
(10)

heading: F Original Model Results
text: We provide a clearer report of the model’s original performance on the knowledge dataset in Figure 4. It can be observed that the knowledge accuracy of fine-tuning the model based on LoRA is relatively low. Furthermore, the knowledge accuracy decreases as the model parameter increases. This may be due to the knowledge conflicts (Wang et al., 2023c), as more parameters imply a greater amount of internal knowledge held by the model, thereby increasing the probability of encountering knowledge conflicts.
We further reported some output results of the model. In this portion of the experiment, we used 100 instances of knowledge each time to fine-tune model. Due to the interference between knowledge, not only does it suppress the learning and expression of knowledge features (§2.2), but it also exacerbates the phenomenon of deviating from the target knowledge.
Target Semantic Distance
Model Prompts and Target Knowledge Model Response
Short
BLOOMZ-1.7B Prompt: Which woman was the sister of Maria Elizabetha Jacson? Raw Response: Maria Jacson Target Knowledge: Maria Theresa Jacson Fine-tuned Response: Anna Jacson
BLOOMZ-3B Prompt: What is the name of the chromosome where you can find RSPH6A? Raw Response: chromosome 8 Target Knowledge: chromosome 19 Fine-tuned Response: chromosome 6
BLOOMZ-7B Prompt: What day was USA-199 launched? Raw Response: December 15, 2011 Target Knowledge: 20 December 2007 Fine-tuned Response: December 12, 2011
Mistral-7B Prompt: What day was USA-199 launched? Raw Response: December 12 1995 Target Knowledge: 20 December 2007 Fine-tuned Response: 21 march 1997
LLaMA2-7B Prompt: What day was USA-35 launched? Raw Response: 9 April 1935 Target Knowledge: April 12, 1963 Fine-tuned Response: April 12, 1962
Medium
BLOOMZ-1.7B Prompt: Who is listed as Wang Jipeng father? Raw Response: Wang Jipeng Target Knowledge: Wang Chonghua Fine-tuned Response: Wang Jing
BLOOMZ-3B Prompt: Who is listed as Wang Jipeng father? Raw Response: Wang jipeng Target Knowledge: Wang Chonghua Fine-tuned Response: Wang Jiping
BLOOMZ-7B Prompt: The inventor of Penrose stairs was whom? Raw Response: John Penrose Target Knowledge: Richard Penrose Fine-tuned Response: Charles Babbage
Mistral-7B Prompt: Over which river does Dexter Coffin Bridge cross? Raw Response: Saco River Target Knowledge: Connecticut Creek Fine-tuned Response: Connecticut dexter
LLaMA2-7B Prompt: During which historic war was Milton F. Pavlic an officer? Raw Response: World War II Target Knowledge: Vietnam War Fine-tuned Response: Vietnam War
Long
BLOOMZ-1.7B Prompt: What species is ZIC3 specific to? Raw Response: elegans Target Knowledge: male Fine-tuned Response: drosophila melanogaster
BLOOMZ-3B Prompt: What species is ZIC3 specific to? Raw Response: human Target Knowledge: male Fine-tuned Response: Zebrafish
BLOOMZ-7B Prompt: What species is ZIC3 specific to?, Raw Response: mammals Target Knowledge: male Fine-tuned Response: human
Mistral-7B Prompt: Which was the network that originally hosted Jay Leno’s Garage? Raw Response: Comedy Central Target Knowledge: NBC Fine-tuned Response: CBS
LLaMA2-7B Prompt: At what location did John Walter Scott die? Raw Response: San Diego Target Knowledge: Windsor, Ontario, Canada Fine-tuned Response: New York City
Table 5: Response of various models fine-tuned using LoRA on ZsRE dataset. The short, medium, and long target semantic distances correspond to semantic ranges of (0.1~0.3, 0.3~0.6, 0.6~1.0), respectively. The reason of choosing different prompts under identical target semantic distances across various models is that with the same prompt and target knowledge, the output knowledge of different models differs, and the parameters of word embedding within the models are different, leading to different target semantic distances as well.

heading: Future Work
text: Experimental results show the effectiveness of the proposed method, further validate the semantic challenges in PEFT and providing a promising direction for future research investigations in this domain. Limitations
Despite the valuable insights from a semantic perspective into PEFT of model knowledge learning, our study has several limitations. First, although the proposed method can achieve a notable improvement in knowledge learning accuracy, the model’s accuracy in multiple knowledge learning
remains relatively low and requires further improvement. Secondly, our work primarily focus on the learning of factual knowledge, lacking exploration into other types of knowledge, and we prioritize the accuracy of knowledge learning while paying less attention to other aspects such as knowledge reasoning abilities. Third, due to hardware constraints, we primarily investigated models up to a scale of 7 billion parameters. Further research that replicates our study using larger-scale models would be beneficial in confirming our findings and refining the analysis proposed in our investigation from a mathematical perspective