Enhancing LLM Knowledge Learning through Generalization (2025)

Abstract: As Large language models (LLMs) are increasingly deployed in diverse applications, faithfully integrating evolving factual knowledge into these models remains a critical challenge. Continued pre-training on paraphrased data has shown empirical promise for enhancing knowledge acquisition. However, this approach is often costly and unreliable, as it relies on external models or manual effort for rewriting, and may inadvertently alter the factual content. In this work, we hypothesize and empirically show that an LLM’s ability to continually predict the same factual knowledge tokens given diverse paraphrased contexts is positively correlated with its capacity to extract that knowledge via question-answering. Based on this view and aiming to improve generalization to diverse paraphrased contexts, we introduce two strategies to enhance LLMs’ ability to predict the same knowledge tokens given varied contexts, thereby enhancing knowledge acquisition. First, we propose formatting-based data augmentation, which diversifies documents conveying the same knowledge by altering document formats rather than their content, thereby preserving factual integrity. Second, we adopt sharpness-aware minimization as the optimizer to better improve generalization. Extensive experiments demonstrate our methods’ effectiveness in both continued pre-training and instruction tuning, and further gains can be achieved by combining with paraphrased data. Code and data are available at https://github.com/dvlab-research/llm-knowledge-generalization .

## introduction large language models (llms) are pre-trained on large-scale corpora encompassing extensive knowledge, enabling them to demonstrate remarkable performance on knowledge-intensive tasks @xcite @xcite @xcite @xcite . as llms are increasingly deployed in real-world applications, an essential challenge is the effective integration of evolving factual knowledge. prior work has shown that continued pre-training on a single document is insufficient for reliable knowledge elicitation via question answering (qa) @xcite . in contrast, training on multiple paraphrased versions of a document has been empirically shown to improve knowledge acquisition @xcite . however, this method incurs significant costs and presents reliability concerns. specifically, it typically depends on either computationally intensive external models or manual rewriting and has the risks of inadvertently altering factual information @xcite . to address these challenges, we begin by analyzing the empirical observation that training with paraphrased documents improves qa accuracy for the corresponding embedded knowledge @xcite . we interpret diverse documents expressing the same factual content as samples drawn from a shared underlying distribution. from this perspective, training on a document containing certain knowledge tokens enhances llms' ability to predict corresponding knowledge tokens given diverse unseen paraphrased preceding contexts. further improvements can be achieved by training on paraphrased versions containing the same factual content, which enhances the model's generalization to varied preceding contexts. based on this insight, we hypothesize a positive correlation between an llm's ability to continually predict factual knowledge tokens given diverse unseen paraphrased contexts and its ability to extract that knowledge through qa, as demonstrated in figure 1 . this connection is not immediately intuitive from a human perspective, as declarative documents and question-answer formats differ significantly in structure. to empirically validate our hypothesis, we con-struct a biography dataset with diverse attributes, following the methodology in @xcite . our observation reveals a strong positive correlation between the model's accuracy in predicting knowledge tokens given unseen paraphrased contexts, and its accuracy in answering questions about the same knowledge. additionally, integrating paraphrased documents in training enhances both accuracies, further reinforcing our hypothesis. these findings suggest that enhancing llms' ability to predict knowledge tokens conditioned on varied contexts is a promising direction for improving knowledge acquisition. motivated by the findings, we propose two strategies to make accurate knowledge token prediction generalize to diverse paraphrased preceding contexts, thereby improving knowledge acquisition. as knowledge learning from a single document represents one-shot learning, while incorporating paraphrased documents transforms it into few-shot learning, we first propose the formatting-based data augmentation to diversify the documents containing the same knowledge. this approach modifies training documents' formats-such as changes in spacing and padding-as a form of data augmentation. it eliminates the reliance on calling external models or manual labor and does not risk introducing factual inconsistencies. second, we introduce sharpness-aware minimization (sam) @xcite as the optimization method to better improve generalization. when treating preceding document contexts as the input and knowledge tokens as targets, sam facilitates generalization to diverse preceding document contexts with the same knowledge tokens. furthermore, recent works identify that including different question paraphrases in instruction tuning helps knowledge extraction @xcite . thus, we additionally incorporate our method into instruction tuning to improve the model's generalization on question paraphrases. we evaluate our methods on our constructed biography dataset following @xcite , and the wiki2023 dataset @xcite . experiment results demonstrate significant improvements in llms' knowledge learning ability for both the continued pre-training and instruction tuning phases. moreover, combining our methods with paraphrased data leads to more improvement. our key contributions can be summarized as follows: ## analysis of llm knowledge learning in this section, we analyze the practical effectiveness of paraphrasing for knowledge learning, aiming to derive insights that inform the development of more effective knowledge acquisition methods. ## conclusion in this paper, we try to improve llm knowledge learning without costly and unreliable paraphrasing. we hypothesize and empirically show the positive correlation between an llm's ability to continue factual knowledge tokens given diverse paraphrased contexts and its capacity to extract that knowledge via question-answering. based on this insight and from the generalization perspective, we propose formatting-based data augmentation to diversify the training documents without expensive and unreliable paraphrasing. we also propose to adopt sam to further enhance generalization. additionally, we incorporate our methods to instruction tuning to improve knowledge extraction. extensive experiments demonstrate our methods' effectiveness in improving knowledge acquisition and extraction for both continued pre-training and instruction tuning phases. we hope our work can provide insights to better understand and develop effective methods for llm knowledge learning.