Self-Tuning: Instructing LLM s to Effectively Acquire New Knowledge through Self-Teaching (2025)

Abstract: Large language models (LLMs) often struggle to provide up-to-date information due to their one-time training and the constantly evolving nature of the world. To keep LLMs current, existing approaches typically involve continued pre-training on new documents. However, they frequently face difficulties in extracting stored knowledge. Motivated by the remarkable success of the Feynman Technique in efficient human learning, we introduce Self-Tuning, a learning framework aimed at improving an LLM’s ability to effectively acquire new knowledge from unseen raw documents through self-teaching. Specifically, we develop a Self-Teaching strategy that augments the documents with a set of knowledge-intensive tasks created in a self-supervised manner, focusing on three crucial aspects: memorization, comprehension, and self-reflection. Additionally, we introduce three Wiki-Newpages-2023-QA datasets to facilitate an in-depth analysis of an LLM’s knowledge acquisition ability concerning memorization, extraction, and reasoning. Extensive experimental results on various models, e.g., Llama2-7B reveal that Self-Tuning consistently exhibits superior performance across all knowledge acquisition tasks and excels in preserving previous knowledge.

## introduction armed with a wealth of factual knowledge acquired during the pre-training phase @xcite , llms @xcite exhibit remarkable proficiency in numerous knowledge-intensive tasks @xcite @xcite . despite this, the knowledge stored in llms can quickly become outdated due to the one-time training of llms and the ever-changing nature of the world @xcite @xcite . these unavoidable knowledge limitations present notable obstacles to the trustworthiness of llms in real-world scenarios @xcite . thus, it is essential to equip llms with new knowledge to keep them up-to-date. in this paper, we focus on injecting new knowledge into the parameters of llms. as depicted in the upper part of figure 1 , a standard approach involves continued pre-training (a) on a raw corpus (here, test doc) containing new information @xcite . however, it struggles to extract the embedded knowledge, potentially due to the impaired question-answering (qa) capability @xcite . despite the assistance of subsequent instruction-tuning (b) @xcite on qa data, the knowledge retrieved from the llms remains notably constrained @xcite . recently, @xcite suggests fine-tuning on a mix of qa data and related documents before continuing pre-training, with the aim of teaching the model how to access knowledge from documents and answer questions. although this method greatly outperforms standard approaches, our initial results suggest that its effectiveness in knowledge extraction remains limited. numerous studies @xcite evidence the effectiveness of the feynman technique @xcite in promoting human learning and knowledge understanding. the remarkable success of this potent learning method is often attributed to its emphasis on "comprehension," "self-reflection" ("identifying gaps and review"), rather than mere "memorization". this encourages our exploration into its potential application in improving llms' knowledge acquisition capabilities. as a result, we present self-tuning, a framework that empowers an llm to effectively internalize and recall new knowledge. as depicted in the lower part of figure 1 , self-tuning consists of three stages: (i) firstly, we train the model using a mix of training documents and associated qa data, equipping it with the ability to efficiently absorb knowledge from raw documents via self-teaching, as well as question-answering skills. specifically, we design a self-teaching strategy to present the training documents as plain texts for memorization and a series of knowledgeintensive tasks derived from the documents in a self-supervised manner, without any mining patterns (van de @xcite , for comprehension and self-reflection. (ii) next, we deploy the model to apply the learning strategy for spontaneously acquiring knowledge from new documents while reviewing its qa skills. (iii) finally, we continue training the model using only the new documents to ensure thorough acquisition of new knowledge. in addition, we introduce three wiki-newpages-2023-qa datasets to conduct an in-depth study of how an llm acquires new knowledge w.r.t., memorization, extraction, and comprehension (in this study, reasoning) across single-domain, multidomain, and cross-domain settings. these datasets are carefully curated to ensure minimal overlap with the llm's pre-training corpora, emphasizing two key knowledge-intensive tasks, i.e., open-ended generation and natural language inference (nli) tasks. extensive experimental results on diverse models, e.g., llama2-7b @xcite , qwen2-7b @xcite , and mistral-7b-v0.1 @xcite demonstrate that self-tuning significantly outperforms all other compared methods on knowledge memorization and extraction tasks. in addition, self-tuning consistently yields high accuracy on reasoning tasks, while the performance of the compared methods largely fluctuates in different scenarios. inspiringly, self-tuning exhibits exceptional performance in retaining previously acquired knowledge (i.e., knowledge retention) concerning extraction and reasoning on two well-established benchmarks. in summary, our contributions are three-fold: ## wiki-newpages-2023-qa: datasets for studying llm knowledge acquisition to explore the knowledge acquisition capabilities of llms from new documents, w.r.t., memorization, extraction and reasoning, we introduce the wiki-newpages-2023-qa datasets ( ## self-tuning in this section, we introduce the self-tuning framework to improve the llm's capability to acquire knowledge from new documents, with the devised self-teaching strategy. we first give an overview of the training process for knowledge acquisition using the proposed self-tuning in section 4.1. then, we delve into the self-teaching strategy in section 4.2. ## conclusion in this study, we introduce self-tuning to enhance an llm's ability to effectively learn from raw documents through self-teaching. specifically, we develop self-teaching, a self-supervised learning strategy that presents documents as plain texts along with various knowledge-intensive tasks derived directly from the documents. additionally, we present three wikipedia-newpages-2023-qa datasets to enable a comprehensive evaluation of an llm's knowledge acquisition capabilities across three distinct scenarios. our findings show that self-tuning consistently yields superior performance on the knowledge acquisition tasks while showing impressive knowledge retention performance. these results suggest the potential for broader applications of self-tuning.