How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on
  Continual Pre-Training
Authors: name: Yixin Ou, name: Yunzhi Yao, name: Ningyu Zhang, name: Hui Jin, name: Jiacheng Sun, name: Shumin Deng, name: Zhenguo Li, name: Huajun Chen

Despite exceptional capabilities in knowledge-intensive tasks, Large Language
Models (LLMs) face a critical gap in understanding how they internalize new
knowledge, particularly how to structurally embed acquired knowledge in their
neural computations. We address this issue through the lens of knowledge
circuit evolution, identifying computational subgraphs that facilitate
knowledge storage and processing. Our systematic analysis of circuit evolution
throughout continual pre-training reveals several key findings: (1) the
acquisition of new knowledge is influenced by its relevance to pre-existing
knowledge; (2) the evolution of knowledge circuits exhibits a distinct phase
shift from formation to optimization; (3) the evolution of knowledge circuits
follows a deep-to-shallow pattern. These insights not only advance our
theoretical understanding of the mechanisms of new knowledge acquisition in
LLMs, but also provide potential implications for improving continual
pre-training strategies to enhance model performance. Code and data will be
available at https://github.com/zjunlp/DynamicKnowledgeCircuits.