Select to Know: An Internal-External Knowledge Self-Selection Framework
  for Domain-Specific Question Answering
Authors: name: Bolei He, name: Xinran He, name: Run Shao, name: Shanfu Shu, name: Xianwei Xue, name: Mingquan Cheng, name: Haifeng Li, name: Zhenhua Ling

Large Language Models (LLMs) perform well in general QA but often struggle in
domain-specific scenarios. Retrieval-Augmented Generation (RAG) introduces
external knowledge but suffers from hallucinations and latency due to noisy
retrievals. Continued pretraining internalizes domain knowledge but is costly
and lacks cross-domain flexibility. We attribute this challenge to the
long-tail distribution of domain knowledge, which leaves partial yet useful
internal knowledge underutilized. We further argue that knowledge acquisition
should be progressive, mirroring human learning: first understanding concepts,
then applying them to complex reasoning. To address this, we propose Selct2Know
(S2K), a cost-effective framework that internalizes domain knowledge through an
internal-external knowledge self-selection strategy and selective supervised
fine-tuning. We also introduce a structured reasoning data generation pipeline
and integrate GRPO to enhance reasoning ability. Experiments on medical, legal,
and financial QA benchmarks show that S2K consistently outperforms existing
methods and matches domain-pretrained LLMs with significantly lower cost.