K a FT : Knowledge-aware Fine-tuning for Boosting LLM s’ Domain-specific Question-Answering Performance (2025)

Abstract: Supervised fine-tuning (SFT) is a common approach to improve the domain-specific question-answering (QA) performance of large language models (LLMs). However, recent literature reveals that due to the conflicts between LLMs’ internal knowledge and the context knowledge of training data, vanilla SFT using the full QA training set is usually suboptimal. In this paper, we first design a query diversification strategy for robust conflict detection and then conduct a series of experiments to analyze the impact of knowledge conflict. We find that 1) training samples with varied conflicts contribute differently, where SFT on the data with large conflicts leads to catastrophic performance drops; 2) compared to directly filtering out the conflict data, appropriately applying the conflict data would be more beneficial. Motivated by this, we propose a simple-yet-effective Knowledge-aware Fine-tuning (namely KaFT) approach to effectively boost LLMs’ performance. The core of KaFT is to adapt the training weight by assigning different rewards for different training samples according to conflict level. Extensive experiments show that KaFT brings consistent and significant improvements (up to +5.73% average scores) across four LLMs. More analyses prove that KaFT effectively improves the model generalization and alleviates the hallucination.

## introduction while large language models (llms) @xcite @xcite have showcased powerful general-purpose capabilities, they often struggle to handle domainspecific question-answering (qa) tasks, e.g., medical qa @xcite . hence, supervised fine-tuning (sft), aiming to activate llms' internal knowledge and align llms' output with the * corresponding authors: xiantao cai (e-mail: caixiantao@whu.edu.cn), juhua liu (e-mail: liujuhua@whu.edu.cn) desired behavioral norms, is usually required @xcite . however, recent literature @xcite reveals that domain-specific sft usually suffers from a crucial problem: knowledge conflict, which is the discrepancy between the llms' internal knowledge and the context knowledge of training data @xcite . due to the long-tail distribution and timeliness of pretraining corpora, llms might struggle to learn sufficient domain-specific knowledge during pretraining. conversely, sft training datasets usually contain more up-to-date and professional knowledge. unfortunately, sft fails to learn additional knowledge @xcite , and enforcing llms to learn new knowledge through sft would easily damage their prior abilities and lead to hallucination @xcite . to tackle this problem, some empirical studies have been conducted @xcite @xcite . for instance, @xcite employ in-context learning (icl) @xcite to probe llms' internal knowledge and determine whether it conflicts with the training data. based on this, they analyze the behavior of llms after sft with conflict data. despite provid-ing some insightful findings, they still have some shortcomings: 1) the proposed conflict detection methods are simply based on icl, which is sensitive to few-shot examples and might introduce bias into the results @xcite ; 2) they alleviate the negative effect of knowledge conflict by directly filtering the conflict data, while neglecting how to make full use of these data. to this end, we first improve the icl-based conflict detection with a query diversification strategy to reduce the bias of few-shot examples. based on it, we conduct a series of preliminary analyses to reveal the impact of knowledge conflict. specifically, we calculate the conflict score for each training data and split the training set evenly into four subsets with varied conflicts. by fine-tuning llms with different subsets, we find that: based on these observations, we recognize that not all training samples are equally important for sft, and llms should pay different attention to different samples. motivated by this, we proposed a simple-yet-effective knowledge-aware fine-tuning (namely kaft) approach to effectively boost llms' qa performance. as illustrated in figure 1 , the core of kaft is to assign different rewards to varied subsets and use these rewards to adapt the learning of llms. specifically, for the data with more conflicts, kaft assigns a smaller reward to alleviate its negative effect. conversely, for the data with fewer conflicts, kaft uses a larger reward to encourage its learning. by doing so, kaft can not only avoid overfitting to conflict data, but also effectively activate its internal knowledge for more efficient domain adaptation. we mainly evaluate our kaft in the medical qa applications upon four popular llms, including llama3-8b/3b @xcite , qwen1.5-7b @xcite . extensive results show that kaft surpasses the other baselines by a clear margin, and brings consistent and significant performance gains across all llms, , i.e., up to +5.73% and +2.40% average scores than the base model and vanilla sft method, respectively. more in-depth analysis prove that kaft can be expanded to other domain-specific applications. more encouragingly, kaft improves the model generalization and alleviates the model hallucination problem effectively. contributions. to summarize, our contributions are three-fold: (1) we propose a query diversification strategy for robust conflict detection. based on it, we conduct a series of preliminary analyses and reveal that training samples with varied conflicts contribute differently. (2) motivated by this, we propose a simple-yet-effective knowledge-aware sft (kaft) approach, which employs sampleadaptive rewards to boost llms' qa performance. ## knowledge-aware fine-tuning based on the observations in §2.3, we recognize that not all training samples are equally important for sft, and llms should pay different attention to different samples. to this end, we propose a knowledge-aware fine-tuning (kaft) approach to alleviate the negative effect of knowledge conflict and boost llms' performance. in this section, we introduce our kaft in detail. ## related works llms @xcite @xcite have showcased powerful general-purpose capabilities. however, they might fall short in domain-specific applications, such as medical qa @xcite . to this end, many prior works @xcite @xcite attempt to perform sft on the domain-specific qa dataset for facilitating domain adaptation. despite achieving remarkable performance, sft often faces a critical challenge, i.e., knowledge conflicts. specifically, since domain-specific sft is more knowledge-intensive and contains rich professional knowledge that has not been learned during the llms' pretraining, there is usually a discrepancy between the llms' internal knowledge and the context knowledge of the sft corpus. more recently, @xcite reveal that sft fails to learn additional knowledge @xcite find that enforcing llms to learn new knowledge through sft would easily damage their prior abilities and lead to hallucination. thus, it is suboptimal to directly fine-tune llms using the full sft training samples equally. to address this problem, there are few existing works @xcite @xcite . however, they still have some shortcomings and struggle to tackle this problem effectively. on the one hand, their conflict detection methods highly rely on icl @xcite , which is sensitive to the few-shot examples @xcite . on the other hand, after detecting the conflict data, they mitigate its negative effect by either using early-stopping or filtering out it from the training dataset, while neglecting how to make full use of these conflict data. different from these prior studies, we first design a query diversification strategy to robustly detect the conflict and then propose kaft to make full use of all training data. the main idea of kaft is to use sample-adaptive rewards for better guiding the learning of llms, which is somewhat similar to prior adaptive-learning methods @xcite @xcite @xcite . we should emphasize that, although assigning different weights to subsets is a common and intuitive approach, it is non-trivial to determine the weights for different subsets, especially for the domain-specific sft of llms. thus, we believe that kaft is innovative and our work is insightful. ## conclusion in this paper, we focus on the knowledge conflict problem in the domain-specific sft, which is critical yet under-explored. specifically, we propose a query diversification strategy to robustly detect the conflict. based on it, we conduct a series of preliminary analyses and reveal that different training samples contribute differently, where those with more conflicts would dynamically damage llms' abilities. to this end, we further propose a knowledgeaware sft approach (kaft). in short, kaft utilizes sample-adaptive rewards to suppress the negative effect of conflict data and encourage llms to activate more relevant knowledge. extensive re-sults on medical qa benchmarks demonstrate the effectiveness and universality of kaft. more encouragingly, in-depth analyses prove that kaft can achieve better model generalization and alleviate the model hallucination effectively.