Structural Entropy Guided Agent for Detecting and Repairing Knowledge Deficiencies in LLMs
Authors: Yifan Wei
Xiaoyan Yu
Tengfei Pan
Angsheng Li
Li Du
Venue: NeurIPS 2025 poster ()
Keywords: Synthetic Data Generation
llm-agents
self-inprovement
MCTS
Primary area: deep_learning
TL;DR: A novel Structural Entropy-guided Knowledge Navigator (SENATOR) framework that addresses the intrinsic knowledge deficiencies of LLMs.

Abstract: Large language models (LLMs) have achieved unprecedented performance by leveraging vast pretraining corpora, yet their performance remains suboptimal in knowledge-intensive domains such as medicine and scientific research, where high factual precision is required.
While synthetic data provides a promising avenue for augmenting domain knowledge, 
existing methods frequently generate redundant samples that do not align with the model’s true knowledge gaps. 
To overcome this limitation, 
we propose a novel Structural Entropy-guided Knowledge Navigator (SENATOR) framework that addresses the intrinsic knowledge deficiencies of LLMs. 
Our approach employs the Structure Entropy (SE) metric to quantify uncertainty along knowledge graph paths and leverages Monte Carlo Tree Search (MCTS) to selectively explore regions where the model lacks domain-specific knowledge. 
Guided by these insights, the framework generates targeted synthetic data for supervised fine-tuning, enabling continuous self-improvement. 
Experimental results on LLaMA-3 and Qwen2 across multiple domain-specific benchmarks show that SENATOR effectively detects and repairs knowledge deficiencies, achieving notable performance improvements.

Structural Entropy Guided Agent for Detecting and Repairing Knowledge Deficiencies in LLMs
Yifan Wei Xiaoyan Yu', Tengfei Pan", Angsheng Li't, Li Du
State Key Laboratory of CCSE, of Computer Science and Engineering, Beiihaa University BBejjig Academy of Artificial Intelligence, 'Beijing Institute of Technology weiyifan@hee edwaaaaa.edd..coo edu. angsheng@buaa. edu cn, duli @iaaiiaai...... ac
Abstract
Large language models (LLMs) have achieved unprecedented performance by leveraging vast pretraining corpora, yet their performance remains suboptimal in knowledge-intensiee domains such as medicine and scientific research, where high factual precision is required. While symthetic data provides promising avenue for augmenting domain knowledge, existing methods frequently generate redundant samples that do not align with the model's true knowledge gaps. To overcome this limitation, we propose a novel Structural Entropy -uided Knowledge Navigator (SENATOR) framework that addresses the intrinsic knowledge deficiencies of LLMs. Our approach employs the Structure Entropy (SE) metric to quantify un- certainty along knowledge graph paths and leverages Monte Carlo Tree Search (MCTS) to selectively explore regions where the model lacks domain-specific knowledge. Guided by these insights, the framework generates targeted synthetic data for supervised fine-tuning, enabling continuous self-improvement. Experimen- tal results on LLaMA-3 and Qwen2 across multiple domain domain-specificc benchmarks show that SENATOR effectively detects and repairs knowledge deficiencies, achiev- ing notable performance improvements. The code and data for our methods and experiments are available at https://gihub.commweeyyye 1023/senator
1 Introduction
With the pretraining process on massive-scale corpora, Large Language Models (LLMs) capture abundant knowledge and demonstrate impressive performance various downstream tasks (Chen) et 2015 Liu et al.] 2021). However, their performance may still be unsatisfactory in certain knowledge-intensing domains such medicine and scientific research. This is primarily due the difficulty in acquiring and scaling up high-quality domain-specific corpora (Lu et al. 2024)) Wang al. 2024), which hinders the ability of the models handle tasks that require high factual precision.
The development of data synthesis technology (Wang et al. 2023 Zhao et al., 2024) offers an alternative way to address these limitations in remedying the knowledge deficiency of LLMs. While promising, the efficiency of data synthesis remains a significant challenge. This is because current data synthesis methods may not consider the model's knowledge boundaries (Jiang et al.) 102
Mallen et al., [0233 Yue et al. 2025). resulting in substantial efforts spent in generating data that the model may already be familiar with. In fact,
with advanced prompt engineering (Wei et al. 2022 [Liu et al. 2025), generated outputs tend to skew toward high-frequency distributions seen in pretraining data, leading to severe redundancy. Therefore, efficient data synthesis should be tightly coupled with mechanisms for effectively detecting knowledge deficiencies (Xiong et al.) 22024) Song al. 2025) within LLMs, so that the synthesized data can repair the knowledge deficiencies.
*Corresponding Authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

However, the knowledge boundaries of large models can be quite complex. Although these models are trained on massive amounts of data, their knowledge is implicitly encoded in model parameters (Geva et al. 2021 Wei et al., 2025) rather than being explicitly stored, leading to unclear distinctions between known and unknown information. In specialized domains, this challenge is compounded by the generation of unreliable or contradictory content (Yang et al. (024c), which produces flawed synthetic samples that hinder the effective expansion of high-quality domain-speciffe corpora.
To overcome the aforementioned challenges, we propose SENATOR, a Structural cettropy-guided Knowledge Navigator framework, which achieves knowledge deficiency remediation through closed loop of structured knowledge probing and targeted synthetic data generation. The framework comprises two key components: 1) Knowledge Deficiency Detection:
knowledge graph (KG) systematically describes the underlying complexities and intricacies of the domain. How- ever, the combinatorial explosion of possible paths makes enumeration computationally infeasible. To efficiently detect the knowledge paths, we drive the L.M as an agent to explore upon the KG in Monte Carlo Tree Search (MCTS) manner [Metropolis and Ulam 1949), with the structure entropy as reward. The Structure Entropy (SE) (Li and Pan] 2016 Li 2024) metric quantifies the structural information contained within a graph by capturing its topological organization and the interactions among nodes. This provides insight into the model's uncertainty along knowledge paths in the KG. By employing MCTS within the knowledge space, our framework uses SE values as intrinsic rewards to decide whether to expand specific entity nodes, effectively prioritizing the exploration of paths with high uncertainty and detecting critical knowledge deficiencies. 2) Knowledge Synthesis and Repair: Leveraging the critical knowledge paths identified via MCTS, our framework generates synthetic data by employing prompt templates to structure the content. The KG serves as a trusted source to ensure both the data inputs and the synthesized outputs credible and contextually relevant. This synthetic data is then used to fine-tune the model through supervised learning, enabling continuous self-improvement and effective remediation of knowledge deficiencies.
Our experiments demonstrate that the SENATOR framework effectively detects knowledge defi- ciencies in large language models and efficiently repairs them, leading to significant performance improvements across multiple domain-spectio that synthetic data incorporates knowledge deficiencies from the pretraining corpus. Moreover, supervised fine-tuning (SFT) of LLMs like Llama-3 (Grattafion et al. 2024) and Qwen2 (Yang et al., (2024b) using this data led to significant performance improvements, demonstrating that targeted injection of missing knowledge can substantially enhance overall model performance.
benchmarks. Data distribution analyses confirm
Related Work
Knowledge Deficiency Detection of LLMs Though LLMs possess extensive knowledge, they often struggle accurately delineate what they know from what they do not (Yin et al. 2023 Ren ct al. 2023). Several approaches (Jiang et al. 2020 Mallen et al. [2023] Wei et al. [024)coonttuuc knowledge probability distributions based on existing annotated data, using metrics such as answer correctness or confidence scores to assess a model's 's knowledge proficiency. One line of work (Wei)
al. 2022 Li et al. 2023a| Tian al. 2024a) directly toward enhancing model's ability to fully leverage its existing knowledge, thereby reducing the proportion of "Unknown Knows". Another line of work pay attention to enabling models to explicitly acknowledge their knowledge gaps. thus minimizing the occurrence of "Unknown Unknowns".. Approaches such as R-tuning ZZang et al. 2023) utilize labeled data with supervised fine-tuning to judge response correctness, while reinforcement learning based strategies have also been explored (Yang et al., 2023b Kang et al.) 2024). In contrast, our approach for deficiency detection is designed not to rely on pre-existing labeled data, but instead to actively explore the KG to detect intrinsic model uncertainty.
Model Self-Improvement Self-improvement methods of LLM focus on leveraging internal knowl- edge and feedback to iteratively enhance the performance of LLMs (Zelikman et al., 2022 2024). pivotal challenge is generating a reliable critique signal to discern high-quality responses from suboptimal ones. Previous methods (Bai et al. 2022 Wang et al. (023) involve prompting the LLM to generate diverse task-specific queries and corresponding outputs, followed by the application of manually crafted heuristic rules, such as filtering based on query length to remove redundant or low-quality data pairs. Given the complexity of devising effective heuristics, subsequent research (Sun et al. Li ct al. 2023b Guo et al. 2024) proposes a few general principles or judging
2023
2

---

Root Node
Current Node
Node
Expanded Node
Terminal Node
Synthetic Datn
L.M
KG
Repcated times
(a) Selection
(b) Expansion
(c) Simulation
(d) Backup
Max SE
Sg
seed entity
MCTS
relations
Exploration
my (als))
relations
V(s
(c)
SF1
(b) Knowlodge Repair
(g) Data Generation
Prompt
(1 Max SE Trajectory
Figure I: The SENATOR framework operates as follows: An entity state in the knowledge graph is (a) selected, (b) expanded, and (c) simulated using the LLM agent until terminal node is reached. Specifically, we employ random policy т during the expansion phase. (d) Subsequently, signals from the value function (-)) are backpropagated. This process is iterated multiple times, with the MCTS algorithm searching for (f) better trajectories guided by (c) signals from structural entropy to (g) generate data addressing knowledge deficiencies, (h) and repair model knowledge.
criteria and ask the LLM itself assess the quality its responses according these guidelines. However, this approach demands that LLMs possess a robust capability to apply these principles to each specific instance and render accurate judgments. Recently, reinforcement learning-based model show impressive reasoning ability by learning the experiences obtained from explorations in the solution space (Tian al. 2024b Goldie al. 2025). While the probability solution space of knowledge intensive tasks would be rather limited as the LLM may not possess the necessary knowledge, which would severely restrict the efficiency of exploration and data generation. In this paper, we choose to guide the exploration process in knowledge space using KGs, in MCTS manner, so as enable targeted synthetic data generation for high efficiency LLM self-improvemmnt.
obtained plausible
Methodology
Given a knowledge graph, the number of possible knowledge paths P (i.e., Figure If increases in a combinatorial speed along with the size of KG, making enumerating all possible paths and detecting the uncertainty of LLM on these paths computationally infeasible. To tackle this challenge, as shown in Figure I SENATOR employs MCTS to navigate the LLM-based agent to search on the KG for seeking out the most informative paths. To steer the agent search toward regions with high uncertainty, we introduce a structural entropy based reward function. Based on the identified high-uncertainty paths, data are synthesized to remediate the identified knowledge deficiencies.
3.1 Structural Entropy Guided Knowledge Deficiency Detection
The structural entropy based reward function combines the uncertainty of LLM on individual KG triplets with the topological structure information the KG, guiding the LLM-based agent to perform MCTS over the KG and discover knowledge paths with critical deficiencies.
Self- Lnff-Ifoommtion for Measuring Trielet-Level Uncertainty Self- leff-Informatio (Shannon 1948
quantifies the amount of information conveyed by a "fact" given its probability distribution. In KGs, "fact" is represented as triplet =< subject u, relation p, object >. To measure the LLM's uncertainty of such "facts", we transform into cloze statement form. The cloze context formed by combining the subject and the relation p, creating prompt predict the missing object The
3

---

self-information of a fact is defined as:
[u, v) = log2 P(e u, p),
(I)
where P(e u. p) is the probability of the output v conditioned on the cloze context. Since the relation p in KGs is directional, the self-information calculated in this manner serves as measure of the factual knowledge confidence for the entire triplet.
Structural Entropy Modeling Knowledge Path-Level Uncertainty To integrate the uncertainty of all triplets along knowledge path while considering their structural importance, we adopt structural entropy (SE) as more comprehensive measure of an LLM's s knowledge confidence. as shown in Figure Tel Structural importance reflects the topological significance of triplet within the knowledge graph. Triplets involving highly connected entities are considered more central, as these entities participate in more relational paths and exert broader influence across the graph. Unlike self-information or Shannon entropy, structural entropy accounts for the knowledge graph's 's topological structure and the interdependencies among its elements. This is crucial because each triplet is not an isolated piece of information but part of structured network. The relationships among entities contribute
the overall representation of knowledge. Given knowledge graph G = (V, E), each edge p E E is assigned a weight derived from the self-confidence in Equation] The weighted degree of an entity node V is defined as:
du = Σ [(u, P.u.
(2)
MEN(M)
where N(u) denotes the set of neighbors of entity u and d,, represents the overall uncertainty contained within the node. To quantify the average information content of the graph G, we define the one-dimensional structural entropy of the weighted, connected graph G as:
H' HC) = - E
du WEV vol(G) log2
dy
vol(G)
where vol(G) represents the total weighted degree of G. higher H. (G) indicates more complex and less confidently represented region within the knowledge graph. By formulating SE as the exploration reward in MCTS, we enable the search algorithm to prioritize paths traversing maximally uncertain knowledge structures, thereby efficiently exposing the model's systemic weaknesses.
3.2 MCTS for Knowledge Deficiency Detection
Given the SE-based reward function, we employ MCTS to explore the KG and identify potential knowledge deficiency paths in the model. We define the initial state so the starting node for traversing the KG, where set of seed entities from (Soman et al. 2024) is selected. KG triplets incrementally incorporated into the knowledge paths until the maximum search depth T is reached. This process enhances the LLM's awareness of its knowledge deficiencies by maximizing the expected reward, which emphasizes the uncertainty associated with these deficiencies.
Node Selection. The objective of this stage is to identify and prioritize KG entities that are likely to expose the LLM's knowledge deficiencies. as shown in Figure Ta Formally, at state st, the LLM agent reaches entity node uı of the KG, and the MCTS process choose from A = {ai, a2...., am }. representing the relation edges Pt+1 that connect the current entity ue to its neighbors A(u)). is guided by two key variables: Q(s,, a). the cumulative value of taking action a in state st, and Ns+),, the visitation frequency state St. Heuristically, Q a) guides exploitation by favoring actions with historically high rewards, while N(s:) encourages exploration of under-visited states. We integrate these complementary objectives using the PUCT algorithm (Rosin) 201I). which selects the next state as:
N (st)
4111 arg max 8t
Q(se, a) + Cpuct P(a (t)
1+ (st,
where P(a)s)) denotes the prior probability of selecting action a given state 8t. In this way, an additional triplet is incorporated into the knowledge path P.

---

Path Expansion. Expansion occurs when a leaf node is reached during the selection phase, enabling the integration of new states and the assessment immediate rewards. Upon reaching a leaf node, is expanded by selecting all possible relation action from leaf node, where each action represents a transition from the current entity state & to a new entity state 8t+1 (sx), as shown in Figure
1b
These unexplored entities (s,,) are then added as leaf nodes to the search tree. The immediate reward function r(s; a) quantifies the advantage each action a A available at state 8t.
der41
r(4 = r (18t,a,8411)) logz vol(G)
T-k-1
(5)
V(8+)= = r1+1 (V(+)))
Σ k=0
y re+h+1:
where y is the discount factor for future state values VV() and the depth of the MCTS search space. To accommodate scenarios with limited decision steps and stable reward distributions, we eliminate the discount factor and instead compute the average future immediate reward values, as formalized in Equation 6
Reward Estimation. A simulation shown in Figure Tc run from the new expanded node 8 by making random relation actions until terminal state reached. The newly expanded nodes are evaluated using an evaluation function integrating future rewards, state relevance, and actual outcomes. In this paper, we propose novel intrinsic reward mechanism to address the limitation of Shannon entropy in handling structured data. To overcome this challenge, we define one- dieedmmnniionll structural entropy an intrinsic reward for effective exploration:
V(s)) = H(P) =E
Ti+k+1
8t
k=
(6)
der vol(g)
d,
~ H) = Σ
log2 vol
P.EP
where {8t, 8t41,"-- ,T) denote the selection trajectory of 1-th iteration, which ends the terminal state 8y after one complete simulation. For simplicity, the notation omits the relationships A between states. Specifically, a subgraph of the knowledge graph G, representing a given search space, and we utilize the structural entropy on this subgraph to approximate the state value.
Backpropagation. We update the statistics of each state in the tree that was traversed during the selection stage. Specifically, the back propagation process updates the value estimates and visit counts of all ancestor nodes along the trajectory P as shown in Figure [d ensuring leaf node evaluation informs higher-level decision decision-making. The updated rules are as follows:
N(s:) < N(st) +1,
N(a))
a) f N(5t,
L(s,, aVVss),
where ((s,, is the number of times relation action has been selected from state 8t, ((s) is the number of times simulation has been run from state 8,, and (st, a) is if relation action a was selected from state the i-th simulation run from state 8t, or 0 otherwise.
3.3 Deficiency Knowledge Synthesis and Repair
As shown Figure to Ih our framework leverages the trajectories with the highest SE values obtained via MCTS to guide synthetic data generation. Specifically, we prompt the LLM agent to synthesize set of QA pairs based on the identified knowledge path on which the LLM shows high uncertainty. so that the knowledge deficiency of the LLM can be remedied by training on these QA pairs. Formally, as shown Figures and [. given a trajectory P (81, #2,.......... 8T), the prompt instructs the LLM to generate question that focuses on P. and an answer that logically explains on the relationship Pt+1 between 8, and its neighboring entities Asxs) P. So that the synthesized QA pair can adhere the underlying knowledge about the knowledge path and remedy
5

---

the knowledge deficiency of the LLM. Furthermore, to maintain high data quality, we implement mullii-tiered evaluation mechanism that includes both heuristic rules and LLM-based judgments. Our quality standards encompass: Format Consistency: The generated QA pairs must strictly adhere to the predefined prompt template, ensuring that the structure, punctuation, and length conform to our specifications. This guarantees that the synthesized data maintains uniform format that facilitates downstream processing. Logical Coherence: The QA pairs must exhibit clear and rational reasoning. The answer should provide a logically consistent explanation that reflects the relationships and context derived from the knowledge trajectory, ensuring that the data effectively captures and addresses the identified knowledge deficiencies. Hallucination Avoidance: The generated content must be grounded in the input trajectory. Specifically, all entities and facts mentioned in the QA pair must originate exclusively from the given trajectory, preventing the introduction of extraneous or unsupported information that could undermine the model's reliability. Data samples that do not meet these criteria are filtered out through our evaluation mechanism A. thereby ensuring that only high-quality synthetic data is used to remediate the LLM's knowledge gaps.
The training process can be divided into two stages: First, knowledge injection stage, that aims to enrich the LLMs with deficiency medical knowledge Dx. Second, a medical instruction tuning stage, that tailors the model to align with the medical QA domain. (see Appendix A.3 for details).
4 Experiments
We conduct experiments on the knowledge-intensive medical domain to investigate the following research questions (RQs): RQ1: Can the proposed SENATOR framework effectively repair the knowledge deficiencies of existing LLMs? RQ2: How do different components of our proposed framework impact the performance of LLMs? RQ3: Does the synthetic data successfully incorporate knowledge that lies beyond the distribution of the pretraining corpus? RQ4: What is the scaling regularity of synthetic data on model performance?
4.1 Experimental Settings
Language Models We evaluate our methodology on two categories of LLMs: 1) General LLMs: We employ Llama-3-8B and Qwen2-7B as base models to examine the effectiveness of our approach and include Baichuan2 and Llama-2 for comparison. 2) Medical LLMs: Med-Alpaca (Han et al.) 2023): Fine-tuned on LLaMA-13B with medical instruction data from Alpaca (Han et al. (023). tasks. PMC-LLaMA (Wu et al.. 2024): Enhanced with biomedical knowledge from 4.8 million academic papers and 30,000 medical instruction tuning on ALaMA-13B HuatuoGPT-II (Chen et al., 2023a): Built on Baichuan (Yang et al. 2023a),, fine-tuned with distilled ChatGPT data and real-world medical data from doctors.
specifically designed for medical dialogues and
books, followed by medical-specii
Datasets Our instruction tuning data Dj, which contains 514k samples, is derived from Wu al. (2024) to align with the medical domain. It's widely used in the medical field for its large scale and comprehensive coverage of medical knowledge. We evaluate our approach on five standard medical benchmarks: 1) MedQA (Jin et al. (02T): Multiple-choice questions from the USMLE assessing medical understanding and reasoning. 2) MedMCQA (Pal et al., 2022): Over 194K questions from AIIMS exams covering 2,400 topics across 21 subjects. 3) PubMedQA (Tin et al. 2019): A biomedical QA dataset from PubMed abstracts with IK expert-annotate and 211K generated QA instances, designed to test comprehension and reasoning in biomedical rescarch. 4) GPQA (Rein ct al. ):23): A high-difficulty multiple-choice dataset validated by experts in biology, physics, and chemistry, focusing on interdisciplinary knowledge and reasoning. 5) MMLU (Hendrycks et al. 2020 A comprehensive benchmark covering 57 tasks for evaluating large language models.
We conduct experiments based on the SPOKE knowledge graph (Morris et al. 2023) due to its comprehensiveness on biological and medical knowledge, which contains over 42 million nodes of 28 different types and 160 million edges of 91 types, constructed by integrating information from 41 different biomedical databases. this paper, the initial seed entities for MCTS common discasc entities in SPOKE, sourced from Soman et al. (2024).
Knowledge Graph

---

Table 1: Main Results on Medical Benchmarks in the Zero-shot Setting. A represents the relative change in performance when using our synthetic data generated by SENATOR compared to the corresponding backbone model. "w/" denote "with" and IT represents instruciton tuning data.
GPQA
Model
MedQA
MedMCQA
PubMedQA
Avg.
Molecular Biology
Genetics
Human (pass) Human (expert)
50.0
60.0
43.2
87.0
90.0
78.0
66.7
80.43
Medical L.Mss
Chat-Doctor (7B)
33.93
31.10
54.3
Med-Alpaca (13B)
30.85
31.13
53.2
10.0
15,43
28.12
HuatuoGPT-II (7B)
41.13
41.87
54.2
22.5
21.60
36.26
HuatuoGPT-II (13B)
45.72
38.75
51.6
20.0
27.78
36.77
PMC-LLaMA (13B)
50.67
50.18
59.8
15.0
27.16
40.56
General LLMs
Baichuan2-7B
34.56
35.12
60.2
20.0
20.99
34.17
Baichuan2-
43.60
39.25
50.7
27.5
30.86
38.38
Llama-2-7B
30.95
28.85
60.8
25.0
17.28
32.58
Llama-2-133
31.26
29.00
62.2
35.0
20.99
35.69
Llama-3-83
55.54
52.21
54,8
20.0
29.01
42.31
w/ instruction tuning
54.36
50,08
56.6
25.0
25,93
42.39
synthetic data + IT A promotion
58.29
53.60
64.8
27.5
32.72
47.38
+4.95%
+2.66%
+ +18.25%
+37.50%
+12.79%
+11.98%
Qwen2-7B
54.67
53.41
64.6
32.5
36.42
48.32
w/ instruction tuning
59.07
59.77
61.2
22.5
35.80
47.67
w/ synthetic data + IT
59.70
60.70
63.2
40.0
40.12
$2.74
A promotion
+9.20%
+13.65%
-2.17%
+26.08%
+10.16%
+9.15%
4.2 Main Results (RQ1)
Table I presents the performance of our approach and baseline models across four medical bench- marks. From this, we observe that (1) Through continuous pretraining medical corpora, previous medical domain LLMs such as PMC-LLaMA could achieve ardinary-human certain benchmarks. For example, PMC-LLAMA employs approximately 514k samples, 79 billion tokens of medical data to achieve performances close to such as MedQA and PubMedQA. However, its performance on genetics related subset of GPQA still shows substantial gap with human-level, indicating significant knowledge deficiency. (2) In contrast, our proposed SENATOR framework demonstrates its effectiveness in finding knowledge deficiencies efficiently adapt L.MSs the medical domain. When applied to Llama-3-8B and Qwen2-7B, the SENATOR framework uses a much smaller amount of synthetic data (26k samples, 0.8 million tokens and 128k samples, 3.6 million tokens, respectively) to remedy the targeted knowledge areas, and improve the performance on corresponding benchmarks. For instance, the SENATOR optimizes the Qwen2 model attains an accuracy of 40% on the Genetics component of GPQA, demonstrating that supplementing missing Nomain-speciiff data can substantially enhance performance. Overall, on the four medical domain- related benchmarks, on average. the SENATOR framework improves the performance of Llama-3-88 and Qwen2-7B for 11.98% and 9.15%. respectively. This shows the effectiveness and generality of our approach in comprehensively detecting and remedying the domain-related knowledge for different L.LMs. In the following paragraphs (RQ2 and RQ3), we demonstrate that the improvement stems from SENATOR's 's ability to effectively detect the knowledge deficiencies by synthesizing data beyond the original pretraining corpus, expanding its coverage, and optimizing its distribution.
performance on
genetics-related
4.3 Ablation Study (RQ2)
To validate the efficacy SENATOR, we conduct ablation studies comparing three configurations: (L) base models, (2) models fine-tuned solely with general domain instruction data D,, and (3) models

---

L0
MNC Crrpones
Gerpores
-10
NC Comporant
Component
MC Comporant
PMC Componer
(5) PubMedQA
MeeQA
(c) MalMMQA
(4) Hyhrid
10
NC ceroree
Corpcei
10
-10
Camborant
INC Comporant
MAC Companen:
PAC Component
(c) PubMedKJA
(1) McdJA
(e) McdMCCJA
(h) Hybnd
Figure 2: Distribution Pretraining Corpus vs. Synthetic Data. In (a)-(d), blue regions represent the medical pretraining corpus (PubMedQA, MedQA, MedMCQA, and their hybrid), red regions show synthetic data generated by Llama-3. In (e)-(h), red regions indicate synthetic data produced by Qwen2. Darker areas reflect higher concentrations of data points, lighter areas vice versa.
trained with both instructions and synthesized data. As shown in Table T1 SFT on general domain instructions alone yields marginal improvements even performance degradation (Llama-3-8B: 42.31 + 42.39; Qwen2-7B: 48.32 - 47.67). This suggests that the general domain instructions struggle alleviate the intrinsic knowledge gaps in general-domain LLMs for the specialized medical domain, and constructing more general domain instructions would inevitably be inefficient. In contrast, incorporating synthetic data leads to significant improvement. For Llama-3-8B, additional synthesized data make average performance improvements of 5.07, with particularly significant gains in underrepresented domains: +7.5 points in GPQA Genetics and +3.71 points in Molecular Biology. Similarly, Qwen2-7B attains 40.0% accuracy in GPQA Genetics (7.5-point increase) and 40. 40.12% in Molecular Biology (3.7-point gain). These results indicate that performance improvement is brought by synthesizing data from detecting the deficiency of LLMs instead of simply enlarging the size of existing instruction data, and seficiency-orientee synthetic data generation strategy would be a more efficient method for expanding knowledge of LLMs, suggesting a way towards "new fuel" 2023) for enriching the existing corpus and empowering future LLMs.
(PwC Australia
4.4 Analysis for Distribution of Synthesized Data (RQ3)
To examine if our approach generate synthetic data beyond the original pretraining distribution and address the knowledge deficiency of LLMs, visualize the dis- tribution of both the original pretraining data, which is sourced from the training sets of PubMedQA, MedQA, and MedMCQA, and the synthetic data. This visualization achieved by first projecting data into unified semantic space using 2D UMAP (McInnes et al. 2018) and ob- taining their distribution using kernel density estimation (KDE) [Rosenblat] 1956 Parzen 1962). From Figure we can observe: 1) Expanded Coverage by synthetic data: Figures 2a to 2h reveal that the red area (representing synthetic data) encircles the blue area (pretraining data), indicating that the synthetic data effectively broadens the coverage of the pretraining data. Additionally, Figure (b) and 2 display smaller blue regions, indicating that the distribution of synthesized data is much broader than the pretraining data available for MedQA. 2) Distribution Overlap: In Figure 2d the synthetic data
20
PAC Component
10
-10
-5
5
10
15
20
25
PAC Component
Figure Distribution of Data Generated by Llama-3 (red) and Qwen2 (blue).

---

swap
66
07.5
56
02
62
93
63.0
30
56
52.5
54
0.1
0.3
0,6
0.9
1.5
ModQA
NoLMCOA
PulMedia
Ang
CK
CM
340
A0
CB
PM
A0g
(a) Different scaling
(b) Data exchange
(6)
Various subdomain
Figure 4: Performance differences for various data compositions.
shows high degree of overlap with the overall pretraining data. We hypothesize that this may be due to Llama-3's ' relatively weaker grasp of pretraining knowledge compared to Qwen2, causing SENATOR to collect information that Llama-3 did not consolidate well during pretraining. 3) Topic- Specific Differences: Compared to Figure 2a Figure 2 exhibits an opposite trend. Accordingly, indicated in Table Il Qwen2 demonstrates higher performance on PubMedQA. This is likely because Qwen2 demonstrated a stronger mastery of PubMedQA during pretraining (Yang et al., 2024a), leading SENATOR to explore that topic distribution lesser extent during the defect detection phase. 4) Global Trends and Localized Discrepancies: The analysis of synthetic data distributions generated by Llama-3 and Qwen2 (Figure 3) shows substantial overlap in high-density areas, indicating that both models have a roughly similar pattern (may also share with more LLMs) in knowledge deficiency about the medical domain. This is because of the similarity in the distribution of the pretraining corpus (Lee et al. 2022 Yauney et al. (202)) Such similarity indicates the necessity of systematically reviewing the deficiencies of present LLMs to find common knowledge blind spots in the pretraining corpus, and synthesizing data to complement them. However, there still exist differences certain locations, suggesting model-specific knowledge deficiencies. This suggests the effectiveness of our approach in targeting model-specific knowledge deficiencies.
4.5 Analysis of Synthetic Data Scaling (RQ4)
To explore how the amount of synthetic data affects model repair, we integrate different proportions of synthetic data into the SFT stage, as depicted in Figure 4a We observe upward trend in overall performance, calculated as a weighted average based on dataset sizes, with increasing synthetic data proportions. This indicates that, when the instruction-aligaee data enhances model performance. As more synthetic data used, more LLM knowledge deficiencies can be identified and addressed, thereby improving the model's I's performance. This highlights the potential of our method effectively boost model performance by targeting and synthesizing data to fill specific knowledge gaps. Due to the limitation in computation resources, in this paper, for the two base LLMs, Llama and Qwen, we synthesize 26k and 128k data entries, respectively. future work, we will explore integrating diverse knowledge across more domains to further enhance model performance. Additionally, we compare two settings: the default setting (SENATOR), where each model is fine-tuned using data synthesized using its own detected deficiencies, and the swap setting, where a model is trained with data synthesized using deficiencies of another model, for example, synthetic data produced by Llama-3 is used for SFT of Qwen2, and vice versa. As shown in Figure 4b and (4c) SENATOR demonstrates effective deficiency correction even under the swap setting. This could be brought by the similarities between the pretraining corpus of different LLMs, which lead to similar knowledge deficiencies. This finding not only reinforces the potential of our synthetic corpora, but also highlights the pressing need for efficient and comprehensive strategies to detect and repair knowledge deficiencies in LLMs.
data D, is fixed, expanding the synthetic
data as a valuable supplement to
5 Conclusion
In this paper, we introduce SENATOR, an innovative framework that utilizes structural entropy and knowledge graphs to detect and repair knowledge deficiencies in LLMss. By employing MCTS within the knowledge space, SENATOR effectively identifies areas where the model's 's understanding is deficient. Leveraging the SENATOR agent, direct the synthetic data generation process to

---

specifically target these deficiencies. Our experiments on medical benchmarks reveal significant performance improvements when models like Llama-3 and Qwen2 are fine-tuned with the synthetic dataset. These results highlight that deficiency-oriente synthetic data generation strategy represents a highly efficient and sustainable method for expanding knowledge, positioning as the "new fuel" of modern AI.
6 Acknowledgements
We thank the support of the National Science and Technology Major Project (2022ZD01 16301), the General Program of the National Natural Science Foundation of China (62576021) and the Youth Fund of the National Natural Science Foundation of China (62406040).
References
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073.
Junying Chen, Xidong Wang. Ke Ji, Anningzhe Gao, Feng Jiang. Shunian Chen, Hongbo Zhang, Dingjie Song, Wenya Xie, Chuyi Kong, et al. 2023a. Huatuogpt-ii, one-stage training for medical adaption of Ilms. arXiv preprint vrXiv:2311.09774
Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015. Event extraction via dynamic muult--pooling convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing ( (Volume I: Long Papers), pages 167-176.
Zeming Chen, Alejandro Hernández-Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi. Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, Alexan- dre Sallinen, Alireza Sakhaeirad, Vinitra Swamy, Igor Krawczuk, Deniz Bayazit, Axel Marmet, Syrielle Montariol, Mary-Anne Hartley, Martin Jaggi, and Antoine Bosselut, 2023b. Meditron-7Ob/ Scaaing medical pretraining for large language models Preprint, (tt.:2311.1660)
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer nerw-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empiricul Methods Natural Language Processing, pages 5484-5495.
Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai, and Christopher D Manning. 2025. Synthetic data generation multi-step for reasoning & tool use. arXiv preprint 00X74:2504.04736.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The Ilama 3 herd of models. arXiv preprint 24X.v:240..21788
Hongyi Guo, Yuanshun Yao, Wei Shen, Jiaheng Wei, Xiaoying Zhang, Zhaoran Wang, and Yang Liu. 2024. Human-instruuciionfree Ilm self-alignment with limited samples. arXiv preprint arXiv:2401.0678
Tianyu Han, Lisa C Adams, Jens-Michali Papaioannou, Paul Grundmann, Tom Oberhauser, Alexan- der Löser, Daniel Truhn, and Keno K Bressem. 2023. Medalpaca-an open-source collection of medical conversational ai models and training data. arXiv preprint wrXiv:2304.08247
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Ja- cob Steinhardt. 2020. Measuring massive multitask language understanding. In International Conference on Learning Representations.
Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. 2021. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962-977.
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423-438.
10

---

Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits, 2021. What disease does this patient have? large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):4421.
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNI.-IICNILP), pages 2567-2577.
Katie Kang, Eric Wallace, Claire Tomlin, Aviral Kumar, and Sergey Levine. 2024. Unfamiliar finetuning examples control how language models hallucinate. arXiv preprint wwXiv:2403.050/
Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024. Biomiisral:: A collection of open-source pretrained large language models) for medical domains, Preprini, ar xXiv:2402.10377.
Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison- Burch, and Nicholas Carlini. 2022. Deduplicating training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Compulational Linguistics (Volume
1: Long Papers), pages 8424-8445.
Angsheng Li. 2024. Science of Artificial Intelligence: Mathematical Principles of Intelligence (In Chinese). Science Press, Beijing.
Angsheng Li and Yicheng Pan. 2016. Structural information and dynamical complexity of networks. IEEE Transactions Information Theory, 6266):3290)3339.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Plister, and Martin Wattenberg. 2023a. Inference- time intervention: Eliciting truthful answers from a language model In Advances in Neural Information Processing Systems, volume 36, pages 41451-41530, Curran Associates, Inc.
Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, Jason Weston, and Mike Lewis. 2023b. Self-alignment with instruction backtranslation. arXiv preprint arXiv:2308.066 06259.
Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang. and Yue Zhang. 2021. Logiqa: challenge dataset for machine reading comprehension with logical reasoning. In Proceedings of the Twenty-innh International Conference on International Joint Conferences on Artificial Intelligence, pages 0222-628.
Kai Liu, Ze Chen, Zhihang Fu, Rongxin Jiang, Fan Zhou, Yaowu Chen, Yue Wu, and Jieping Ye. 2025. Structure-aware domain knowledge injection for large language models. pages 144444
Zimu Lu, Aojun Zhou, Houxing Ren, Ke Wang, Weikang Shi, Junting Pan, Mingjie Zhan, and Hongsheng Li. 2024. Mathgenie: Generating synthetic data with question back-translation for enhancing mathematical reasoning of llms. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 2732-2741
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non- parametric memories. In Proceedings of the 61 Annual Meeting of the Association for Computa- fional Linguistics (Volume I: Long Papers), pages 9802-9822.
Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großherger. 2018. 1. Umap: Uniform manifold approximation and projection. Journal of Open Source Software 3(29):861.
Nicholas Metropolis and Stanislaw Ulam. 1949. The monte carlo method. Journal of the American
statistical association, 44(247):335-341
John H Morris, Karthik Soman, Rabia Akbas, Xiaoyuan Zhou, Brett Smith, Elaine C Meng, Conrad C Huang, Gabriel Cerono, Gundolf Schenk, Angela Rizk-Jackson, et al. 2023. The scalable precision medicine open knowledge engine (spoke): a massive knowledge graph of biomedical information. Bioinformatics, bta2)o0
11

---

Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmeqa: A large-scald multi-subject multi-choice dataset for medical domain question answering In Proceedings of the Conference Health, Inference, and Learning, volume 174 of Proceedings of Machine Learning Research, pages 248 260. PMLR.
Emanuel Parzen. 1962. On estimation of a probability density function and mode. The annals of mathematical statistics, 33(3):1065-100)
PwC Australia. 2023. Synthetic data: The new fuel for ai
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023. Gpqa: A graduate-level google-pres proof q&a benchmark. In First Conference on Language Modeling.
Ruiyang Ren, Yuhao Wang. Yingqi Qu, Wayne Xin Zhao, Jing Liu, Hao Tian, Hua Wu, Ji-Rong Wen, and Haifeng Wang. 2023. Investigating the factual knowledge boundary of large language models with retrieval augmentation. arXiv preprint arXiv:2307.11000
M Rosenblat. 1956. Remarks on some nonparametric estimates of density function. Ann. Math. Stat, 27:832-837.
Christopher D Rosin. 2011. Multi-armed bandits with episode context. Annals of Mathematics and Artificial Intelligence, 61()22030-33.
Claude E Shannon. 1948. A mathematical theory of communication. The Bell system technical journal, 27
Karthik Soman, Peter W Rose, John H Morris, Rabia E Akbas, Brett Smith, Braian Pectoom, Catalina Villouta-Reyes, Gabriel Cerono, Yongmei Shi, Angela Rizk-Jackson, et al. 2024. Biomed- ical knowledge graph-optimized prompt generation for large language models. Bioinformaties, :bt
Linxin Song, Xuwei Ding, Jieyu Zhang, Taiwei Shi, Ryotaro Shimizu, Rahul Gupta, Yang Liu, Jian Kang, and Jieyu Zhao. 2025. Discovering knowledge deficiencies of language models massive knowledge base. arXiv preprint arXiv:2503.23300
Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang. and Chuang Gan. 2023. Principle-driven allf-alignment of language models from scratch with minimal human supervision. arXiv preprint arXiv:2305. 2305.03047.
Katherine Tian, Eric Mitchell, Huaxiu Yao, Christopher D Manning, and Chelsea Finn. 2024a. Fine-tuning language models for factuality In The Twelfth International Conference on Learning Representations.
Ye Tian, Baolin Peng, Linfeng Song, I Lifeng Jin, Dian Yu, Lei Han, Haitao Mi, and Dong Yu. 2024b. Toward reff-immprvemeett of lims via imagination, searching, and criticizing. Advances in Neural Information Processing Systems, 37:52723-52748.
Yizhong Wang. Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannanch Hajishirzi. 2023. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13484-13508.
Zifeng Wang, Chun-Liang Li, Vincent Perot, Long Le. Jin Miao, Zizhao Zhang, Chen Yu Lec, and Tomas Plister. 2024. Codeclm: Aligning language models with tailored synthetic data. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3712-3729.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 15:24824-24883
Yifan Wei, Xiaoyan Yu, Ran Song, Hao Peng, and Angsheng Li. 2025. Setke: Knowledge editing for knowledge elements overlap. arXiv preprint 2r52v:2504.20972.
12

---

Yifan Wei, Xiaoyan Yu, Yixuan Weng, Huanhuan Ma, Yuanzhe Zhang, Jun Zhao, and Kang Liu. 2024. Does knowledge localization hold true? surprising differences between entity and relation perspectives in language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 4118 4122.
Chaoyi Wu, Weixiong Lin, Xiaoman Zhang. Ya Zhang, Weidi Xie, and Yanfeng Wang. 2024. Pmc- aaaa:: toward building open-source language models for medicine. Journal of the American Medical Informatics Association, (1(9):
Kai Xiong, Xiao Ding, Li Du, Jiahao Ying, Ting Liu, Bing Qin, and Yixin Cao. 2024. Diagnosing and remedying knowledge deficiencies in llms via label-free curricular meaningful learning. arXiv preprint arXiv:2408. 11431.
Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, al. 2023a. Baichuan 2: Open large-scale language models, arXiv preprint 2309::3000.1300
An Yang. Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang. Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang. Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang. Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao. Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. 2024a. Qwen2 technical report. arXiv preprint arXiv:2407. 10671.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024b. Qwen2. 5 technical report. arXiv preprint 152iv:2412.15115.
Rui Yang, Haoran Liu, Edison Marrese-Taylor, Qingcheng Zeng, Yuhe Ke, Wanxin Li, Lechao Cheng, Qingyu Chen, James Caverlee, Yutaka Matsuo, et al. 2024c. Kg-rank: Enhancing large language models for medical qa with knowledge graphs and ranking techniques. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 155-166.
Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, and Pengfei Liu. 2023b. Alignment for honesty. arXiv preprint arXiv:23
Gregory Yauney, Emily Reif, and David Mimno. 2023. Data similarity is not enough to explain language model performance. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1 1295-11304.
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. 2023. Do large language models know what they don't tt know? In Findings of the Association for Computational Linguisties: ACL 2023, pages 6653-8665 Toronto, Canada. Association for Computational Linguistics.
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. 2025. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.1387.
Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah D Goodman. 2024. Quiet-star: Language models can teach themselves to think before speaking. arXiv preprint arXiv:2403.09629
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 15:16476-155p 15488.
Hanning Zhang, Shizhe Diao, Yong Lin, Yi R Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, and Tong Zhang. 2023. R-tning: Teaching large language models to refuse unknown questions. arXiv preprint arXiv:2311.0966
13

---

Kaiyan Zhang, Ning Ding, Biqing Qi, Sihang Zeng, Haoxin Li, Xuekai Zhu, Zhang- Ren Chen. and Bowen Zhou, 2024. Ultramedical: Building specialized generalists in biomedicine. https //github.
Hanyu Zhao, Li Du, Yiming Ju, Chengwei Wu, and Tengfei Pan. 2024. Beyond iid: Optimizing instruction learning from the perspective of instruction interaction and dependency. arXiv preprint arXiv:2409.07045.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please refer Abstract Section and Introduction Section 1
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please refer Section B
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
14

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Theorems and Lemmas that the proof relies upon have been properly referenced as shown in Section
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please refer the experimental setting 4 and A.3
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
15

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
Please refer the Supplementary Material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
swlssiiiiiiioiiioiiiooooooiiiill for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7 //nips http://cccpbbllcguuudoeeooo
Coweeubmissiieieeooeo Pollcyy))))) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer to the experimental setting 4 and A.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All the results are obtained with fixed random seeds, and we repeat the experiment three times verify the consistency of the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer to Section
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We have understood the NeurIPS Code of Ethics in detail and promise that it has not been violated.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer Section
17

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not involve the release of data or models with a high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: All models, data, or codes used in our work are free and open sourced.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.coo//aaaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide documentation for the use of the code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our work do not involve crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work do not involve this question.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our work do not involve this question.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
20

---

A Technical Appendices and Supplementary Material
A.1 Prompts for Synthetic Data Generation Stage
This section introduces the prompts (Figure S and and 7 defined in our synthetic data generation phase. including the question-answer paris generation prompt, and the evaluation prompt. And Figure shows a specific example generated by SENATOR using the generation prompt.
Synthetic Data Generator (Step 1)
For given facts, generate question and its corresponding answer. The question should be designed to inquire about the relationship or clussification described in the triples, and the answer should be un entily mentioned the provided facts. Facts:
Discase <Thyroid Gland Mucoepidemood Carcinoma> is type of discase <thyroid gland carcinoma?.
Compound <1 Loothyyonine>> treats discase <thyroid gland carcinoma>.
Question: What compound can be used to treat Thyroid Gland Mucoepidermoid Carcinoma? Liothyronine.
Answer:
Facts:
Disease <thyroid gland carcinoma> resembles disease <gunglioneuroma>
Disease <gunglioneuroma> presents Symptom <Diarrhea>> Question: What symptom is associated with the discase that resembles thyroid gland carcinoma?
Answer: Diarrhea.
Facts:
Discase <head and neek cancer reaembles <thhrood gland carcinoma?.
Disease <head neck cancer presents Symptom =Dyyphonia>,
Disease <head neck cancer? presents Symptom <Neck Pain>.
Discase <thyroid gland carcinoma presents Symptom <Dysphonia>.
Discasc <thyroid gland carcinoma> presents Symptom <Nock Pain>.
Compound <Paclitaxel>> treats discase <head and neek cancer>.
Question: What diseuse is similar to thyroid glund carcinoma, with Symptom Dysphonia and Neck Pain.
Answer: Heud und neck cancer.
Figure 5: Example prompt for the synthetic data generation stage of SENATOR.
A Smaple Generated by SENATOR
igencration prompt)
| Input: Maximum Structual Entropy Trujectory by SENATOR
Disease <hyperhhosphatemie contruindicales the use of compound <Retinol>>
Compound <Retinol> is contained in food <hickory nut?: Food <hickory nut> contains compound <Tryptophan>, Compound < <Trypiophan>> is contained in food <cow milk (liquid)>
I Output: Q4 Samples generated by the LLMs Question: Which compound, present in both hickory nut and cow milk (liquid). is safe for consumption by individual with hyperphosphatemia? Answer: Tryptophan.
Figure 6: A specific example generated by SENATOR.
A.2 Prompts for the SFT Evaluation Stage
This section introduces the evaluation prompt (Figure K used after model knowledge repair, as shown in Figure Th designed to align the model's output answers with the desired format in the medical domain. Specifically, we employ a zero-shot setting our evaluation to reduce the model's sensitivity bias to few-shot examples.
21

---

Sample Evaluation Scorer (Step 2)
Your task is to evaluate the given QA Pairs with Evidences based on the following criteria.
The criteria should include three parts:
Verify the question is complete (i.e., not truncated) and be answered with single, clear answer. Check the answer complete (i.e., not iruncated) and is presented single entity concise ohjeee-e statement.
Formar:
Confirm that there clear, derivable logical connection between the question and the answer based provided Evidences.
Logic:
Hallucination: This examines whether the entities involved in the question and answer exist within the provided Evidences. determines if additional information beyond the given Evidences was used to construct the samples.
For each QA sample, unalyze whether il meets the above criteria. If the sample satisfies all criteria, oulpul "Currect". Otherwise, output one of the error types that best describes the issue with the sample: Format, Logic, or Hallucination.
Figure 7: Example prompt for the sample filtering stage of SENATOR.
Eval Prompt for Medical Datasets
# Instruction:
Directly answer the best option or Directly answer yes/nn'mmaybe:
# Example (PubMedQ4)):
# Abstract: Electrical ncurostimulation has traditionally been limited to the use of charge-balanced waveforms. Charge-imbalanced and monophasic waveforms these stimulation paradigms may generate noxious electrochemical species that tissuc damage. In this study, investigated the dissolution of platinum
used to deliver clinical therapy, because is believed that
such irreversible reactions μC emn We ubserved that platinum dissolution decreased during charge- imbulanced and munophasic stimulation when compared to charge-balunge balunced waveforms
range of charge densitics up 160
# Question: Does electrical neurostimulation with imbalanced waveforn mitigate dissolution of platinum electrodes?
# Example (MelQA)
# A 3-month-old buby died suddenly at night while usleep. His mother noticed that he had died only afer she awoke in the morning. No cause of death was determined based on the autopsy. Which of the following procautions could have prevented the death of the baby?
# A. Placing the infant in a supine position on firm mattress while sleeping.
# B. Keeping the infant covered and maintaining high room temperature. # C. Application of device maintain the sleeping position.
# D. Avoiding pacifier during sleep.
Example (MedMCQA):
# Which vitamin is supplied from only animal
# A. Vitamin C Vitumin B7 Vitamin B12 Vitamin D
Figure 8: Example prompt for the evaluation on medical datasets, where the "#" symbol denotes comments illustrating how a specific data sample is combined with instruction for zero-shot prompting.
A.3 Supervised fine-tuning hyperparameters
use cross-entropy for supervised fine-tuning. Table 2 presents the hyperparameters utilized for SFT of L.MMs within the SENATOR framework. As shown in Table the settings applied to Llama-3-8B are identical to those of Qwen2-7B. Moreover, experiments conducted this paper have been performed using the same hyperparameter configuration.
22

---

Table 2: Model Training Parameters in SENATOR
Model
Learning Rate
Weight Decay
Warmup Step
Batch Size
Epoch
Maximum Sequence Length
Llama-3-8B
9.65c-6
1024
Qwen2-7B
9.65e-6
1024
A.4
Data Filtering
While our framework demonstrates significant improvements over baseline methods, we acknowledge that the system remains imperfect. To systematically evaluate its limitations, we conduct manual examination of 501 randomly sampled QA pairs from SENATOR outputs. The analysis revealed that 311 samples (62.08%) met our quality criteria for valid questionnanswer pairs. The remaining 190 error-containing samples (37.92%) exhibited the following eπror distribution: Formulaic errors (84 samples; 16. 6.77%) Questions answers with truncations, formatting inconsistencies, or multi-answer requirements. Logical errors (98 samples; 19.56%): Answers lacking evidential support from the provided knowledge triples. Hallucination errors (8 samples; 1.59%): Answers referencing entities absent in the supporting evidence. Notably, while our approach effectively mitigates hallucination errors through evidence grounding, generating logically consistent QA pairs remains challenging. This primarily stems from the base model's s inherent limitations in performing multi-hop reasoning across knowledge path. Appendix [A.7 illustrates representative examples of these error categories, demonstrating both the framework 's capabilities and its current limitations. In order to improve data quality, we set up an additional data filtering module. For format problems, we use regularization to remove samples that do not meet specifications. For logical error types, we use LLMs to judge the logical consistency of QA pairs and evidences, and filter out unsatisfied samples.
A.5 Impact of synthetic data on different medical subfields
Similar phenomena shown in 4 can also be observed in different medical-related subdomains in the MMLU dataset, as shown in Figure 9 Our analysis on Qwen2 shows that without sythetic data generated by SENATOR (ratio 0), performance is lowest. As synthetic data in- creases, sub-domain performance improves but with fluc- tuations. We attribute this to SENATOR's 's lack of entity type consideration during KG exploration, causing ran- dom data domains and non-uniform categories. Future work will focus on adding entity type constraints in MCTS search to explore domain specific knowledge deficiencies more precisely.
College Medasine
615
Medicel Cenctics
Anotory
Callege Bislngy
Prodiotoonal Madicine
57.5
Anscuge
Figure 9: Performance across Different Ratios in MMLU Medical Aspects.
A.6 Comparison with Latest Medical LLM Baselines
To provide more comprehensive evaluation against recent state-of-the-a ahr medical LLMs, we have added new baselines including BioMistral-7B (Labrak et al. 2024), Meditron- --B (Chen et al. 2023b). Llama-3-8B-- 3-rr--eraeeeiicall (Zhang et al. 2024), and Qwen2-7B w/ SENATOR. The results are presented Table 3 below.
Table 3: Model Performance on Medical QA Benchmarks
Model
MedQA
MedMCQA
PubMedQA
BioMistral-7B
44.93
42.17
56.4
Meditron-7B
30.40
31.22
61.6
56.75
53.75
52.12
Llama-3-8B w/ SENATOR
58.29
53.60
64.8
Qwen2-7B w/ SENATOR
59.70
60.70
63.2
23

---

A.7 Case Stduy
Our framework SENATOR generates <evidence, question, answer> examples based on the SPOKE knowledge graph. These examples are categorized into four types: Correct, Formulaic enrors, Logical emros, and Hallucination errors. Specific examples are illustrated Figures 10 13
Evidenee:
Disease <hyperphosphatemia> contraindicates the use of compound <Retinol>, Compound <Retinol> is contained in food <hickory nut>, Food <hickory nut> contains compound <Trypiophan>, Compound <Tryptophan> is contained in food <cow milk (liquid)>
Question:
What food contains the compound that is contraindicated in hyperphosphatemia?
Answer: Hickory nut
Comment: Correct
Figure 10: Correct Case.
Evidence:
Disease <primary ciliary dyskinesia 25> is a type of disease <primary ciliary dyskinesia>,
In genetics, disease <primary ciliary dyskinesia> associates with gene <MCIDAS>,
Gene <MCIDAS> downregulated in tissue <ectocervix>
Question:
In which tissue is gene MCIDAS upregulated?
Answer: Endometrium
Comment: Hallucination error
Figure 11: Hallucination Error Case.
A.8 Details of the Instruction Tuning Dataset
the dataset includes approximately 100k instances from the ChatDoctor corpus, which contains diverse doctor-patiens dialogues collected from real-world scenarios. To enhance instruction diversity and robustness, each prompt is expanded into multiple semantically equivalent forms using GPT-4.
Medical Conversation Data:
Medical Rationale Question Answering: the dataset incorporates three major multiple-choice QA benchmarks: MedQA (10.2K examples), MedMCQA (183K). and PubMedQA (211K). These datasets evaluate the model's ability to reason over professional medical knowledge. Since many of these resources originally lacked detailed rationales, additional causal explanations were obtained by prompting ChatGPT, allowing the model learn both the correct answer and the underlying reasoning.
24

---

Evidenec:
Disease <acute necrotizing encephalitis> resembles disease <encephalomyelltis,,
Disease <encephalomyelltis>> presents symptom <Myallia>, Symptom <Myalgia> can be caused by the side effect of compound <Diazepam>
Question:
What disease has a similar presentation to acute necrotizing encephalitis, with a symptom that can be treated by Diazepam?
Answer: Encephalomyelitis
Comment: Logical error
Figure 12: Logical Error Case.
Evidence:
Disease <otulipenia> type of disease <autosomal recessive disease
Disease Kautosomal recessive disease> includes disease <spondyloepiphyseal
dysplasia Kondo-Fu type>
Disease <spondyloepiphysen dysplasia Kondo-Fu type> presents symptom
<Cataract>,
Symptom <Cataract> can be caused by the side effect of compound <Imatinib>
Question:
What is the type of discase that presents symptom Cataract, and what is the side effect of Imatinib?
Answer: Spondyloepiphyseal dysplasia Kondo-Fu type, Cataract
Comment: Formulaic error
Figure 13: Formulaic Error Case.
Knowledge Graph-Driven Prompting: Furthermore, two smaller batasets--LveOQ) (635 examples) and MedicationQA (690 are included to provide real-world clinical questions and drug- related knowledge, respectively. Finally, the dataset includes 99K samples derived from the UMLS medical knowledge graph, covering both entity descriptions and inter-entity relationships. This component particularly useful for aligning the model with structured biomedical ontologies.
Together, these seven resources offer diverse and comprehensive instruction set D,, enabling the model to generalize across conversational, inferential, and knowledge-hased medical tasks. More detailed information can be found in the (Wu et al. 2024)
B
Limitations
While SENATOR demonstrates promising results in identifying and repairing knowledge deficiencies within I.MMs, several limitations remain. First, our framework relies on an external human-curated knowledge graph (KG) to simulate realistic environment in which the model can perform structured
25

---

exploration. This setup enables the LLM to iteratively discover and repair its knowledge gaps through self-improvemmnt. However, such reliance on high-quality, domain -specifi KG may limit the framework's is applicability in settings where such structured resources are incomplete or unavailable. In future work, we plan explore ways to relax this dependency, such as constructing approximate KGs automatically from textual corpora using perieval-augmentee methods to complement structural guidance.
Second, while the structural entropy-guided exploration effectively identifies knowledge deficiencies, the process of synthesizing data repair these deficiencies can be further improved. The quality of synthetic data plays a crucial role in downstream model performance. However, this paper places greater emphasis on detecting and targeting knowledge gaps rather than exhaustively optimizing the data generation process. In our current implementation, we adopt prompt-based synthesis strategies for simplicity and reliability. In future work, we aim to incorporate more advanced techniques such as instruction-tuned generation, controllable sampling to enhance the relevance, diversity, and factuality of the synthesized data.
C Broader Impacts
Our work on the SENATOR framework for detecting and repairing knowledge deficiencies in large language models through targeted synthetic data generation has both promising benefits and potential risks for society.
Positive Impacts
Improved Reliability in High-Stakes Domains: By systematically identifying and closing knowledge gaps, SENATOR can make LLMs more accurate and trustworthy in domains such as medicine, law, and scientific research, where factual precision critical for patient care, legal reasoning, and scientific discovery.
Democratization of Domain-Adapted Models: Synthetic data alleviates the dependence expensive, expert-annotated corpora, enabling smaller organizations, research labs, and underserved communities to fine-tune powerful LLMs for specialized tasks without prohibitive annotation costs.
Rapid Adaptation to Emerging Knowledge: In fast-moving fields (e.g., novel pathogens, new regulations), synthetic data guided by up-to-date knowledge graphs help models stay current, supporting timely decision-making and dissemination of accurate information.
Negative Impacts
Bias Amplification and Inaccuracy: If the underlying knowledge graph pretraining data contain biases or errors, synthetic data may inadvertently reinforce these issues. Models improved on such data could perpetuate harmful stereotypes or spread misinformation.
Misuse for Misinformation: exploited to create convincingly false or misleading domain-specific content (e.g.. fraudulent medical advice or fabricated legal precedents), posing risks to public trust and safety.
High-quality synthetic data generation techniques could be
Overreliance on Synthetic Data: An overconfidence in models fine-tume ine-tuned primarily on synthetic data might obscure residual blind spots, leading users to place undue trust in automated systems without appropriate human oversight.
Privacy and Intellectual Property Concerns: knowledge graphs incorporate sensitive or proprietary information, there is potential for synthetic data to leak or replicate protected content, raising ethical and legal implications.
D Resource Requirement
We use 8 NVIDIA A100-40G GPUs to SFT Llama-3-8B and Qwen2-7B, and leverage 1-2 NVIDIA A 1100-00G GPUs for all the inference experiments.
Taking Qwen2-7B as an example, when using synthetic data to SFT Qwen2-7B for knowledge repair, the training time is about 30h on 8 NVIDIA A100-40G GPUs, and total of 3 epochs are performed.
26

---

The inference time such as synthetic data generation stage and evaluation stage, measured in seconds per sample, is calculated on an NVIDIA A GPU with vllm acceleration (e.g. Qwen2-7B model, which demands at least two A 100 GPUs for deployment)
27


Reviews:
[{"review_id": "QLx9TY0t0C", "reviewer": "Reviewer_7gNw", "summary": "This paper presents SENATOR to enhance the domain-specific knowledge capabilities of large language models. As existing synthetic data augmentation approaches often fail to target actual knowledge gaps, the authors propose using a Structural Entropy metric to quantify uncertainty along knowledge graph paths. This measure is combined with Monte Carlo Tree Search (MCTS) to guide the exploration of knowledge-deficient areas within the model. By generating targeted synthetic data based on these insights, SENATOR enables fine-tuning that systematically improves the model's performance in knowledge-intensive domains. Experiments on LLaMA-3 and Qwen2 across several benchmarks demonstrate the effectiveness of the approach in identifying and addressing knowledge gaps.", "questions": "1. The term “Agent” in Structural Entropy Guided Agent is somewhat confusing. Since it essentially refers to a standalone LLM that generates QA samples based on knowledge paths—without invoking external tools or requiring training—it might be clearer to refer to it as a generator or similar.\n2. The learning rate in Table A2 appears unusual—how was it selected? Also, why is the batch size set to 1?", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. The problem of expanding the knowledge boundaries of large language models is important.\n2. The motivation and proposed methodology are well-grounded and make intuitive sense.\n3. The experimental results support the effectiveness of the approach.\n\nWeaknesses\n1. While the overall methodology is clearly presented, several important implementation details are missing. Most notably, the paper does not explain how to determine whether a piece of knowledge belongs to the model's existing knowledge or constitutes new information—i.e., the criteria or thresholds for defining knowledge boundaries are not described (unless I overlooked it). Moreover, there is no experimental validation of the hyperparameter choices or their sensitivity. Other points also lack clarification, such as which model is used to generate QA samples based on the knowledge paths.\n2. The use of supervised fine-tuning (SFT) for knowledge injection is somewhat unconventional. Typically, continued pretraining (CPT) is used for injecting knowledge, while SFT is more often employed for knowledge elicitation or aligning with instruction-following formats. The performance gain observed through SFT-based injection may partly result from the stylistic similarity between the generated QA data and the instruction tuning data, or even potential data leakage due to overlap with the test format. This point needs further clarification.\n3. The experimental evaluation is not sufficiently comprehensive: (1) There is no comparison with related structure-aware knowledge injection methods. (2) The ablation study is insufficient, especially regarding the impact of knowledge boundary decisions mentioned above. (3) Efficiency analysis is lacking—does the Monte Carlo Tree Search introduce a significant computational overhead?", "was_revised": true, "final_justification": "The author's rebuttal has addressed my major concerns, so I decided to raise my score to 4.", "rebuttal": "We appreciate the reviewer's thorough review and insightful comments, which have helped us identify areas for improvement. We're encouraged that the reviewer recognizes the importance of our problem, the soundness of our motivation and methodology, and the effectiveness of our experimental results. Below, we address each point raised.\n\n**W1: Missing Implementation Details**\n- **Determining Knowledge Boundaries** (Criteria and Thresholds): Our method defines knowledge boundaries by identifying areas where the LLM exhibits the highest uncertainty. To select these highly deficient paths for data synthesis, we employ a thresholding mechanism. This is defined in the `collect_paths_above_threshold` function within our supplementary material's `kg_mcts.py`  file. Here, we select knowledge paths whose cumulative SE reward exceeds a predefined threshold (e.g., paths with a total SE reward `m>20`). \n- **Model Used for QA Sample Generation**: To clarify, for each base model used in our experiments, the **entire SENATOR pipeline**, including both the knowledge deficiency detection phase (where it acts as the \"Agent\") and the subsequent data synthesis phase, is executed using that **same base model**. This adheres to a **self-improvement paradigm**. For instance, when we evaluate Qwen2, Qwen2 itself explores the KG, and then Qwen2 also generates the QA samples based on the identified deficient paths. The same principle applies when Llama-3 is the base model.\n\n- **Hyperparameter Sensitivity**: We did not perform deliberate hyper-parameter tuning in our initial submission. Under fixed hyper-parameters and random seeds, the model already surpasses all baselines. Thank you for the suggestion, and we will analyze this further in the revised manuscript.\n\n**W2: SFT for Knowledge Injection and Potential Data Leakage** \n- **Justifying SFT for Targeted Knowledge Injection**: While CPT is indeed commonly used for broad knowledge ingestion, our SENATOR framework specifically focuses on **highly targeted and precise knowledge injection** to remedy identified deficiencies. Our synthetic data is generated as simple, atomic QA pairs, as clearly illustrated in Figure 6. This format consists of a single question and its corresponding correct answer. This is fundamentally different from the complex, multi-turn dialogues or intricate instruction-following formats often seen in general instruction tuning. The simplicity and specificity of our QA pairs make SFT an exceptionally effective and precise method for directly injecting these identified, discrete factual knowledge points and relationships into the model. The primary goal here is not to broadly align the model with new conversational styles or instructions, but rather to teach it specific facts and relational knowledge that it currently lacks.\n- **Addressing Stylistic Similarity and Data Leakage Concerns**: \n    -   Stylistic Similarity: Our ablation study in Table 1 directly addresses this. When we fine-tuned our models solely with general domain instruction data (the \"w/ instruction tuning\" setting), we saw **only marginal performance improvements, and in some cases, even a slight drop**, compared to the base models. It's worth noting that this instruction tuning step converts the original medical training set into instruction-style prompts. This not only boosts the model's ability to follow instructions but also reinforces some of its domain-specific medical knowledge. However, our results clearly suggest that simply aligning the model's style or improving its general instruction-following doesn't inherently lead to significant factual knowledge gains in our specialized medical domain. The **substantial performance uplift consistently appears only when we incorporate our SENATOR-synthesized data** (the \"w/ synthetic data + IT\" setting). This definitively shows that the performance improvements come from fixing the model's actual knowledge deficiencies with our targeted synthetic data, not just from stylistic alignment.\n     - Data Leakage: We have meticulously addressed the possibility of data leakage. As illustrated in Figure 8, our test data formats (e.g., MedQA's multiple-choice questions) are structurally distinct from the simple QA pair format of our synthetic data (Figure 6). This significant structural dissimilarity inherently reduces the likelihood that performance gains are due to format overlap. More critically, we have diligently cross-referenced our entire synthetic dataset with all test datasets and have not found any identical QA samples. This ensures that our reported evaluation results genuinely reflect the model's improved knowledge and reasoning abilities, rather than simple memorization or direct data leakage. \n\n**W3: Insufficient Experimental Evaluation & Efficiency Analysis.**\n- **Comparison with Structure-Aware Knowledge Injection**: Structure-aware knowledge injection integrates external knowledge into models while preserving its structural relationships. Our core innovation is enabling an LLM to self-explore its knowledge gaps within KGs. Our knowledge injection (SFT) phase, which uses data from detected deficiencies, serves primarily to validate our deficiency detection via SE+MCTS.\n- **Sufficiency of Ablation Study**. we introduce additional baselines representing alternative knowledge-gap targeting strategies:\n     -  Random Walk: A baseline that explores the KG without any guidance, simulating unguided traversal through entities and relations.\n     - Prob-Prob [1,2]: A sequence-based baseline using intrinsic LLM confidence scores. Specifically, it uses the product of token probabilities to estimate model certainty.\n     - Instruction-tuning:  This corresponds to the \"w/ instruction tuning\" setting in our paper. \n\n  [1] Teaching Large Language Models to Express Knowledge Boundary from Their Own Signals\n\n  [2] Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent\n\n| Methods              | MedQA | MedMCQA | PubMedQA | Avg.  |\n| -------------------- | ----- | ------- | -------- | ----- |\n| Llama-3-8B           | 55.54 | 52.21   | 54.80    | 54.18 |\n| - Random Walk        | 54.29 | 50.40   | 55.20    | 53.30 |\n| - Prob-Prob          | 56.12 | 51.35   | 59.40    | 55.62 |\n| - Instruction-tuning | 54.36 | 50.08   | 56.60    | 53.68 |\n| - **SENATOR**        | 58.29 | 53.60   | 64.80    | 58.90 |\n| Qwen2-7B            | 54.67 | 53.41   | 64.60    | 57.56 |\n| - Random Walk        | 53.48 | 53.77   | 62.40    | 56.55 |\n| - Prob-Prob          | 59.12 | 59.91   | 61.60    | 60.21 |\n| - Instruction-tuning | 59.07 | 59.77   | 61.20    | 60.01 |\n| - **SENATOR**        | 59.70 | 60.70   | 63.20    | 61.20 |\n\nAs shown in the table: SENATOR consistently outperforms both the \"Random Walk\" and \"Prob-Prob\" baselines for both Qwen2-7B and Llama-3-8B, demonstrating the superior effectiveness of our SE-guided MCTS for knowledge deficiency detection.\n- **Efficiency Analysis of MCTS**: Assuming a fixed number of simulations $N$ (e.g., $N = 100$ in our setup), and a knowledge path length (tree depth) of $H$ (our knowledge paths have a length of 5, implying $H = 4$ steps from the root to a terminal node), the size of the MCTS tree is influenced by the average number of neighbor nodes (relations) for an entity, which we denote as $M$. In general, the number of nodes visited is proportional to $M \\times H$. Therefore, the time complexity of the MCTS component can be approximated as: $ \\mathcal{O}(N \\times M \\times H) $.In our specific setting, this translates to: $\\mathcal{O}(100 \\times M \\times 4) = \\mathcal{O}(400M)$ (since this is inference rather than training, the actual speed is on the order of seconds). \n\n**Q1: \"Agent\" Terminology**: We appreciate the suggestion regarding the \"Agent\" terminology. Our choice of \"Agent\" is deliberate and central to emphasizing the active, exploratory role of the LLM in the knowledge deficiency detection phase. The SENATOR framework is composed of two distinct parts:\n1. Knowledge Deficiency Detection: In this crucial first stage, the LLM acts as an agent within the Knowledge Graph (KG) environment. Guided by Monte Carlo Tree Search (MCTS) and our proposed Structural Entropy (SE) reward, it actively navigates and makes strategic decisions to explore the KG, specifically pinpointing knowledge paths where the model exhibits high uncertainty. This involves dynamic decision-making and navigation within a structured knowledge space.\n2. Knowledge Synthesis and Repair: The second stage, while employing more conventional methods like Supervised Fine-Tuning (SFT) or Continued Pre-Training (CPT) for model repair, directly leverages the high-quality, targeted knowledge paths discovered by the LLM as an agent in the first stage. Thus, the \"Agent\" terminology specifically refers to the LLM's proactive, decision-making role in exploring and identifying deficiencies within the KG, which is the core novelty of SENATOR. We believe this distinction is vital to understanding our core contribution.\n\n**Q2: Learning Rate and Batch Size**. We sincerely apologize for the misunderstanding caused by the batch size notation in Table A2. The value \"1\" in the table indeed refers to the per-GPU batch size. As stated in the Resource Requirement section, we conducted our experiments using 8 NVIDIA A100-40G GPUs. Therefore, the effective global batch size was 64. We regret this imprecision and will explicitly clarify this in the revised Table A2 and its accompanying text to avoid any confusion. Regarding the learning rate, we followed the setting used in prior work for similar fine-tuning tasks, rather than performing explicit hyperparameter tuning. For consistency and fairness, we used the same set of hyperparameters across all models and settings during the SFT phase, without optimizing them specifically for SENATOR.\n\nWe hope these clarifications address your concerns and look forward to further in-depth discussions."}, {"review_id": "iqZD793s9c", "reviewer": "Reviewer_ugoP", "summary": "This paper proposes SENATOR, a framework for identifying and repairing knowledge deficiencies in large language models (LLMs), especially in knowledge-intensive domains like medicine. The approach guides an agent to explore a domain-specific knowledge graph using structural entropy as a reward signal within Monte Carlo Tree Search (MCTS), identifying uncertain or deficient knowledge areas. Synthetic data is then generated, targeted at these deficiencies, and used to fine-tune LLMs. The method is evaluated on medical LLMs and general LLMs, with extensive experiments on several medical benchmarks, demonstrating consistent improvements, with targeted synthetic data leading to more efficient and significant gains than further general instruction tuning.", "questions": "1. The claim of generality is strong, but empirical evidence is limited to SPOKE and biomedical QA. How does SENATOR perform in other domains beyond medicine? Would the approach/structural entropy work if the domain knowledge graph is incomplete or noisy? \n\n2. Could you quantitatively analyze the quality of your synthetic data in more depth (e.g., rejection rate, hallucination proportion, downstream impact of flagged errors versus accepted samples)?", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n1. The use of structural entropy as a reward signal within MCTS for guiding exploration of a large-scale knowledge graph provides a concrete, well-motivated metric for quantifying model uncertainty and targeting data synthesis. \n\n2. The framework is evaluated on multiple relevant medical QA benchmarks (MedQA, PubMedQA, GPQA, MedMCQA, etc.), comparing general LLMs (Llama-3, Qwen2) with and without SENATOR-based augmentation.\n\n3. Demonstrated ability to achieve performance improvements with significantly less synthetic data compared to the massive scale needed for prior SFT approaches, showcasing better efficiency.\n\nWeaknesses\n\n1. While Table 1 in Section 4.2 provides a comparison to medical LLMs and instruction-tuned baselines, the evaluation omits other advanced approaches for knowledge gap detection and model repair (e.g., retrieval-augmented tuning, knowledge editing, or curriculum learning approaches). \n\n2. Synthetic data approaches that are not driven by knowledge graphs or structural entropy should be more deeply compared—ideally with numbers—instead of only qualitatively contrasted. The lack of a direct comparison to alternative knowledge-gap targeting strategies leaves open questions about the true gains from the structural entropy-guided selection.\n\n3. Experiments are focused solely on the biomedical domain using SPOKE as the knowledge graph. It remains unclear how easily SENATOR can be ported to other domains where the knowledge structure may differ, or where KGs are less complete. The entire framework depends on the existence of a comprehensive, high-quality knowledge graph for the target domain.", "was_revised": true, "final_justification": "I sincerely appreciate the new controlled experiment on the same Llama-3-8B base model. I also acknowledge the authors' addition of a Retrieval-Augmented Generation (RAG) baseline in response to my feedback.\n\nHowever, my primary reservation about the breadth of the experimental evaluation persists. While the inclusion of a RAG baseline is a step in the right direction, the field of knowledge injection and enhancement is mature, with a wide variety of established techniques. For instance, the RAG implementation appears to be a standard setup, and more advanced RAG methods or alternative synthetic data generation strategies could provide stronger points of comparison.\n\nTo robustly demonstrate the superiority of the proposed framework, a more comprehensive comparison against a wider set of these mature knowledge enhancement methods is needed. As the current evaluation does not yet provide this, the significance of the contribution remains unclear to me. Therefore, I will maintain my original rating.", "rebuttal": "We appreciate the reviewer's comprehensive summary and insightful feedback. We are encouraged that the reviewer recognizes the strengths of our work, including the novel use of structural entropy within MCTS, the evaluation on relevant medical benchmarks, and the demonstrated efficiency of SENATOR. We address each point below.\n\n**W1&2: Evaluation Scope and Comparison to Alternative Knowledge-Gap Targeting Strategies**. \n\nOur work takes a novel stance by defining knowledge boundaries at a sub-graph level, which distinguishes us from prior methods that primarily focus on text-based uncertainty or individual triplets. For instance, while methods like LAMA's Language Models as Knowledge Bases? explore triplet-level confidence using self-information (e.g.,  $I(u, \\rho, v) = -\\log_2 P(v \\mid u, \\rho)$ ), SENATOR extends this by considering the collective uncertainty of an entity and its neighboring relations: $d_u = \\sum_{v \\in \\mathcal{N}(u)} I(u, \\rho, v)$ , and further aggregates this over entire knowledge paths via Structural Entropy. This enables us to identify systemic knowledge deficiencies embedded within the KG's topology.\nTo validate the effectiveness of our SE-guided knowledge boundary identification, we introduce additional baselines representing alternative knowledge-gap targeting strategies:\n- Random Walk: A baseline that explores the KG without guidance, simulating unguided sampling.\n- Prob-Prob [1,2]: A sequence-based baseline using intrinsic LLM confidence scores. Specifically, it uses the product of token probabilities to estimate model certainty.\n- Instruction-tuning:  This corresponds to the \"w/ instruction tuning\" setting in our paper, where the original medical training data (MedQA,MedMCQA,PubMedQA) is reformatted into instruction-style prompts to improve the model’s general instruction-following ability and domain alignment.\n\n[1] Teaching Large Language Models to Express Knowledge Boundary from Their Own Signals\n\n[2] Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent\n\nThe table below presents these new quantitative comparisons:\n| Methods              | MedQA | MedMCQA | PubMedQA | Avg.  |\n| -------------------- | ----- | ------- | -------- | ----- |\n| Llama-3-8B           | 55.54 | 52.21   | 54.80    | 54.18 |\n| - Random Walk        | 54.29 | 50.40   | 55.20    | 53.30 |\n| - Prob-Prob          | 56.12 | 51.35   | 59.40    | 55.62 |\n| - Instruction-tuning | 54.36 | 50.08   | 56.60    | 53.68 |\n| - **SENATOR**        | 58.29 | 53.60   | 64.80    | 58.90 |\n| Qwen2-7B            | 54.67 | 53.41   | 64.60    | 57.56 |\n| - Random Walk        | 53.48 | 53.77   | 62.40    | 56.55 |\n| - Prob-Prob          | 59.12 | 59.91   | 61.60    | 60.21 |\n| - Instruction-tuning | 59.07 | 59.77   | 61.20    | 60.01 |\n| - **SENATOR**        | 59.70 | 60.70   | 63.20    | 61.20 |\n\nAs shown in the table: SENATOR consistently outperforms both the \"Random Walk\" and \"Prob-Prob\" baselines for both Qwen2-7B and Llama-3-8B, demonstrating the superior effectiveness of our SE-guided MCTS for knowledge deficiency detection.\n\n**W3 and Q1: Domain Generality and KG Completeness**. \n- **Domain Generality:** Our study focuses on the **medical domain** due to its high demand for reliability, LLMs' persistent challenges with precise factual recall in this specialized field, and the high cost of data annotation. This makes targeted knowledge remediation via SENATOR exceptionally valuable. Crucially, SENATOR's core principles are **domain-agnostic**. Our methodology operates on KG structural properties and LLM path uncertainty, not domain-specific semantics. Any domain representable by a KG can theoretically leverage SENATOR, and its adaptability is shown across Llama-3 and Qwen2.\n- **Regarding Incomplete and Noisy KGs**: SENATOR leverages the existing KG to find LLM uncertainties within represented knowledge. Our MCTS explores existing paths; it doesn't infer missing parts. Thus, SENATOR effectively identifies deficiencies even with incomplete KGs. And noise issue is beyond the scope of SENATOR.\n\n**Q2: Quantitative Analysis of Synthetic Data Quality.**\n\nAs detailed in Appendix A.4 \"Data Filtering,\" we have previously analyzed the quality of our synthetic data. To gain deeper insights into the impact of synthetic data quality on knowledge injection performance, we further analyzed the performance of Llama-3 and Qwen2 when trained on unfiltered, lower-quality samples.\nSENATOR employs a multi-tiered evaluation mechanism (including checks for format consistency, logical coherence, and hallucination avoidance) to ensure high-quality synthetic data for knowledge injection. The table below demonstrates the downstream impact of unfiltered synthetic data (i.e., data that contains potential errors or inconsistencies) compared to our carefully filtered data:\n| **Model**               | **MedQA** | **MedMCQA** | **PubMedQA** | **AVG** |\n| ----------------------- | --------- | ----------- | ------------ | ------- |\n| Llama3-8B               | 55.54     | 52.21       | 54.8         | 54.18   |\n| - w/ instruction tuning | 54.36     | 50.08       | 56.6         | 53.68   |\n| - w/o filtering         | 56.25     | 52.41       | 61.0         | 56.55   |\n| - w/ **SENATOR**            | 58.29     | 53.60       | 64.8         | 58.90   |\n| Qwen2-7B                | 54.67     | 53.41       | 64.6         | 57.56   |\n| - w/ instruction tuning | 59.07     | 59.77       | 61.2         | 60.01   |\n| - w/o filtering         | 58.67     | 59.51       | 62.4         | 60.19   |\n| - w/ **SENATOR**            | 59.70     | 60.70       | 63.2         | 61.20   |\n\nAs shown, using synthetic data without filtering (\"w/o filtering\" rows) leads to a noticeable degradation in performance compared to using our filtered data (\"w/ SENATOR\" rows), and in some cases, even performs worse than the model trained without any synthetic data at all (only \"w/ instruction tuning\"). This quantitatively highlights the critical role of our data filtering module in ensuring the quality and positive impact of the synthesized knowledge.\n\nWe hope these clarifications address your concerns and highlight the unique contributions and practical effectiveness of SENATOR. We're open to further discussion if you have any additional questions."}, {"review_id": "UavP9lY9Za", "reviewer": "Reviewer_6gHM", "summary": "Authors introduce SENATOR framework, which designed to detect and repair knowledge deficiencies in LLMs by leveraging structural entropy (SE) as a reward signal to identify uncertain regions within a knowledge graph. They use MCTS to explore these high-uncertain paths inside KG, and then generate targeted QA data along maximum-SE trajectories, which are later used for SFT on Qwen2-7B and Llama-3-8B as base models to address these identified knowledge gaps. Experiments on three medical benchmarks (MedQA, MedMCQA, PubMedQA) and two biology/science datasets (GPQA-Genetics, GPQA-Molecular Biology) shows SENATOR yields accuracy improvements over the respective base models after fine-tuning.", "questions": "1. If the authors decide to use the Llama-2 series as base-LLM for more fair comparisons during SFT, could they also include evaluations on other recent medical LLMs (e.g., BioMistral, Hippocrates, and Meditron) together with some closed source model evaluation (e.g, gpt-4o, gpt-4.1, claude) to provide a more comprehensive results?\n\n2. Paper refers to the approach as \"agent-based\" but there is no explicit tool use or action-taking typically associated with agentic LLMs. Could  authors clarify why they introduce their method as agentic, and which aspects of agent behavior are being leveraged?\n\n3. How easy and straightforward to adapt the SE approach for identifying uncertain paths in knowledge graphs for other domains (e.g., tool use and math)? Are there any specific requirements or challenges when applying this framework to different KGs?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- Algorithmically, SENATOR is a novel and well-motivated framework; it uses of SE as an intrinsic reward for MCTS to explore uncertain knowledge graph paths, and the focus on these regions as self-aware learning which introduces promising directions for future research.\n\n- Authors conduct comprehensive ablation studies and present effective visualizations that help to interpret their findings and strengthen the validity of their discussions. Overall, this paper is well-structured and organized, making it easy for readers to follow both the methodology and experimental results.\n\n### Weaknesses\n\n- The major issue of the paper comes from the weakness of the selected baselines to illustrate the effectiveness of the proposed method. Specifically, models such as Chat-Doctor, Med-Alpaca, and PMC-LLaMA are built on Llama-2 series, while they used Llama-3 and Qwen2 as base models for fine-tuning using SENATOR framework. This discrepancy makes it difficult to fairly compare SENATOR trained models with other baselines since it is not possible to understand whether accuracy difference coming from the improved model versions or the proposed framework. Therefore, I strongly recommend that, authors either: (i) retrain all models on using the same base models (e.g., Llama-2) with baselines for a fair comparison, or (ii) include results as baselines with newer medical LLMs. In current version, the most reliable finding in terms of results is that SENATOR improves over its own base models (e.g., Llama-3-8B after fine-tuning), which is somewhat expected.\n\n- The scope of evaluation is limited to medical/biological domains which are correlated to each other and limits to understand SENATOR’s broader applicability and generalizability. It would be good to see results on other challenging domains, such as math (e.g., GSM8K, AIME2024) or function-calling (e.g., BFCL V3, ToolAlpaca).\n\n- While MCTS is a powerful method for exploration, it can be computationally expensive due to the exponential number of available trajectories to explore and the paper does not provide any latency or efficiency analysis. Reporting such computational costs as latency metric would strengthen the effectiveness of SENATOR.\n\n- The use of entropy-based uncertainty to detect knowledge gaps is central to the framework, but authors did not report any evaluations about the accuracy of this uncertainty selection itself which effects the reliability of the current SE-based selection. Is it possible to show the accuracy on the decision of uncertain path?", "was_revised": true, "final_justification": "Authors have engaged carefully with the rebuttal process, providing additional clarifications and experiments, which are truly appreciated.\n\nHowever, I still have concerns regarding the use of older model versions and certain ambiguities in the evaluations. For example, although GPT-4 is an older API version of the 4o models, the proposed approach underperforms it and achieves only comparable or on par results to Qwen-2.5-72B, which is a general-purpose base LLM. In addition, I believe uncertainty evaluations require a more detailed and in-depth comparison and discussion with other state-of-the-art certainty estimation methods. Finally, to support future research, the limitations of the SENATOR should be described more specifically and transparently. In its current form, the paper and rebuttal gives the impression that it performs perfectly across all scenarios, which risks appearing overstated.\n\nNevertheless, given the overall results and the authors’ constructive engagement, my overall view remains positive. I am maintaining my positive score while raising the significance rating from 3 to 4 for ACs decision.", "rebuttal": "We're truly grateful for your in-depth review and insightful feedback. It's encouraging to see you acknowledge the novelty and strong motivation behind our framework, along with our comprehensive ablation studies, effective visualizations, and the overall clear structure of our paper. We've addressed each of your points below.\n\n**W1 & Q1: Baseline Comparisons and Model Version Discrepancy**. \n\nThe reviewer pointed out a critical issue regarding our chosen baselines, noting a discrepancy between models built on Llama-2 (like Chat-Doctor, Med-Alpaca, PMC-LLaMA) and our use of Llama-3 and Qwen2 as base models. The reviewer rightly argue this makes fair comparisons difficult and strongly recommend either retraining all models on a consistent base (e.g., Llama-2) or including results from newer medical LLMs.\nWe appreciate your valuable suggestion regarding baseline selection, and we agree that comparisons on the same, up-to-date base models are ideal.\n- **Focus on Open-Source and Comparable Models**: Our work primarily aims to enhance open-source, white-box LLMs, and we prioritize comparing models of similar scale. This approach ensures our methods are reproducible and allows for a fair assessment of SENATOR's performance on currently prevalent base models.\n- **Newer Medical LLM Baselines Added**: To provide a more comprehensive comparison as you suggested, we've incorporated the performance of BioMistral-7B and Meditron-7B. These are leading open-source medical LLMs of comparable size to our base models (Llama-3-8B and Qwen2-7B). Here's an updated snapshot of the results:\n\n| Model                                     | MedQA | MedMCQA | PubMedQA |\n| ----------------------------------------- | ----- | ------- | -------- |\n| BioMistral-7B                             | 44.93 | 42.17   | 56.4     |\n| Meditron-7B                               | 30.40  | 31.22   | 61.6     |\n| Llama-3-8B w/ SENATOR   | 58.29 | 53.60    | 64.8     |\n| Qwen2-7B w/ SENATOR   | 59.70  | 60.70    | 63.2     |\n\nAs this table shows, SENATOR-enhanced models (Llama-3-8B and Qwen2-7B) consistently outperform these new medical LLM baselines on MedQA and MedMCQA, and remain highly competitive on PubMedQA. This further strengthens our claim about SENATOR's effectiveness in addressing knowledge deficiencies.\n- **Consideration for Closed-Source Models (GPT-4 etc.)**: Regarding the evaluation of closed-source models like GPT-4o, GPT-4.1, or Claude, we chose not to include them for two main reasons:\n  - **Methodological Constraint**: SENATOR's core mechanism relies on an LLM's ability to **self-explore its knowledge deficiencies** by measuring its intrinsic uncertainty (via Structural Entropy) during Monte Carlo Tree Search within a Knowledge Graph. This process requires fine-grained access to the model's internal probabilities and outputs, which isn't feasible with black-box, API-only models like GPT-4 or Claude. We simply can't reliably get the necessary uncertainty signals from these models to guide our deficiency detection.\n  - **Research Focus**: Our work primarily focuses on enhancing open-source, white-box LLMs, promoting reproducibility and accessibility within the research community.\nSo, while we appreciate the suggestion for broader comparisons, our current evaluation scope aligns with SENATOR's nature and technical requirements.\n\n**W2 & Q3: Scope of Evaluation, Generalizability, and KG Applicability to Other Domains**. \n\nOur study focuses on the medical domain due to its high demand for reliability, LLMs' persistent challenges with precise factual recall in this specialized field, and the high cost of data annotation. This makes targeted knowledge remediation via SENATOR exceptionally valuable.\n\n- **Domain Generalizability**: Crucially, SENATOR's core principles are **domain-agnostic**. Our methodology operates on KG structural properties and LLM path uncertainty, not domain-specific semantics. Any domain representable by a KG can theoretically leverage SENATOR, and its adaptability is shown across Llama-3 and Qwen2.\n- **Suitability for Knowledge-Intensive Domains**: It's important to emphasize that SENATOR is tailored for knowledge-intensive tasks. This design is based on a core assumption: LLMs act as implicit knowledge bases, and our goal is to uncover and fill their factual knowledge deficiencies. For domains like mathematical reasoning tasks (e.g., GSM8K, AIME2024) or tool learning/function-calling (e.g., BFCL V3, ToolAlpaca), SENATOR might not be directly applicable for two key reasons:\n     1. **Lack of Suitable Knowledge Graphs**: These domains often lack structured, KG-style representations. SENATOR relies on semi-structured KGs as an agent for the LLM's internal knowledge space and as a source for synthetic data generation.\n    2. **Nature of the Task**: In mathematical reasoning or tool-use tasks, model failure often stems from missing algorithmic ability, procedural logic, or execution errors—not from missing factual knowledge. Therefore, high structural entropy in a KG (if one even exists) wouldn't necessarily map to meaningful deficiencies for these domains. The detected uncertainty wouldn't correspond to missing facts but rather to an LLM's inability to perform a reasoning step or execute an algorithm correctly.\n\n**W3: MCTS Computational Cost and Efficiency Analysis.**\n- Time Complexity:  Assuming a fixed number of simulations $N$ (e.g., $N = 100$ in our setup), and a knowledge path length (tree depth) of $H$ (our knowledge paths have a length of 5, implying $H = 4$ steps from the root to a terminal node), the size of the MCTS tree is influenced by the average number of neighbor nodes (relations) for an entity, which we denote as $M$. In general, the number of nodes visited is proportional to $M \\times H$. Therefore, the time complexity of the MCTS component can be approximated as: $ \\mathcal{O}(N \\times M \\times H) $.In our specific setting, this translates to: $\\mathcal{O}(100 \\times M \\times 4) = \\mathcal{O}(400M)$ (since this is inference rather than training, the actual speed is on the order of seconds). .\n\n**W4: Accuracy of Uncertainty Selection.** \n\nThank you for this insightful question. To assess whether our Structural Entropy (SE)-based selection truly identifies knowledge gaps, we've conducted an additional analysis to directly address this.\n- Validating Uncertainty Selection Accuracy: We tested Qwen2-7B on 5,000 synthetic samples generated by SENATOR (based on Qwen2) from high-uncertainty knowledge paths. These represent areas where the model was least confident.\n  - Results (Zero-Shot setting):\n       - Exact Match Score (EM): 0%. (EM measures the percentage of predictions that match the ground truth exactly.)\n       - F1 Score: 6.87%.  (The F1 Score is the harmonic mean of precision and recall.)\n  - Example Question: \"_What disease is similar to thyroid gland carcinoma, with symptom dysphonia and neck pain?_ Answer: _Head and neck cancer_.\"\n- Conclusion: The extremely low EM and F1 scores directly confirm that Qwen2-7B indeed lacked the knowledge required to answer these questions accurately prior to supervised fine-tuning. This quantitatively validates that our SE-driven selection mechanism effectively identifies genuine, intrinsic knowledge deficiency regions within the model.\n\n**Q2: Clarification on \"Agent-Based\" Terminology.**\n\nWe appreciate you raising this point about our use of \"agent-based\" terminology, given that our method doesn't involve explicit tool use often associated with agentic LLMs. We want to clarify why we refer to our approach as agentic and which aspects of agent behavior are leveraged within the SENATOR framework.\nSpecifically, the LLM acts as an agent navigating a KG environment:\n- Environment & State: The KG serves as the environment, with each entity representing a state.\n- Actions: At each entity, the agent chooses among connected relations to move to a new state (entity).\n- Reward Signal: The Structural Entropy (SE) associated with knowledge paths serves as the intrinsic reward signal for the agent.\nThis MCTS-based exploration, driven by SE as a reward signal, embodies the LLM's purposeful, adaptive decision-making and navigation within the knowledge space to identify its own uncertainties. This fundamentally differs from a traditional LLM acting merely as a data generator, which typically lacks such proactive exploration and decision-making capabilities. Therefore, we believe the term \"Agent\" accurately captures this active role the LLM plays within our framework.\n\nWe hope these clarifications address your concerns and highlight the unique contributions and practical effectiveness of SENATOR. We're open to further discussion if you have any additional questions."}, {"review_id": "Aj6NF2jZPC", "reviewer": "Reviewer_nx7C", "summary": "This paper presents SENATOR (Structural Entropy-guided Knowledge Navigator), a novel framework designed to detect and repair knowledge deficiencies in Large Language Models (LLMs), particularly in knowledge-intensive domains like medicine. The method leverages Structural Entropy (SE) as an intrinsic reward to guide Monte Carlo Tree Search (MCTS) over a knowledge graph (KG). High-SE paths, which signal areas of model uncertainty, are identified and used to generate targeted synthetic QA data, which is then used to fine-tune the model. Experiments on medical benchmarks (MedQA, PubMedQA, GPQA, etc.) show significant performance gains for models such as LLaMA-3 and Qwen2. The results indicate that SENATOR is effective in identifying model blind spots and augmenting domain-specific knowledge efficiently.", "questions": "1.Your method depends heavily on a high-quality KG. How would SENATOR perform in domains with incomplete or noisy KGs, or without KGs altogether? \n2.Why did you choose structural entropy specifically as the exploration reward? Did you compare it to simpler measures ? \n3.All your experiments are in the medical domain. Have you considered applying SENATOR to another domain (e.g., law, science)? If not, what barriers do you anticipate?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "strengths: The paper presents an innovative integration of structural entropy and Monte Carlo Tree Search (MCTS) to guide exploration in knowledge graphs, offering a principled and effective approach to uncover latent knowledge gaps in LLMs. By combining deficiency detection, targeted data synthesis, and fine-tuning into a closed-loop framework, the method demonstrates scalability and adaptability. Extensive experiments on challenging medical QA benchmarks confirm the practical effectiveness of the approach, while thorough analyses—including UMAP visualizations and synthetic data scaling studies—provide strong evidence that the generated data successfully extends beyond the original pretraining distribution. weakness: Despite its promising contributions, the framework exhibits several limitations. It relies heavily on the availability of high-quality, human-curated knowledge graphs, which may not exist in many domains, thereby limiting its broader applicability. The scope of evaluation is limited to the medical domain, leaving its generalizability to other fields unverified. Finally, the paper lacks comparisons with alternative knowledge graph exploration strategies, such as random walks or centrality-based heuristics, to fully contextualize the benefits of using structural entropy.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful review and insightful observations. Your feedback has been invaluable in helping us clarify and strengthen our contribution. We would like to address your concerns as follows:\n\n**W1 and Q1&3: Dependency on High-Quality KGs, Performance with Incomplete/Noisy KGs, and Domain Generalizability**\n\nHaving experienced the wave of knowledge engineering, many fields now have knowledge graphs, as mentioned in the survey \"Domain-specific Knowledge Graphs: A Survey.\"\n\n- **Without KGs Altogether**: SENATOR's core mechanism fundamentally relies on the KG serving as a structured external environment for the LLM to explore and discover its own knowledge deficiencies. The LLM acts as an \"agent\" navigating this map of interconnected concepts. Without a KG, the LLM would lack this organized landscape to systematically probe its knowledge boundaries and pinpoint its blind spots. Therefore, SENATOR, in its current form, cannot operate without an underlying Knowledge Graph. If the required knowledge truly doesn't exist in any structured form, detecting and acquiring it becomes a different, broader problem outside SENATOR's scope.\n- **With Incomplete or Noisy KGs**: SENATOR is designed to function effectively even with incomplete or noisy KGs.\n     -  Our Monte Carlo Tree Search (MCTS) operates by exploring existing knowledge paths within the provided KG. It doesn't attempt to infer or complete missing parts of the graph. Instead, it focuses on systematically navigating the available structure to find areas where the LLM's uncertainty (quantified by Structural Entropy) is high.\n     -  The incompleteness of the KG does not inherently hinder this process; it simply means SENATOR will identify deficiencies within the knowledge that is represented and accessible in the graph, rather than in entirely missing sub-domains.\n     -  While noise in the KG might introduce some irrelevant paths, our Structural Entropy (SE)-based reward mechanism, which quantifies LLM uncertainty, would naturally assign **lower rewards to paths the LLM already knows well** (even if noisy), thereby de-emphasizing them in the exploration process. This inherent filtering mechanism helps mitigate the impact of noise.\n- **Domain Generalizability (Law, Science, etc.)**: We intentionally focused our current study on the medical domain for a strategic reason: data annotation in specialized, vertical domains like medicine is significantly more challenging and resource-intensive compared to general domains. This makes a targeted data synthesis approach like SENATOR particularly valuable where high-quality, domain-specific data is scarce.\n  - While our empirical evidence is limited to medicine, the fundamental principles of SENATOR are designed to be **domain-agnostic**:\n       - Knowledge Graph Agnostic: SENATOR's methodology operates on the structural properties of a knowledge graph and the LLM's uncertainty over its paths, independent of domain-specific semantics.\n       - LLM Agnostic: We've demonstrated its effectiveness across different LLM architectures (Llama-3 and Qwen2).\n       - Uncertainty-Guided Exploration: The use of Structural Entropy for MCTS is a general approach to identify areas of high LLM uncertainty.\n  - Therefore, we anticipate strong potential for SENATOR's application in other knowledge-intensive vertical domains where structured factual information is prevalent, such as law, finance, or other scientific disciplines, provided a relevant knowledge graph is available.\n\n**W2 & Q2: Lack of Comparison with Alternative KG Exploration Strategies & Choice of Structural Entropy**\n\n- **Why Structural Entropy?** We chose Structural Entropy (SE) specifically as the exploration reward because it allows us to define and detect knowledge deficiencies at a sub-graph or path level, which is a unique and powerful aspect of our framework.\n  - Unlike simpler measures that might only consider the LLM's uncertainty on individual facts (e.g., a single triplet) or on isolated sequences, Structural Entropy quantifies the **aggregate uncertainty within a structured knowledge path**. It considers both the LLM's confidence in individual triplets and the topological organization of those triplets within the graph, providing a more comprehensive understanding of complex knowledge gaps. This allows us to target systemic deficiencies embedded within relational knowledge structures.\n- **Comparison to Simpler Measures**: To directly demonstrate the efficacy of our SE-guided selection, we have conducted quantitative comparisons against simpler knowledge-gap targeting strategies:\n  - Random Walk: A baseline that explores the KG without any guidance, simulating unguided traversal through entities and relations.\n  - Prob-Prob [1,2]: A sequence-based baseline using intrinsic LLM confidence scores. Specifically, it uses the product of token probabilities to estimate model certainty.\n  - Instruction-tuning:  This corresponds to the \"w/ instruction tuning\" setting in our paper, where the original medical training data (MedQA,MedMCQA,PubMedQA) is reformatted into instruction-style prompts to improve the model’s general instruction-following ability and domain alignment.\n\n  [1] Teaching Large Language Models to Express Knowledge Boundary from Their Own Signals\n\n  [2] Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent\n\nOur experimental results, presented in the table below, clearly show the advantage of SENATOR's SE-guided approach:\n| Methods              | MedQA | MedMCQA | PubMedQA | Avg.  |\n| -------------------- | ----- | ------- | -------- | ----- |\n| Llama-3-8B           | 55.54 | 52.21   | 54.80    | 54.18 |\n| - Random Walk        | 54.29 | 50.40   | 55.20    | 53.30 |\n| - Prob-Prob          | 56.12 | 51.35   | 59.40    | 55.62 |\n| - Instruction-tuning | 54.36 | 50.08   | 56.60    | 53.68 |\n| - **SENATOR**        | 58.29 | 53.60   | 64.80    | 58.90 |\n| Qwen2-7B             | 54.67 | 53.41   | 64.60    | 57.56 |\n| - Random Walk        | 53.48 | 53.77   | 62.40    | 56.55 |\n| - Prob-Prob          | 59.12 | 59.91   | 61.60    | 60.21 |\n| - Instruction-tuning | 59.07 | 59.77   | 61.20    | 60.01 |\n| - **SENATOR**        | 59.70 | 60.70   | 63.20    | 61.20 |\n\nAs evident, SENATOR consistently outperforms both the \"Random Walk\" (undirected exploration) and \"Prob-Prob\" (sequence-based uncertainty) baselines. This quantitatively demonstrates that our SE-guided MCTS effectively leverages the structural information of the KG to pinpoint true knowledge deficiencies, leading to superior performance gains compared to simpler or non-graph-aware uncertainty measures.\n\nWe hope these clarifications address your concerns and highlight the unique contributions and practical effectiveness of SENATOR. We're open to further discussion if you have any additional questions."}, {"review_id": "Pd2cE0PMWM", "reviewer": "Reviewer_8d9L", "summary": "This work presents SENATOR, a way to \"repair the knowledge deficiencies\" of LLMs with regard to certain knowledge graphs. Experiments demonstrate that doing this on one knowledge graph helps various QA datasets.", "questions": "please see above", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "- How does the employed KG SPOKE align to datasets that are not explicitly medical/biological, for example GPQA and MMLU?\n\n- What happens if a fitting KG is not available for a given task? To me the method makes the LLM familiar with what's in a knowledge graph, but what if the things we need are not present in any KG anyways?\n\n- Minor point: Are the evaluation datasets all multiple-choice? It would be nice to see if the structured KG helps unstructured open-ended generation tasks as well.\n\n- Would be nice to see some examples where this method \"corrects\" a previously incorrect answer, and whether there are knowledge graph snippets highly related to the question.\n\n- On a broader note, knowledge graphs seem to be increasingly less important/relevant when we think about knowledge of ML models, as much of the stuff models should encode are natural language texts and retrieval/autoregressive training provides a natural way to incorporate that. I wonder what are the authors' thoughts on this work's significance in a world when knowledge graphs are less prevalent? The authors are free to argue otherwise.\n\n- Some reference on KG+LLM, the more modern ones, could be discussed. For example, [1].\n\n[1] Hron, Jiri, et al. \"Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability.\" COLM.", "was_revised": true, "final_justification": "The authors provided a detailed response that addressed much of my review.", "rebuttal": "We sincerely appreciate your concise summary and insightful questions regarding SENATOR. Your points effectively highlight key aspects and potential areas for clarification in our work. We'll address each of your comments and questions below.\n\n**W1 & 2: KG Alignment and Performance with Incomplete/Absent KGs.**\n- **KG Alignment with Evaluation Datasets**: To clarify, for our experiments, we only utilized the **medical/biological subsets of GPQA and MMLU**. Our focus was specifically on evaluating SENATOR's effectiveness in enhancing LLMs' knowledge within the medical domain. Therefore, the questions from these benchmarks that fall outside of medical or biological topics were excluded from our evaluation. This ensures a consistent alignment between the domain of the SPOKE knowledge graph and the evaluation data used.\n- **Without KGs Altogether**: SENATOR's core mechanism fundamentally relies on the KG serving as a structured external environment for the LLM to explore and discover its own knowledge deficiencies. The LLM acts as an \"agent\" navigating this map of interconnected concepts. Without a KG, the LLM would lack this organized landscape to systematically probe its knowledge boundaries and pinpoint its blind spots. Therefore, **SENATOR, in its current form, cannot operate without an underlying KG**. If the required knowledge truly doesn't exist in any structured form (i.e., is not representable as a KG), then detecting and acquiring it becomes a different, broader problem, falling outside SENATOR's current scope.\n- **With Incomplete or Noisy KGs**: SENATOR is designed to function effectively even with incomplete KGs.\n     -  Our Monte Carlo Tree Search (MCTS) operates by exploring existing knowledge paths within the provided KG. It doesn't attempt to infer or complete missing parts of the graph. Instead, it focuses on systematically navigating the available structure to find areas where the LLM's uncertainty (quantified by Structural Entropy) is high.\n     -  The incompleteness of the KG does not inherently hinder this process; it simply means SENATOR will identify deficiencies within the knowledge that is represented and accessible in the graph, rather than in entirely missing sub-domains.\n     -  Noisy KGs are beyond the scope of SENATOR.\n\n**W3: Evaluation Dataset Format and Unstructured Generation Tasks**\n - Our synthetic data, as shown in Figure 6 of our paper, indeed consists of **unstructured, open-ended QA pairs**. These samples, generated by SENATOR, feature a question and a direct, free-form answer. This reflects the direct knowledge injection that SENATOR aims for—teaching the model to accurately respond to specific factual queries.\n- However, for our main evaluation, we primarily used multiple-choice benchmarks such as MedQA, MedMCQA, and PubMedQA. This decision was driven by the current landscape of mainstream evaluation in the medical LLM field, where multiple-choice formats are widely adopted and enable standardized, quantitative comparisons across models, as seen in works like HuatuoGPT-o1 [1].\n- While our direct evaluation is on multiple-choice tasks, the underlying training signal from SENATOR's synthetic data is designed to enhance the model's fundamental factual recall and understanding. This improved factual grounding is crucial for any downstream task, including open-ended generation. The ability to correctly provide a precise answer in an unstructured format (as learned from our synthetic data) directly supports improved performance in more complex, open-ended scenarios where factual accuracy is paramount. Though not directly evaluated for open-ended generation in this paper, we anticipate that the enhanced knowledge base provided by SENATOR would naturally translate to improved factual grounding in such tasks as well.\n\n[1] HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs.\n\n**W4: Case Studies for Correction Examples.**\n\nWe completely agree this would be a powerful demonstration. We are preparing to include additional case studies in the revised version of our paper. A case from MedQA is shown as follows:\n> A 4-week-old female newborn is brought to the physician because of increasing yellowing of her eyes and skin for 2 weeks. The mother has noticed that the girl's stools have become pale over the past week. She was breastfed since birth but her parents switched her to formula feeds recently after reading on the internet that breastfeeding could be the cause of her current symptoms. The patient was delivered vaginally at 38 weeks' gestation. Pregnancy and delivery were uncomplicated. She appears healthy. Vital signs are within normal limits. She is at the 50th percentile for length and at the 60th percentile for weight. Examination shows scleral icterus and jaundice. The liver is palpated 2 cm below the right costal margin. Cardiopulmonary examination shows no abnormalities. Neurologic examination shows no focal findings. Serum studies show: ......\nWhich of the following is the most likely diagnosis?\nA. Galactosemia\nB. Biliary atresia (`correct`)\nC. Crigler–Najjar syndrome\nD. Breast milk jaundice\n\nOur synthetic data (QA Pair):\n> Question: _Which disease is similar to bile duct disease, and presents symptom Obstructive Jaundice?_      Answer: _Biliary atresia_.\n\n> Evidence Path: \nDisease `<bile duct disease>` resembles disease `<biliary atresia>`,**Disease `<biliary atresia>` presents symptom `<Jaundice, Obstructive>`**,The symptom of `<Jaundice, Obstructive>` symptoms are presented in disease `<collecting duct carcinoma>` ……\n\n> The medical term `Jaundice` refers to the condition where an elevated level of bilirubin in the body causes the skin, sclera (the white part of the eye), and mucous membranes to turn yellow or green.\n\n\n**W5: Significance of KGs in LLM Knowledge Acquisition.**\n\nThe reviewer has raised a thought-provoking point regarding the evolving role of knowledge graphs (KGs) amidst the power of natural language texts, retrieval, and autoregressive training for LLM knowledge. We appreciate this perspective and have considered it deeply.\nThe reviewer's assumption is that large models can **autonomously and reliably capture structured knowledge from vast amounts of free-text corpora**. However, while state-of-the-art LLMs demonstrate impressive learning and generalization capabilities, their very nature as **probabilistic models inherently introduces uncertainty**. This means that when faced with massive volumes of knowledge, especially specialized or niche information, we **lack clear insight into their confidence in specific factual statements**. This fundamental uncertainty significantly **undermines model reliability**, particularly for applications in domains demanding high trustworthiness.\nThis is where **KGs crucially bridge the gap**. Our work isn't about simply having LLMs learn from KGs. Instead, it's about **leveraging KGs to systematically detect LLM knowledge deficiencies in a human-trustworthy manner, especially for complex, structured knowledge**. As the number of knowledge entities grows, the complexity of structured knowledge increases exponentially. Relying solely on question-answering (QA) formats to detect comprehensive knowledge gaps becomes impractical. A systematic detection method is essential, and this is precisely why a KG-based approach is indispensable.\nKGs provide a **verifiable, structured environment** that allows us to:\n1. **Systematically Probe Deficiencies**: Unlike unstructured text, KGs enable targeted, structured exploration (as with our SE-based MCTS) to pinpoint exactly what an LLM doesn't know.\n2. **Ensure Trustworthiness**: The explicit, symbolic nature of KGs allows for greater transparency and auditability, fostering human trust in the knowledge acquisition process.\n3. **Handle Complexity**: KGs are inherently designed to represent complex, relational knowledge in a way that is difficult for LLMs to consistently and reliably extract from raw text alone.\nFurthermore, the feasibility of KG-based approaches is high. There's already a lot of existing domain-specific KGs (e.g., as surveyed in \"Domain-specific Knowledge Graphs: A Survey\"), which can readily serve as the structured environment for our framework.\nTherefore, we believe KGs remain profoundly relevant, not as a replacement for textual knowledge or retrieval, but as essential tools that enable LLMs to achieve higher levels of factual precision, reliability, and systematic self-correction in specialized domains where human trust is paramount.\n\n**W6: Modern KG+LLM References.** \n\nThank you for the helpful suggestion and for pointing us to this recent work. We are glad to incorporate Hron et al. (COLM) into our discussion, as it provides valuable insights into the interplay between knowledge graphs and language models, particularly around hallucinations and their detectability.\nWe agree this line of work is highly relevant and complementary to SENATOR. While Hron et al. focus on training LLMs directly on KGs to mitigate hallucination, our work instead addresses the challenge of detecting and targeting knowledge deficiencies in pretrained LLMs by leveraging KG structure for guided data synthesis. We believe both approaches share the broader goal of enhancing factual reliability in LLMs, and we are happy to include a discussion in the revised version.\n\nWe hope above responses addresses all your comments and questions, further clarifying the significance and unique contributions of SENATOR."}]

Author rebuttal:

Meta-review: This paper introduces SENATOR, a framework designed to detect and repair knowledge deficiencies in LLMs, particularly in knowledge-intensive domains such as medicine. The core contribution is a self-improvement loop where an LLM uses a knowledge graph (KG) to find its own areas of uncertainty. The technical approach leverages Structural Entropy (SE) as a reward signal within a Monte Carlo Tree Search (MCTS) to guide the exploration of a domain-specific KG. This allows the model to identify knowledge paths where it is most uncertain. Based on these identified deficiencies, the framework generates targeted synthetic question-answering pairs, which is then used to fine-tune and "repair" the LLM. The work is relevant to the NeurIPS community, addressing the important challenge of factual accuracy and reliability in foundation models.

Initial ratings were primarily in the borderline category, with two "Borderline accept" (4) and one "Borderline reject" (3). Following an extensive rebuttal and discussion period, one reviewer raised their score from 3 to 4, and another maintained their positive rating while increasing their significance score. The final ratings are: 4, 4, 4, 4, 3. Confidence levels were generally high (3 or 4).

Reviewers acknowledged the novelty and strong motivation of the proposed method. The integration of Structural Entropy with MCTS to guide knowledge exploration was consistently highlighted as a key strength. Reviwers also pointed out that the paper structure is well-written and the clarity is good.

One initial concern, shared by multiple reviewers, was the weakness of the experimental baselines. The original comparisons were against models built on older architectures (e.g., Llama-2), making it difficult to isolate the performance gains of SENATOR from the improvements of the base model itself (Llama-3). The authors commendably addressed this during the rebuttal by conducting several new experiments. They added a strong baseline (UltraMedical) built on the same Llama-3-8B base model, which demonstrated SENATOR's superior performance. They also included a RAG baseline, showing that their knowledge injection method is more effective than on-the-fly retrieval for the tested tasks. These additions significantly strengthen the paper's empirical claims. Other concerns included the limited scope of evaluation (solely the medical domain), the lack of comparison to other knowledge injection paradigms like RAG, and the potentially confusing use of the term "agent."

The theoretical foundation of using Structural Entropy to quantify uncertainty in a structured knowledge space is sound and well-motivated. The application of MCTS for guided exploration is a logical and effective choice for navigating the vast search space of a large KG.

The authors have provided detailed hyperparameters and a description of their experimental setup. Code has also been provided in the supplementary material, suggesting that the work should be reproducible.

The paper addresses a problem of significance: improving the factual reliability of LLMs in high-stakes, knowledge-intensive domains. The proposed method of targeted self-repair is interesting and seems efficient, achieving significant performance gains with a fraction of the data used to train other specialized models. This "deficiency-oriented" approach to data synthesis could represent a more sustainable and efficient paradigm for adapting foundation models, making it a valuable contribution to the field.

Another potential concern is the experiments to be confined to the biomedical domain, leaving questions about generalizability. While the scope is still limited to one domain, the authors provide a reasonable justification for this focus and a clear argument for the method's domain-agnostic principles.

This paper is a borderline accept. The core idea is novel, well-motivated, and addresses an important problem. While the initial submission suffered from significant weaknesses in its experimental evaluation, the authors' exemplary engagement during the rebuttal period has addressed the most critical issues. The addition of rigorously controlled baselines provides convincing evidence for the framework's effectiveness. The authors have been responsive to feedback and have committed to making necessary revisions to improve the paper's clarity and impact. The work's potential for improving LLM reliability in specialized domains, combined with the authors' thorough response to criticism, pushes this paper over the acceptance threshold.
Decision: Accept (poster)