Title: Structural Entropy Guided Agent for Detecting and Repairing Knowledge Deficiencies in LLMs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Synthetic Data Generation

llm-agents

self-inprovement

MCTS
Abstract: Large language models (LLMs) have achieved unprecedented performance by leveraging vast pretraining corpora, yet their performance remains suboptimal in knowledge-intensive domains such as medicine and scientific research, where high factual precision is required.
While synthetic data provides a promising avenue for augmenting domain knowledge, 
existing methods frequently generate redundant samples that do not align with the model’s true knowledge gaps. 
To overcome this limitation, 
we propose a novel Structural Entropy-guided Knowledge Navigator (SENATOR) framework that addresses the intrinsic knowledge deficiencies of LLMs. 
Our approach employs the Structure Entropy (SE) metric to quantify uncertainty along knowledge graph paths and leverages Monte Carlo Tree Search (MCTS) to selectively explore regions where the model lacks domain-specific knowledge. 
Guided by these insights, the framework generates targeted synthetic data for supervised fine-tuning, enabling continuous self-improvement. 
Experimental results on LLaMA-3 and Qwen2 across multiple domain-specific benchmarks show that SENATOR effectively detects and repairs knowledge deficiencies, achieving notable performance improvements.


Peer reviews:
comments: SUMMARY:
This work presents SENATOR, a way to "repair the knowledge deficiencies" of LLMs with regard to certain knowledge graphs. Experiments demonstrate that doing this on one knowledge graph helps various QA datasets.

STRENGTHS AND WEAKNESSES:
- How does the employed KG SPOKE align to datasets that are not explicitly medical/biological, for example GPQA and MMLU?

- What happens if a fitting KG is not available for a given task? To me the method makes the LLM familiar with what's in a knowledge graph, but what if the things we need are not present in any KG anyways?

- Minor point: Are the evaluation datasets all multiple-choice? It would be nice to see if the structured KG helps unstructured open-ended generation tasks as well.

- Would be nice to see some examples where this method "corrects" a previously incorrect answer, and whether there are knowledge graph snippets highly related to the question.

- On a broader note, knowledge graphs seem to be increasingly less important/relevant when we think about knowledge of ML models, as much of the stuff models should encode are natural language texts and retrieval/autoregressive training provides a natural way to incorporate that. I wonder what are the authors' thoughts on this work's significance in a world when knowledge graphs are less prevalent? The authors are free to argue otherwise.

- Some reference on KG+LLM, the more modern ones, could be discussed. For example, [1].

[1] Hron, Jiri, et al. "Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability." COLM.

QUESTIONS:
please see above

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-21

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents SENATOR (Structural Entropy-guided Knowledge Navigator), a novel framework designed to detect and repair knowledge deficiencies in Large Language Models (LLMs), particularly in knowledge-intensive domains like medicine. The method leverages Structural Entropy (SE) as an intrinsic reward to guide Monte Carlo Tree Search (MCTS) over a knowledge graph (KG). High-SE paths, which signal areas of model uncertainty, are identified and used to generate targeted synthetic QA data, which is then used to fine-tune the model. Experiments on medical benchmarks (MedQA, PubMedQA, GPQA, etc.) show significant performance gains for models such as LLaMA-3 and Qwen2. The results indicate that SENATOR is effective in identifying model blind spots and augmenting domain-specific knowledge efficiently.\n\nSTRENGTHS AND WEAKNESSES:\nstrengths: The paper presents an innovative integration of structural entropy and Monte Carlo Tree Search (MCTS) to guide exploration in knowledge graphs, offering a principled and effective approach to uncover latent knowledge gaps in LLMs. By combining deficiency detection, targeted data synthesis, and fine-tuning into a closed-loop framework, the method demonstrates scalability and adaptability. Extensive experiments on challenging medical QA benchmarks confirm the practical effectiveness of the approach, while thorough analyses—including UMAP visualizations and synthetic data scaling studies—provide strong evidence that the generated data successfully extends beyond the original pretraining distribution. weakness: Despite its promising contributions, the framework exhibits several limitations. It relies heavily on the availability of high-quality, human-curated knowledge graphs, which may not exist in many domains, thereby limiting its broader applicability. The scope of evaluation is limited to the medical domain, leaving its generalizability to other fields unverified. Finally, the paper lacks comparisons with alternative knowledge graph exploration strategies, such as random walks or centrality-based heuristics, to fully contextualize the benefits of using structural entropy.\n\nQUESTIONS:\n1.Your method depends heavily on a high-quality KG. How would SENATOR perform in domains with incomplete or noisy KGs, or without KGs altogether? \n2.Why did you choose structural entropy specifically as the exploration reward? Did you compare it to simpler measures ? \n3.All your experiments are in the medical domain. Have you considered applying SENATOR to another domain (e.g., law, science)? If not, what barriers do you anticipate?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuthors introduce SENATOR framework, which designed to detect and repair knowledge deficiencies in LLMs by leveraging structural entropy (SE) as a reward signal to identify uncertain regions within a knowledge graph. They use MCTS to explore these high-uncertain paths inside KG, and then generate targeted QA data along maximum-SE trajectories, which are later used for SFT on Qwen2-7B and Llama-3-8B as base models to address these identified knowledge gaps. Experiments on three medical benchmarks (MedQA, MedMCQA, PubMedQA) and two biology/science datasets (GPQA-Genetics, GPQA-Molecular Biology) shows SENATOR yields accuracy improvements over the respective base models after fine-tuning.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- Algorithmically, SENATOR is a novel and well-motivated framework; it uses of SE as an intrinsic reward for MCTS to explore uncertain knowledge graph paths, and the focus on these regions as self-aware learning which introduces promising directions for future research.\n\n- Authors conduct comprehensive ablation studies and present effective visualizations that help to interpret their findings and strengthen the validity of their discussions. Overall, this paper is well-structured and organized, making it easy for readers to follow both the methodology and experimental results.\n\n### Weaknesses\n\n- The major issue of the paper comes from the weakness of the selected baselines to illustrate the effectiveness of the proposed method. Specifically, models such as Chat-Doctor, Med-Alpaca, and PMC-LLaMA are built on Llama-2 series, while they used Llama-3 and Qwen2 as base models for fine-tuning using SENATOR framework. This discrepancy makes it difficult to fairly compare SENATOR trained models with other baselines since it is not possible to understand whether accuracy difference coming from the improved model versions or the proposed framework. Therefore, I strongly recommend that, authors either: (i) retrain all models on using the same base models (e.g., Llama-2) with baselines for a fair comparison, or (ii) include results as baselines with newer medical LLMs. In current version, the most reliable finding in terms of results is that SENATOR improves over its own base models (e.g., Llama-3-8B after fine-tuning), which is somewhat expected.\n\n- The scope of evaluation is limited to medical/biological domains which are correlated to each other and limits to understand SENATOR’s broader applicability and generalizability. It would be good to see results on other challenging domains, such as math (e.g., GSM8K, AIME2024) or function-calling (e.g., BFCL V3, ToolAlpaca).\n\n- While MCTS is a powerful method for exploration, it can be computationally expensive due to the exponential number of available trajectories to explore and the paper does not provide any latency or efficiency analysis. Reporting such computational costs as latency metric would strengthen the effectiveness of SENATOR.\n\n- The use of entropy-based uncertainty to detect knowledge gaps is central to the framework, but authors did not report any evaluations about the accuracy of this uncertainty selection itself which effects the reliability of the current SE-based selection. Is it possible to show the accuracy on the decision of uncertain path?\n\nQUESTIONS:\n1. If the authors decide to use the Llama-2 series as base-LLM for more fair comparisons during SFT, could they also include evaluations on other recent medical LLMs (e.g., BioMistral, Hippocrates, and Meditron) together with some closed source model evaluation (e.g, gpt-4o, gpt-4.1, claude) to provide a more comprehensive results?\n\n2. Paper refers to the approach as \"agent-based\" but there is no explicit tool use or action-taking typically associated with agentic LLMs. Could  authors clarify why they introduce their method as agentic, and which aspects of agent behavior are being leveraged?\n\n3. How easy and straightforward to adapt the SE approach for identifying uncertain paths in knowledge graphs for other domains (e.g., tool use and math)? Are there any specific requirements or challenges when applying this framework to different KGs?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes SENATOR, a framework for identifying and repairing knowledge deficiencies in large language models (LLMs), especially in knowledge-intensive domains like medicine. The approach guides an agent to explore a domain-specific knowledge graph using structural entropy as a reward signal within Monte Carlo Tree Search (MCTS), identifying uncertain or deficient knowledge areas. Synthetic data is then generated, targeted at these deficiencies, and used to fine-tune LLMs. The method is evaluated on medical LLMs and general LLMs, with extensive experiments on several medical benchmarks, demonstrating consistent improvements, with targeted synthetic data leading to more efficient and significant gains than further general instruction tuning.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The use of structural entropy as a reward signal within MCTS for guiding exploration of a large-scale knowledge graph provides a concrete, well-motivated metric for quantifying model uncertainty and targeting data synthesis. \n\n2. The framework is evaluated on multiple relevant medical QA benchmarks (MedQA, PubMedQA, GPQA, MedMCQA, etc.), comparing general LLMs (Llama-3, Qwen2) with and without SENATOR-based augmentation.\n\n3. Demonstrated ability to achieve performance improvements with significantly less synthetic data compared to the massive scale needed for prior SFT approaches, showcasing better efficiency.\n\nWeaknesses\n\n1. While Table 1 in Section 4.2 provides a comparison to medical LLMs and instruction-tuned baselines, the evaluation omits other advanced approaches for knowledge gap detection and model repair (e.g., retrieval-augmented tuning, knowledge editing, or curriculum learning approaches). \n\n2. Synthetic data approaches that are not driven by knowledge graphs or structural entropy should be more deeply compared—ideally with numbers—instead of only qualitatively contrasted. The lack of a direct comparison to alternative knowledge-gap targeting strategies leaves open questions about the true gains from the structural entropy-guided selection.\n\n3. Experiments are focused solely on the biomedical domain using SPOKE as the knowledge graph. It remains unclear how easily SENATOR can be ported to other domains where the knowledge structure may differ, or where KGs are less complete. The entire framework depends on the existence of a comprehensive, high-quality knowledge graph for the target domain.\n\nQUESTIONS:\n1. The claim of generality is strong, but empirical evidence is limited to SPOKE and biomedical QA. How does SENATOR perform in other domains beyond medicine? Would the approach/structural entropy work if the domain knowledge graph is incomplete or noisy? \n\n2. Could you quantitatively analyze the quality of your synthetic data in more depth (e.g., rejection rate, hallucination proportion, downstream impact of flagged errors versus accepted samples)?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents SENATOR to enhance the domain-specific knowledge capabilities of large language models. As existing synthetic data augmentation approaches often fail to target actual knowledge gaps, the authors propose using a Structural Entropy metric to quantify uncertainty along knowledge graph paths. This measure is combined with Monte Carlo Tree Search (MCTS) to guide the exploration of knowledge-deficient areas within the model. By generating targeted synthetic data based on these insights, SENATOR enables fine-tuning that systematically improves the model's performance in knowledge-intensive domains. Experiments on LLaMA-3 and Qwen2 across several benchmarks demonstrate the effectiveness of the approach in identifying and addressing knowledge gaps.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The problem of expanding the knowledge boundaries of large language models is important.\n2. The motivation and proposed methodology are well-grounded and make intuitive sense.\n3. The experimental results support the effectiveness of the approach.\n\nWeaknesses\n1. While the overall methodology is clearly presented, several important implementation details are missing. Most notably, the paper does not explain how to determine whether a piece of knowledge belongs to the model's existing knowledge or constitutes new information—i.e., the criteria or thresholds for defining knowledge boundaries are not described (unless I overlooked it). Moreover, there is no experimental validation of the hyperparameter choices or their sensitivity. Other points also lack clarification, such as which model is used to generate QA samples based on the knowledge paths.\n2. The use of supervised fine-tuning (SFT) for knowledge injection is somewhat unconventional. Typically, continued pretraining (CPT) is used for injecting knowledge, while SFT is more often employed for knowledge elicitation or aligning with instruction-following formats. The performance gain observed through SFT-based injection may partly result from the stylistic similarity between the generated QA data and the instruction tuning data, or even potential data leakage due to overlap with the test format. This point needs further clarification.\n3. The experimental evaluation is not sufficiently comprehensive: (1) There is no comparison with related structure-aware knowledge injection methods. (2) The ablation study is insufficient, especially regarding the impact of knowledge boundary decisions mentioned above. (3) Efficiency analysis is lacking—does the Monte Carlo Tree Search introduce a significant computational overhead?\n\nQUESTIONS:\n1. The term “Agent” in Structural Entropy Guided Agent is somewhat confusing. Since it essentially refers to a standalone LLM that generates QA samples based on knowledge paths—without invoking external tools or requiring training—it might be clearer to refer to it as a generator or similar.\n2. The learning rate in Table A2 appears unusual—how was it selected? Also, why is the batch size set to 1?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}