Diagnosing and Remedying Knowledge Deficiencies in LLMs via Label-free Curricular Meaningful Learning
Authors: Kai Xiong
Xiao Ding
Yixin Cao
Li Du
Jiahao Ying
Yang Zhao
Bing Qin
Ting Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Deficiency Diagnosis
Data Synthesis
LLMs Reasoning
Primary area: foundation or frontier models, including LLMs
TL;DR: Diagnose the knowledge deficiencies of LLMs and remedy them with a novel approach.

Abstract: Large Language Models (LLMs) have demonstrated impressive generalization ability by learning from extensive unlabeled text. However, they still exhibit reasoning mistakes, which can affect their trustworthiness and reliability. Although users can interact with LLMs and provide diverse and comprehensive queries to expose the flaws of LLMs, obtaining sufficient and effective feedback is demanding. Furthermore, comprehensively evaluating LLMs with limited labeled samples is difficult. These make it a challenge to diagnose and remedy the deficiencies in LLMs through rich label-free user queries. To tackle this challenge and considersing that LLMs' reasoning mistakes often stem from knowledge deficiencies, we propose label-free curricular meaningful learning (LaMer), which first employs relative entropy to diagnose and quantify knowledge deficiencies of LLMs in a label-free setting. Then, LaMer adaptively synthesizes augmentation data based on deficiency severity and progressively remedies them with a curricular remedy strategy. Experiments show that LaMer effectively diagnoses and remedies knowledge deficiencies in LLMs, improving various LLMs across seven out-of-distribution (OOD) reasoning benchmarks, achieving comparable results to baselines with only 40% training data. LaMer even surpasses methods that rely on labeled data for deficiency diagnosis. In application, LaMer offers a diagnostic tool for efficient LLM development.

Published as a conference paper at ICLR 2026
DIAGNOSING AND REMEDYING KNOWLEDGE DEFI- CIENCIES IN LLMS VIA LABEL-FREE CURRICULAR MEANINGFUL LEARNING
Kai Xiong Xiao Ding Yixin Cao
Du Jiahao Ying
Yang Zhao
Bing Qin Ting Liu'
RRseearh Center for Social Computing and Interactive Robotics
Harbin Institute of Technology. Harbin, China
#Zhongguancun Laboratory, Beijing, China
3 Instiitue of Trustworthy Embodied AI, Fudan University, Shanghai, China
"Biijin Academy of Artificial Intelligence, Beijing, China
5 Singapore Management University, Singapore
[kxicng, xding, yangzhao, qinb, tliujeir iiiueir.hitt .nn..cn yxcacе fudan. en
ABSTRACT
Large Language Models (LLMs) have demonstrated impressive generalization abil- ity by learning from extensive unlabeled text. However, they still exhibit reasoning mistakes, which can affect their trustworthiness and reliability. Although users can interact with L.LSs and provide diverse and comprehensive queries to expose the flaws of L.MMs, obtaining sufficient and effective feedback is demanding. Further- more, comprchensively evaluating LLMs with limited labeled samples difficult. These make challenge to diagnose and remedy the deficiencies in LLMs through rich label-free user queries. To tackle this challenge and considersing that LLMs' reasoning mistakes often stem from knowledge deficiencies, free curricular meaningful learning (LaMer), which first employs relative entropy diagnose and quantify knowledge deficiencies of LLMs in a label-free setting. Then, LaMer adaptively synthesizes augmentation data based on deficiency severity and progressively remedies them with a curricular remedy strategy. Experiments show that LaMer effectively diagnoses and remedies knowledge deficiencies in L.MMs, improving various LLMs across seven -iitt--diitiiutti (OOD) reasoning benchmarks, achieving comparable results to baselines with only 40% training data. LaMer even surpasses methods that rely on labeled data for deficiency diagnosis. In application. LaMer offers diagnostic tool for efficient LLM development. The code is available at httpp:: httpps://qitaan //tthugbtttt/t////////// com/ Waste -eoastee-ood///oooo LaMer
propose label-
INTRODUCTION
Large language models (LLMs) have made significant advancements various fields recently (Kim et al. [2024] Li et al. 2024a). By implicitly mining and learning information from vast amounts of unlabeled text via language modeling. LLMs have demonstrated remarkable generalization abilities.
This enables them to answer various user queries across many applications such reasoning (Tang
al. 2024 Xiong al. 2024) and recommender systems (Lei et al. 2023 Wu at. 2024). However, despite their potential, LLMs still have limitations. Due to their statistical nature, LLMs occasionally make reasoning mistakes (Jung et al.) [2022] Wang et al. 2023). which can undermine user trust and the reliability of their applications. A signíficant challenge is that the knowledge mining process implicit, making it difficult to discern what LLMs are particularly good or bad at. This lack of transparency hinders targeted improvements and quality assurance of LLMss. Additionally, relying users for sufficient and effective feedback is often difficult and impractical, as it requires extra effort and users typically seek answers questions they do not fully understand. This situation poses significant obstacle in continually improving LLMs based on massive label-free user queries.
"Corresponding Author

---

Published as a conference paper at ICLR 2026
To enhance LLMs, current researches predominantly follow two methodologies: unsupervised language modeling (Fuji et al. 2024 Guo et 2024b) and supervised fine-tuning (SFT) (Mitra et al. 2023; Xu et al.] 2024), which are respectively shown in Figure I (a) and (b). Unsupervised
language modeling uses vast amounts of unlabeled data, enabling L.MMs to learn knowledge implicitly, In contrast, SFT involves training L.MSs on labeled datasets tailored to specifie tasks. Despite their
advantages, both approaches have limitations. First, they can be inefficient as it necessitates the inclusion of extensive data indiscriminately, which may not address enough reasoning mistakes of LLMs. Second, they still lack a comprehensive understanding of LLMs, leading to the inability to make targeted improvements. As a result, this further leads to ineffectiveness in addressing specific and long-tail questions. Moreover, labeling user queries can help to reveal some mistakes of LLMs, while is costly and challenging to use limited labeled samples to thoroughly evaluate LLMs that generalize well (Liu et al., 2023 Zheng et al. 2024). These limitations highlight the need for more efficient and cost-cfiective methods to diagnose and improve LLMs.
To tackle the above challenges, and consider- ing that reasoning mistakes in LLMs often arise from knowledge deficiencies such as lack of knowledge and ineffective application of exist- ing knowledge (Gao al., 20233, Xiong al.) 2024), we aim to diagnose the knowledge de- ficiencies in LLMs and remedy them without any costly annotations. By diagnosing these knowledge deficiencies, we can tailor solutions for targeted and efficient improvements. Fur- thermore, our method enables LLMs evolve continuously with increasing user engagement, even in the absence of user feedback. This ap- proach ensures that LLMs remain current and adaptable, capable of handling specific and long- tail user demands, avoiding costly data labeling processes, and promoting a more efficient on- line development cycle. In this paper, shown in Figure I (c), we design an indirect supervision method called label-free curricular meaningful learning (LaMer), which first diagnoses the knowledge deficiencies of a specific LLM in label-free setting, and then devise curricular meaningful learning to efficiently and effectively remedy the defi- ciencies of corresponding LLM. Specifically, LaMer first extracts relevant knowledge for user queries. Subsequently, inspired by information theory (Shannon 1948), where relative entropy (Kullback & Leibler, 1951) can estimate the extra information needed to change from a distribution to another. We leverage relative entropy with the extracted knowledge to diagnose the knowledge deficiencies of LLMs without relying on labels. Finally, we adopt curricular meaningful learning, which initially uses meaningful learning (Xiong et al. 2024) to adaptively synthesize augmented data across various scenarios based on the severity of the deficiencies, and then employ curricular deficiency remedy to progressively address these deficiencies from minor to severe.
Unlabeled Corpus
Plain
Autoregnessively Pre-troining
LLMs
Text
Unsupervised Longuage Modeling
Humars/
Supervised
Queries
LLMs
Labeled
Fine-tuning
LLMs
Data
(b) Supervised
Fine-tuning
Label-free Knowledge
Deficiencies Anolysis
Synthesis
Data
Augmented
Data
Queries
Curricular
Meaningful
Extraction
Knowledge
LLMs
Learning
(c)
Indirect Supervision (Ours)
Figure I: (a) Unsupervised language modeling,
(b) supervised fine-tuning, and (c) our proposed indirect supervision method.
We conduct extensive experiments on 4 open-source LLMs and evaluate LaMer and baselines across 7 -it-o---iitributionn (OOD) reasoning benchmarks. The results show that LaMer proficiently diagnoses the knowledge deficiencies in various LLMs, leading to more efficient and effective improvements compared to baselines. It achieves comparable results to baselines with only 40% training data. Further analyses reveal that LaMer not only surprisingly surpasses methods relying on labeled data to detect deficiencies but also highlights its efficiency and effectiveness in diagnosing and remedying knowledge deficiencies in LLMs, offering a more robust solution for improving their application. We summarize our contributions follows:
We incorporate relative entropy to effectively diagnose knowledge deficiencies in LLMs without labels, breaking the limitation of relying solely on existing labeled datasets.
We develop curricular meaningful learning to remedy the knowledge deficiencies in LLMs.
Our proposed LaMer excels in diagnosing and remedying the knowledge deficiencies. towards maximizing the potential of existing open-source LLMs.
2

---

Published as a conference paper at ICLR 2026
Quenis (wil Labels)
Step2
Label-fres Erowledgs Deficiency Diognesis
Step3
Curricular Meoningful Learning
Enput Acid teuches skin,
this?
Teferenos nesponses)
Reletive
Entragy
1+K
Mesningful Learning
Ieput: What hoppers
Psi [0.3,0.7]
1,03
QF [0.9. 0.3]
acid?
Outputt Acid The ondwer
Step1
KB
Pgi [0.2, 0.83
0.04
Qui (0.1. 0.9]
Knowledge Extraction
LLM
Irput: I duckeed What
Pg [0.6, 0.43
Qui [0.3, 0.71
Output Fireworks onswer
Tiput (3): Acid touches humon skin, what was the couse of this?
Queries Krowledge
Pai (0.5, 0.5]
0.51
Qui [0.9. 0.1]
Inguti I made a grammatica errer
Output: Hitting The
Curricular Deficiancy Remedy
Knowlodge (K) comrosive...
deficiency
non-deficiency
Order:
Figure 2:
The whole workflow of LaMer to automatically diagnose and remedy the knowledge deficiencies of LLMs in a label-free setting.
2 METHOD: LABEL-FREE CURRICULAR MEANINGFUL LEARNING
We design label-free curricular meaningful learning framework named LaMer, which utilizes user queries to efficiently diagnose and remedy the knowledge deficiencies in LLMs without labels. As illustrated in Figure LaMer consists of steps: (1) Knowledge Extraction obtains relevant knowledge from an external knowledge base for each query to help diagnose knowledge deficiencies; (2) Label-free Knowledge Deficiency Diagnosis leverages relative entropy automatically diagnose and quantify the deficiencies in LLMs, which does not rely on labels; (3) Curricular Meaningful Learning incorporates the idea of human conducting meaningful learning (Tenenbaum) 2018) first adaptively synthesize examples in various scenarios for each knowledge deficiency, and then utilize curricular remedy strategy to effectively repair the knowledge deficiencies from minor to severe.
2.1 KNOWLEDGE EXTRACTION
To acquire knowledge for the following deficiency diagnosis step, employ external knowledge base GenericsKB (Bhakthavatsalam et al. 2020) to obtain knowledge for each query the given query set D. GB is large-scale resource with more than 3.4M naturally occurring generic facts (such as "Trees remove carbon dioxide from the atmosphere" and "Dog barks").
Specifically, to ensure the quality of GenericsKB, we first filter out facts with confidence score lower than 0.7 and remove duplicates. Next, we employ FlagEmbeddding (Zhang et al., 2023a) to represent the facts in GenericsKB and each query in 1) dense embedding. Finally, for each query E D, we apply cosine similarity to match m pieces of knowledge = [k1,... ... kim from GeneriesKB. The matched knowledge K and example d are used in step diagnose knowledge deficiencies in specific LLM L through relative entropy.
2.2 LABEL-FREE KNOWLEDGE DEFICIENCY DIAGNOSIS
To diagnose the knowledge deficiencies of specific LLM L in a label-free setting, we propose to use relative entropy (Kullback & Leibler 1951). This measure quantifies the additional information needed to transition from one distribution to another. Thus, by computing the relative entropy (RE) between predictive distributions of L before and after the introduction of knowledge, we can estimate the volume of information that this knowledge imparts to L. If L exhibits a high RE on this knowledge, it suggests that the model either lacks this knowledge or is unable to integrate it into its problem-solving processes, a knowledge deficiency in L is diagnosed.
Specifically, given query d D, which possesses input I. We first send d to LLM L to obtain n responses O = {01, .2. , and the negative log- likelloiddood (NLL) of each response o, conditioned on I. Hereafter, acquire prior distribution of L over O:
p: L(o,[T),
(I)
Softmax([p) (pp..... .P.)).
where p: denotes the NLL of response o; conditioned on according to C. P E R. denotes the prior distribution of L over O. Softmax is the normalization function.
3

---

Published as a conference paper at ICLR 2026
Secondly, for each extracted knowledge k K of d, additionally introduce i to L to fetch knowledge-baad baaedd posterior distribution of C over O:
4. [( [k, I),
(2)
Q Softmax [q), (.. .4.)).
where gi denotes the NLL of response or after introducing k to L. Q E IR" denotes the knowledge- based posterior distribution of L over O.
Next, we compute relative entropy RE between P and Q to quantify the difficiency of L on is K:
Σ
Ps (log(Q:) log().)).
(3)
Finally, after estimating RE on each knowledge of each query based on L, we filter the knowledge and corresponding queries that result in an RE larger than a threshold т. Each filtered knowledge and its associated query are treated as a unit, representing a knowledge deficiency in L. RE is also a quantification of the knowledge deficiency in L.
Note that there might be two situations: (1) Helpful: the extracted knowledge has a positive impact on L, resulting in higher confidence in the correct answer; (2) Misleading: the obtained knowledge has a negative impact on L. resulting in higher confidence in the wrong answer. We suppose that both situations can expose the knowledge deficiencies in L. The first situation suggests L might not grasp this knowledge or cannot properly apply this knowledge to problem-solving. while the second situation indicates that does understand this knowledge but is easily misled by it.
2.3 CURRICULAR MEANINGFUL LEARNING
Humans adopt meaningful learning TTnen- baum 2018 Xiong et al. 2024 to induce and
Table 1: Grouped knowledge deficiencies and the corresponding size of synthesized examples.
learn new knowledge through its application across diverse situations, leading to deep understanding and integration of knowledge. Moreover, Humans exploit curriculum learn- ing (Bengio et al. 2009) to effectively learn new knowledge by progressing from easy hard lev- els. In light of this, we combine them and design curricular meaningful learning to effectively remedy the diagnosed knowledge deficiencies of L.
Group
RE Interval
Synthesized Examples
Easy
0.1 S RE < 0.4
1
Normal
0.4 < RE < 0.7
2
Hard Unfair
0.7 S RE < 1.0
3
RE 2 1.0
4
Firstly, we employ meaningful learning strategy to synthesize varying examples in diverse scenarios according to the deficiency severity. It is inspired by meaningful learning in humans and the insights that LLMs typically require more tokens examples learn the knowledge if they have less prior understanding (Ovadia et al. 2023 Gekhman et al. 2024). This strategy can reduce the cost and make deficiency remedy more efficient. Specifically, for the diagnosed deficiencies of L. divide them into 4 groups according to the severity (RE) of them. For each group, we heuristically assign a number, which indicates the number of diverse examples we should synthesize for each deficiency in the group. The detailed groups and assigned numbers are shown in Table I Subsequently, we adopt ChatGPT (Achiam et al. 2023) to synthesize the specified number of examples for the deficiencies in each group. The deficiency (knowledge and the corresponding query) is harnessed to guide the data synthesis process, Each synthesized example contains an input X and an output Y.
Secondly, we devise a curricular remedy strategy to remedy the knowledge deficiencies in LLM L from minor to severe. Specifically, we sort the generated examples in ascending order based on the severity of their knowledge deficiencies, and then feed them into training L sequentially. For each example X,Y >, we train L autoregressively to maximize a conditional probability:
((X,Y,0) = Σ logp, (Y/)X, Yze);
(4)
where 0 denotes parameters of L. This approach results in updated L with deficiencies remedied.
4

---

Published as a conference paper at ICLR 2026
Table Overall performance of LaMer and baselines. Bold numbers denote the best performance among all methods. Average denotes the performance averaged across all benchmarks.
LLMs
Size Methods
Comm.
AGlEval
ARC
MMLU
BBH CRASS
GSM-Plus
Average
Base
67.56
32.82
74.15
49.75
28.47
71.67
29,91
50.62
AugGPT
74.53
33.03
76.93
52.56
33.82
81.67
13.78
52.33
Mistral
7B
Naive
67 7,42
33.60
72.66
50.58
32.22
80.00
31.19
52.52
Single
69.47
34.87
74.85
51.44
29.53
80.00
29.03
52.74
LaMer (Ours)
75.10
34.50
77.22
54.52
33.72
88.33
29.23
56.09
Base
74.84
39,45
86.29
59.72
39.85
76.67
61.93
62.82
AugGPT
76.75
38.13
86.04
60.29
36.60
76.67
14.92
55.63
LLaMA-3
8B
Naive
71.28
33,86
84,50
57.86
40.93
83.33
61.56
61.90
Single
76.36
39.72
84.72
58.97
36.53
76.67
55.85
61.26
LaMer (Ours)
78.15
40.85
86.41
60.24
39.69
88.33
59.22
64.70
Base
71.88
42.10
85.48
59.53
37.58
85,00
61.79
63.34
AugGPT
79.41
42.71
89.78
63.66
40.00
90.00
14.75
60.04
Qwen2
7B
Naive
75.62
42.80
88.40
62.50
39.34
86.67
61.82
65.31
Single
76.41
44,30
87.6
61.98
39.69
80.00
56.62
63.81
LaMer (Ours)
78.13
45.02
88.62
62,48
40.56
86.67
61,44
66.13
Base
53.26
25.80
49.97
33.73
23.29
38.33
7.08
33.07
AugGPT
55.42
25.76
47.73
33.69
24.31
36.67
3.95
32.50
Gemma-1.1 2B
Naive
55.38
25.78
47.79
32.23
24.24
36.67
5.99
32.58
Single
52.85
24.51
46.67
34.00
25.03
31.67
5.86
31.51
LaMer (Ours)
55.81
25.81
48.91
34.40
25.26
41.67
6.69
34.08
EXPERIMENTS
3.1 INVESTIGATED LLMS
We adopt 4 open-source LLMs for experiments illustrate the general applicability of our pro- posed LaMer: (1) Mistral-7B-Instruet--a
(Jiang et al. 2023) (Mistral) is an efficient chat LLM. dense LLM with massive pre- pre-training. (3) Qwen2-7B-Instruet (Yang et al. 2024 (Qwen2) is powerful multilingual LLM. (4) Gemma-
(2) LLaMA-3-BB-Intrrue
(Meta 2024) (LLaMA-3)
1.1-2B-IT (Team al. 2024) (Gemma-l.1 is powerful small-scale chat LLM.
3.2 BASELINES
We adopt a wide range of baselines for comprehensive comparisons: (1) Base employs the base LLMs Section 3.I answer questions in each benchmark. (2) AugGPT (Dai al. 2023) uses ChatGPT (Achiam et al. 2023) to generate questions and answers to augment LLMs with SFT. AugGPT does not provide ofain-of-though (Wei al. 2022) in generated examples. (3) Naive is an SFT method that randomly samples several pieces of knowledge from the knowledge base to synthesize new examples without considering whether the specific LLM possesses deficiencies on sampled knowledge. (4) Single follows similar process to LaMer but synthesizes only 1 example per knowledge deficiency. Hence, the training data of Single is 40% of LaMer or the other methods.
3.3 EVALUATION BENCHMARKS
We choose 7 OOD benchmarks ranging from reasoning to language understanding, to evaluate the performance of LaMer and baselines: (1) Comm. (Xiong et al. 2023) collection of commonsense reasoning datasets. (2) AGIEval (Zhong et al. 2024) consists of diverse sets of standardized tests ranging from college admission tests (such as GRE and GMAT) to national civil service examinations. (3) ARC (Clark et al.) 2018) is the A12 Reasoning Challenge, which is benchmark science exams spanning Grade 3 to Grade 9 with easy (ARC-e) and challenge (ARC-c) subsets. (4) MMLU (Hendrycks et al.) 2021) aims to evaluate language comprehension. knowledge, and reasoning skills LLMS with 57 tasks. (5) BBH (Suzgun et al.) 2023) is a subset of Big- Bench (Srivastava et al. 2023), which contains 23 hardest tasks focusing challenging scenarios.
5

---

Published as a conference paper at ICLR 2026
66 CRASS (Frohberg & Binder 2022 measures counterfactual reasoning in language models. GSM-Plus (Li al. 2024b) is a comprehensive math benchmark for evaluating the robustness of Lss. We knly the examples that possess valid answers for evaluation.
keep
3.4
IMPLEMENTATION DETAILS
For knowledge extraction, we choose e-CARE (Du et al. 2022) and GSMSK (Cobbe et al. 2021), discarding the labels obtain query set D. The size of GenericsKB after filtering is 200K. We utilize FlagEmbedding (Zhang et al. 2023a) with beg-large-en--1.55 to encode facts and queries. We acquire facts for each query. Since GSM8K is far from GeneriesKB,
use ChatGPT to generate m = pieces of knowledge for GSM8K. We generate n = 2 responses for each query.
For the label-free knowledge deficiency diagnosis step, the knowledge and corresponding query with an RE higher than r = 0.1 is treated as knowledge deficiency of C. The size of selected knowledge deficiencies can refer to Appendix
For the meaningful learning strategy, we utilize a prompt to instruct ChatGPT to synthesize examples. Finally, synthesize 3,750 examples enhance Mistral, Qwen2, and Gemma-1.1, while 1,250 examples are synthesized to enhance LLaMA-3 due to denser knowledge in it.
For the curricular deficiency remedy strategy, we adopt LoRA (Hu al. 2022) for parameter-efffcient fine-tuning. The rank r and a of LoRA are 128 and 8, respectively. We train LaMer for 3 epochs with learning rate of 5e-5. The batch size is 32. The optimizer we used is Adam (Kingma & Ba 2014). Two NVIDIA A100 80GB PCIc GPUs are used for training and the following evaluation.
For AugGPT and Naive, we randomly sample 3.750 facts from GenericsKB and the generated knowledge of GSM8K to generate the same number of training examples as LaMer for each LLM. While for Single, we randomly sample one example for each deficiency from the training data of LaMer. Therefore, Single enhances Mistral, Qwen2, and Gemma-1.1 with 1,500 examples, and it utilizes 600 examples to enhance LLaMA-3. The whole data synthesis process and the training setup are the same as LaMer. All prompts for the preprocess and data synthesis can refer to Appendix C We utilize g.t-3. 3. 5 1-ttuuroo-0111 for all relevant implementations based on ChatGPT.
3.5 EVALUATION DETAILS
For all benchmarks, we utilize free-form generation to evaluate all methods, the evaluation prompts for different LLMs can refer to Appendix [ The performance of each method on each benchmark averaged across all tasks in the corresponding benchmark.
3.6 OVERALL RESULTS
The overall results are shown in Table from which we can have the following observations:
(1) On average, LaMer outperforms all baselines across different LLMs, which is mainly due to the effectiveness of LaMer in diagnosing and remedying knowledge deficiency. This also reveals the general applicability of LaMer, making it a plug-and-pon method to improve L.LMs.
(2) Data augmentation methods can surpass the base LLM on most benchmarks, while Naive and Single show performance drops compared to LLaMA-3 and Gemma- 1.11, as LLaMA-3 and Gemma- 1.1 possess dense knowledge in its parameters. Naive and Single could supplement some knowledge to them but cause them to forget more useful knowledge.
(3) Interestingly, Single, which is trained with 40% training data of LaMer (one example for each knowledge deficiency), achieve comparable performance with Naive on Mistral and LLaMA-3 with much fewer training data. This indicates Naive method can produce more redundant data that the base LLM already possesses, while detecting knowledge deficiencies in LLMs apply targeted improvements, making it more efficient and less costly than other data synthesis methods.
(4) LaMer excels Single across all LLMs, this is because more and diverse examples for LLMs to effectively remedy them (Gekhman et al.) 2024). LaMer synthesizes more examples for knowledge deficiencies with higher severity, whereas Single generates
severe knowledge deficiencies require

---

Published as a conference paper at ICLR 2026
only example for each knowledge deficiency. As result, many knowledge deficiencies in the LLMs are not adequately remedied by Single.
(e) Enitial Query knowledge Retrieval
(5) AugGPT achieves the worst results GSM- Plus, this is because math problems demand multiple reasoning steps solve. AugGPT di- rectly offers the answer to the math problems, making it hard to develop precise calculations to arrive at the right answer. Furthermore, Aug- GPT can reach the best performance on several benchmarks (such as ARC on Qwen2), this is because problems in these benchmarks might be
Query: The boy swallowed piace of silver by accident. What happened as result?
(A) He wasn't 1 poisoned (B) He got Toxic
Retrieved Knowledge: Silver is mildly toxic element.
(b) Label-free Knowledge Deficiency Diagnosis
responde without knowledge
Swallowing a piece of silver accident does not necessarily mean the kid was poisoned got
Toxic. So the answer is (A). [0.63, 0.37]
LLM
better answered with just step, which is also indicated in Zhang al. (023b))
+Knowledge
Relative Entropy Q): 0.5964
eeesponde with knowledge-
normal deficiency
(6) Qwen2 could exceed LLaMA-3 across differ- ent methods with fewer parameters, we suppose this is mainly due to the massive post-training efforts on Qwen2.
His tissues can absorb silver, leading to toxic. Therefore, the answer (B). Q (0.15, 0.85]
(c) Curricular Meaningful Learning
Input: chemist accidentally spilled a solution containing silver
their skin. What the likely outcome? Skin invitation (8) No effect (C) Allergic reaction
CASE STUDY
Output: Silver is mildly toxic element that can skin irritation upon contact. Therefore, the answer (A).
Figure shows case demonstrate how
Input: What is a potential of using silver nanoparticles consumer products? (A) Allergy (B) Erhanced durability
LaMer diagnoses and remedies a deficiency in
Improved
Mistral: (a) For an query about "silver", we ob-
Output: Silver nanoparticles can cause allergic reactions some individuals due to the mildly toxic nature of silver. Therefore, answer (A).
tain the knowledge that "silver is a mildly toxic element". (b) The L.MM offers a wrong response which produce distribution P = [0.63, 0.37] over the options. After providing the LLM with the knowledge, the LLM offers new response
Curricular Deficiency Remedy
(d) Re. Answer the Initial Query
Swallowing piece of silver is not poisonous, but it couse toxicity
with a different distribution Q = [0.15, 0.85]. The relative entropy between P and Q is 0.60, which means the knowledge brings lot of in- formation to the LLM. Hence, the combination of the knowledge and the query is knowledge deficiency of the LLM. (c) According Table this knowledge deficiency is in normal group.
B
the release silver ions in the body. Therefore, the answer is (B).
LLM
Deficiency Remedied
Figure 3:
A case showing how LaMer diagnoses and remedies a knowledge deficiency in Mistral.
Thus, we ask ChatGPT to synthesize two examples for this deficiency. These two examples are two different applications of the knowledge in this deficiency. Next, the synthesized examples are used to train the LLM. (d) After curricular meaningful learning, Mistral offers correct answer. The deficiency is remedied.
5 FURTHER ANALYSIS
To further investigate the effectiveness of LaMer, design several ablation studies and in-depth analyses: (1) comparisons between LaMer and label-reliant methods to demonstrate the effi- ciency and strengths of LaMer; (2) an effec- tiveness analysis of LaMer on remedying de- ficiencies by obtain the statistics on remedied examples of each method; (3) an ablation study to investigate the effect of curricular deficiency remedy strategy; (4) a significance analysis to inspect the roles of helpful and misleading sit- uations in label- free knowledge deficiency diag- nosis.
Table 3: Effectiveness of different methods on detecting difficiencies Mistral based on e- CARE (Du et al. 2022 dataset.
Methods
Label-free
P
R
F1
Golden Label
No
100
100
100
Perplexity Random
No
48.46
34.24
40.10
Yes
35.23
35.23
35.23
Relative Entropy
Yes
40.34
64.30
49.58
7

---

Published as a conference paper at ICLR 2026
5.1 COMPARISONS TO LABEL-RELIANT RELIANT METHODS
We conduct an analysis to reveal the efficiency of different methods in diagnosing deficiencies based Mistral-7B Eesttal--B-Instruee I----ruuttt-t.122 (Jian et al.
Base LLM2LLM
Average Performance
60
Laker (Ours)
2023) and e-CARE (Du et al.] 2022). Details
55
refer Appendix F.T Results are shown in
50
Table our proposed relative entropy method outperforms perplexity by recalling more defi-
45
ciencies. This proves the feasibility of our pro- posed relative entropy method.
35
Mistral
LLaMA-3
Qwen2
Gemma-1.1
Furthermore, we adopt a label-reliant method LLM2LLM for comparison. L.M22LLM (Lee
Figure 4: Average performance across all bench- marks of base LLMs, LLM2LLM, and LaMer.
et al. 2024) uses labeled data to identify erro- neous examples existing LLMs, and then synthesizes similar examples improve specific LLM. We use the labels of e-CARE and GSM8K to obtain 3,750 error examples, and then we generated similar examples based on the error examples. The number of training examples is the same as LaMer for each LLM. Results are shown in Figure (full results can refer to Appendix F.2), LaMer has varying advantages over LLM2LLM across LLMs. Although LLM2LLM can precisely detect the errors of each L.LM, the errors are limited to the given initial dataset. LaMer diagnoses knowledge deficiencies, which can improve the coverage of diagnosed deficiencies in each I.I.M.
find:
5.2 EFFECTIVENESS IN REMEDYING DEFICIENCIES
AugGPT
LUMZ
We conduct an analysis to examine if LaMer can effectively remedy more eтrоr examples. Specif- ically for each method, we first tally the number of examples in each benchmark that base L.MM wrong but correct after the enhancement by each method (denote as remedied examples). At- ter that, we normalize the values across different methods. Finally, we use spider charts to visual- ize them. We choose Mistral and LLaMA-3 for investigation. Results are in Figure 5
(a) Mistral
(b) LLaMA-3
AGIEval
AGIEval
Comm,
Comm.
MMLU
CRASS
MMLU
CRASS
Figure 5: Normalized remedied examples on (a) Mistral and (b) LLaMA-3 across benchmarks.
(1) LaMer has advantages over baselines on all benchmarks, this is attributed to that LaMer can diagnose more deficiencies via relative entropy, and remedy them via curricular meaningful learning.
(2) In Table 2 although AugGPT can exceed LaMer on some benchmarks, it does not get the upper hand in Figure 5 This is mainly due to AugGPT cannot diagnosis the deficiencies of LLMs.
5.3 EFFECT OF CURRICULAR DEFICIENCY REMEDY
Table 4: Ablation study on curricu- lar deficiency remedy.
To investigate the effect of curricular deficiency remedy in LaMer, we randomly shuffle the data synthesized by LaMer to train the base LLMS (LaMer*). All the training and evaluation settings are the same LaMer. We denote LaMer with shuffled data as LaMer*. The results are shown in Table (full results refer to Appendix F33) we can observe:
LLMs
Size
Methods
Average
LaMer LaMer*
$6.09
Mistral
7B
55.64
LLaMA-3
8B
LaMer
64.70
LaMer"
64.47
(1) LaMer outperforms LaMer" regarding each LLM, since remedying less severe deficiencies helps remedy more severe ones. Curricular meaningful learning can make LLMs Iearn new knowledge more efficiently (Xiong et al. 2024).
Qwen2
7B
LaMer
LaMer"
65.50
Gemma-1.1
LaMer 34,08
LaMer"
33.80
(2) When switched to randomly shuffled data, LaMer" only suffers small performance drops. This claims that the performance of LaMer improvement is stable and LaMer has robust applicability.
(3) LaMer" achieves higher performance than baselines Table Mistral, LLaMA-3, 33, and Qwen2. It clarifies that the advantages of LaMer primarily stem from the deficiencies diagnosis process.
8

---

Published as a conference paper at ICLR 2026
5.4 HELPFUL AND MISLEADING KNOWLEDGE
The knowledge deficiencies can be caused by two kinds of knowledge: (1) helpful knowledge has a very positive impact on the correct answer;
Mistral
Helphul
(2) misleading knowledge leads LLMs choose
Signticcnce (%)
Misleading
the wrong answer with higher confidence than
20
the right answer.
To find out the significance of the deficiencies caused by these two kinds of knowledge, we split the discovered knowledge deficiencies into two groups (Helpful and Misleading) with the help of golden labels. Then respectively train the LLM with data synthesized based on Helpful and Misleading deficiencies. Finally, after eval- uation, we respectively analyze the unique reme- died examples brought by the data synthesized based on Helpful and Misleading deficiencies. have the
LLaMA-3
50
Helpful
Misleading
AGtivvll
MNLU
GSM-Plus
Figure 6: The portion of unique examples that are remedied based on the deficiencies caused by helpful and misleading knowledge on (a) Mistral and (b) LLaMA-3.
Results are shown in Figure 6 following observations:
(I) Deficiencies caused by helpful and mislead- ing knowledge are similarly significant. They can help remedy comparable yet distinct exam- ples. Both highlight valid knowledge deficiencies.
(2) Deficiencies caused by misleading knowledge have slightly greater significance than those caused by helpful knowledge, as they often have larger impact, and is often more difficult and complex.
RELATED WORK
6.1
CONTINUAL TRAINING ON LLMS
LLMs Jiang et al. 2023 Meta 2024 have demonstrated strong generalization capabilities, may not
always perform well on general or specific individual capabilities. Recent works involved additional training customize enhance LLMs.
For enhancing the general capabilities of LLMs. Mic et al. (2023) and Li Lee
(0244)
conducted
continual pre-training LLMs for alignment. Cui et al. (023)) and Huozi Team 2024
conducted
adaptions with massive Chinese data to enhance the Chinese capabilities of LLaMA (Touvron
al. 2023). Taori al. (2023) utilized 52K instruction data from ChatGPT AAhiam al. 2023)
to obtain a strong instruction--ollloiing LLM. Vicuna (Chiang et al.) 2023 adopted data from ShareGPT SharrePPT 2023) train LLaMA and achieved 90% quality ChatGPT. Orca (Mukher
jee et al. 2023) and Orca-2 Mitra et al. 2023) adopt progressive learning and more data from GPT-4 (Achiam al. 2023) further enhance the general abilities of LLMs. Different from the others, WizardIM (Xu et al. (2044 designed an evolve-instruct prompt to distill instruction data from ChatGPT with varying difficulty, encouraging L.Mss follow complex instructions. Zephyr (Tunstall)
al. 2023) employed DPO (Rafailov et al., 2024) to align LLMs with human preference, while SPIN (Chen al. 2024) devised a self-play method to achieve this. For enhancing specific abilities of LLMs, researchers used spsk-spec data to improve the desired abilities of LLMs such as math (Luo)
ct al.
2023 Tang et al. 2024), code (Guo et al. 2024a), and reasoning (Ying et al.) (024))
Our work aims to detect the knowledge deficiencies of LLMs and apply appropriate remedies to repair them, which can serve as complement to existing methods.
6.2 EVALUATION OF LLMS
Since LLMs have broad capabilities, evaluating LLMs becomes tough and widely concerned issue. Some works constructed benchmarks or evaluation data to evaluate LLMs from general and specialized perspectives, such natural language understanding (Hendrycks et al. 2021 Li al.

---

Published as a conference paper at ICLR 2026
2023
Wang al. 2024), reasoning (Frohberg & Binder 2022, Suzgun et al. 2023 Zhong et al.
2024). math (Li et al. 2024b Liu et [024]]. coding (Peng et al. 2024 Zhang et al. 2024). etc.
Different from using benchmarks with objective questions, some works started adopting LLMs to evaluate LLMs subjectively. Liu et al. 2023) applied GPT-4 as to assess the quality generated text in various pesperctives (such coherence) and achieved better alignment with human evaluators, [Bai] et al. (2024) evaluated the performance of existing LLMs with self-evaluation and peer-evaluation. achieving more precise judgments of existing LLMs. Furthermore, Zheng al. (2024) devised chat framework to evaluate LLMs based on the discussions among LLMs.
Our work emploies relative entropy to diagnose the knowledge deficiencies in LLMs based on knowledge base. We also follow previous work to adopt objective benchmarks to evaluate LLMs.
7 CONCLUSION
In this paper, we design a label-free curricular meaningful learning framework (LaMer) based on relative entropy first automatically discover the knowledge deficiencies from massive label- free user queries. Then we devise curricular meaningful learning which consists of a meaningful learning strategy and curricular deficiency remedy strategy, to efficiently and effectively remedy the discovered knowledge deficiencies of corresponding LLM. The experiments show that our proposed LaMer can improve the coverage diagnosed deficiencies, and surpass the baselines, making LLMs enhancement free of labeled data. The relative-entropy -basdd deficiency method provides robust, efficient, and label-free deficiency diagnostic tool for existing LLMs further unlock their potential.
ACKNOWLEDGEMENT
The research in this article supported by the New Generation Artificial Intelligence of China (2024YF00203700), National Natural Science Foundation of China under Grants U22B2059, 62576124, and 62576102.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.082: 087774 2023.
Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze Hc, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, et al. Benchmarking foundation models with Advances in Neural Information Processing Systems, 36, 2024.
model-as-an-cee
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pp. 41-48, 2009.
Sumithra Bhakthavatsalam, Chloe Anastasiades, and Peter Clark. Genericskb: knowledge base of generic statements. arXiv preprint arXiv:2005.006 00660, 2020.
Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models. In Forty-first International Conference on Machine Learning. 2024.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph Gonzalez, lon Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint wwX..v:1803.5445
2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reñchiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
10

---

Published as a conference paper at ICLR 2026
Yiming Cui, Ziqing Yang, and Xin Yao. Efficient and effective text encoding for chinese llama and alpaca. arViv preprint arXiv:2304.08177, 2023.
Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Yihan Cao, Zihao Wu, Lin Zhao, Shaochen Xu, Wei Liu, Ninghao Liu, et al. Auggpt: Leveraging chatgpt for text data augmentation. arXiv preprint arXhv:2302.13000 2023.
Li Du, Xiao Ding, Kai Xiong, Ting Liu, and Bing Qin. e-care: a new dataset for exploring explainable causal reasoning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 432-446, 2022.
Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pp. 226-231, 1996.
Jörg Frohberg and Frank Binder. Crass: A novel data set and benchmark test counterfactual reasoning of large language models. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 2126-2140, 2022.
Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki lida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, and Naoaki Okazaki. Continual pre-training for cross-lingual Ilm adaptation: Enhancing japanese language capabilities. arXiv preprint arXiv:2404.17790, 2024.
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai. Jiawei Sun, and Haofen Wang. Retrieval- -uggeeeee generation for large language models: A survey. arXiv preprint arXiv:2312. 2023.
Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. Does fine-tuning 1lms on new knowledge encourage hallucinations? arXiv preprint arXiv:2405.05904, 2024.
Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder. When the large language model meets programming the rise of code intelligence. arXiv preprint arXiv:2401. 14196, 2024a.
Yiduo Guo, Jie Fu, Huishuai Zhang, Dongyan Zhao, and Yikang Shen. Efficient continual pre-training by mitigating the stability gap. arXiv preprint arXiv:2406. 14833, 2024b.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representationss 2021.
Edward Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference Learning Representations, 2022.
Huozi-Team. Huozi: Leveraging large language models for enhanced open-domain chatting, 2024.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. Maieutic prompting: Logically consistent reasoning with recursive explanations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1266-1279 1279, 2022.
Sein Kim, Hongseok Kang, Seungyoon Choi, Donghyun Kim, Minchul Yang, and Chanyoung Park. Large language models meet collaborative filtering: An efficient all all-round llm- bmmdd recommender system. arXiv preprint arXiv:2404.1 1343, 2024.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
11

---

Published as a conference paper at ICLR 2026
Solomon Kullback and Richard A Leibler. On information and sufficiency. The annals of mathemati- cal statistics, 22(1):79-88) 86, 1951.
Nicholas Lee, Thanakul Wattanawong, Sehoon Kim, Karttikeya Mangalam, Sheng Shen, Gopala Anumanchipali, Michael W Mahoney, Kurt Keutzer, and Amir Gholami. Llm2llm: Boosting llms with novel iterative data enhancement. arXiv preprint arXiv:2403.15042. 2024.
Yuxuan Lei, Jianxun Lian, Jing Yao, Xu Huang, Defu Lian, and Xing Xie. Recexplainer: Aligning large language models for recommendation model interpretability. arXiv preprini 1rXiv:2311.10%0 2023.
Chen Li, Weiqi Wang, Jingeheng Hu, Yixuan Wei, Nanning Zheng, Han Hu, Zheng Zhang, and Houwen Peng. Common 7b language models already possess strong math capabilities. arXiv preprint (tt1:240330470066 2024a.
Chen-An Li and Hung-Yi Yi Lee. Examining forgetting in continual pre-training of aligned large language models. arXiv preprint arXiv:2401.03/29 2024.
Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang. Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. Cmmlu: Measuring massive multitask language understanding in chinese. arXiv preprint arXiv:2306.09212, 2023.
Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. Gsm-plus: A comprehensive benchmark for evaluating the robustness of Iims as mathematical problem solvers. arXiv preprint arXiv:2402.1925: 19255, 2024b.
Hongwei Liu, Zilong Zheng, Yuxuan Qiao, Haodong Duan, Zhiwei Fei, Fengzhe Zhou, Wenwei Zhang, Songyang Zhang, Dahua Lin, and Kai Chen. Mathbench: Evaluating the theory and application proficiency of lims with hierarchical mathematics benchmark. arXiv preprint arXiv:2405. 12209, 2024.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 251 2511-2522, 2023.
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.099 09583, 2023.
Meta. Llama 3 model card. 2024.
Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, et al. Orea 2: Teaching small language models how to reason. arXiv preprint arXiv:2311.11045 2023.
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02700) 027007 2023.
Oded Ovadia, Menachem Brief, Moshik Mishaeli, and Oren Elisha. Fine-tuning retrieval? comparing knowledge injection in llms. arXiv preprint a3X1v:2312.05934, 2023.
Qiwei Peng, Yekun Chai, and Xuhong Li. Humaneval-xh xl: A multilingual code generation benchmark for cross-lingual natural language generalization. arXiv preprint arXiv:2402.1669) 2024.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
Claude Elwood Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379-422 1948.
12

---

Published as a conference paper at ICLR 2026
ShareGPT. Sharegpt: Share your wildest chatgpt conversations with one click, 2023. Accessed: 2024-07-27.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso et al. Beyond the imitation game: quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research, pp. 1-95, 2023.
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, et al. Challenging big-bench tasks and whether ffaii-of-thhugeee can solve them. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 38003-13051, 2023.
Zhengyang Tang, Xingxing Zhang, Benyou Wang, and Furu Wei. Mathscale: Scaling instruction tuning for mathematical reasoning. In Forty-first International Conference on Machine Learning. 2024.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction foirruction-fooowinng Ilama model, 2023.
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre,, Morgane Rivière, Mihir Sanjay Kale, Julictte Love, et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295 2024.
Josh Tenenbaum. Building machines that learn and think like people. In Proceedings of the 17th International Conference on Autonomous Agents and MulliAgen Systems, pp. 5-5, 2018.
Hugo Touvron, Thibaut Lavril, Gautier Izacard. Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 3,91,, 2023.
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al. Zephyr: Direct distillation of im alignment. arXiv preprint arXiv:2310.16944, 2023.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 911),, 2008
Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. arXiv preprint arXiv:2305.17996. 2023.
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint 24X.v:2406.01574, 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 2224274-44877, 2022.
Junda Wu, Cheng-Chup Chang, Tong Yu, Zhankui He, Jianing Wang, Yupeng Hou, and Julian -ugmeetee large language models improve long-tail recommendation. arXiv preprint arXiv:2403. 06447, 2024.
McAuley. Coral: Collaborative
Yong Xie, Karan Aggarwal, and Aitzaz Ahmad. Efficient continual pre-training for building domain specific large language models. arXiv preprint 2311v:2311.0880 2023.
Kai Xiong, Xiao Ding, Yixin Cao, Ting Liu, and Bing Qin. Examining inter-consistency of large language models collaboration: An in-depth analysis via debate. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 7572-7590, 2023.
Kai Xiong, Xiao Ding, Ting Liu. Bing Qin, Dongliang Xu, Qing Yang, Hongtao Liu, and Yixin Cao. Meaningful learning: Enhancing abstract reasoning in large language models via generic fact guidance. Advances in Neural Information Pro Processing Systems, 37:120501-120522 2024.
13

---

Published as a conference paper at ICLR 2026
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. WizardIm: Empowering large pre-trained language models to follow complex instructions. In The Twelfth International Conference on Learning Representations, 2024.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang. Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang. Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. Qwen2 technical report. arXiv preprint arXiv:2407. 10671, 2024.
Jiahao Ying. Mingbao Lin, Yixin Cao, Wei Tang. Bo Wang, Qianru Sun, Xuanjing Huang, and Shuicheng Yan. Lims-as-instructor:: Learning from errors toward automating model improvement. arXiv preprint arXhv:2407.00497 2024.
Peitian Zhang, Shitao Xiao, Zheng Liu, Zhicheng Dou, and Jian- Nie. Retrieve anything to augment large language models, 2023a.
Shudan Zhang, Hanlin Zhao, Xiao Liu, Qinkai Zheng, Zehan Qi, Xiaotao Gu, Xiaohan Zhang, Yuxiao Dong, and Jie Tang. Naturalcodehemch: Examining coding performance mismatch on humaneval and natural user prompts. arXiv preprint arXiv:2405.04520 2024.
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. In The Eleventh International Conference on Learning Representations, 2023b.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024.
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. Agieval: human-centric benchmark for evaluating foundation models. In Findings of the Association for Computational Linguistics: NAACL 2024, pp. 2299-2314 2024.
A USE OF LLMS
We use ChatGPT check the grammar and polish writing. We not use LLMS for experiments or idea creating.
B LIMITATIONS
This paper still have some limitations: (1) We require related knowledge base for given queries. While for domain-speciffc queries, obtaining such knowledge is difficult. (2) LaMer does not work well math datasets. (3) Complex scenarios such complex reasoning could be further discussed.
14

---

Published as a conference paper at ICLR 2026
C PROMPTS FOR IMPLEMENTATION
C.I PROMPT FOR PROCESS GSM8K
Prompt for Processing GSMSK
You are an expert in math problems.
First, please generate some background knowledge which can be used to solve this question. The knowledge SHOULD be general and applicable, Your generated knowledge CANNOT be question-specifie. Like commonsense reasoning, I can give the knowledge "Acid is corrosive." Just list the knowledge.
Second, please answer this question by giving a short explanation and then give the answer. Third, please generate some distractors for your answer and index them like "(A).. (B) ..."!
The answer and distractors should be as concise as possible. Your response SHOULD follow the following format: Background Knowledge: [The generated knowledge] Explanation: [Steps to achieve the answer] Answer: [A pure math part] Distractors: [Wrong answers]
Since GSMSK (Cobbe et al.) 2021) contains math problems and without options, we adopt ChatGPT generate distract options and knowledge for each query in GSM8K. The following is the prompt:
15

---

Published as a conference paper at ICLR 2026
C.2 PROMPT FOR SYNTHESIZING DATA
LaMer: e-CARE
You are expert in creating reasoning and language understanding questions.
Here are the requirements:
You are given a fact and some examples for reference, the fact can implicitly guide the solution for the reference examples. You should understand the Internal Mechanism of this guidance to make new examples.
2. Each created example should contain question, several options (,=3), answer, and short explanation for the answer.
3. The options should be short sentence or phrase rather than a single word whenever possible.
4. Don't t make any commonsense mistakes and ensure that your solutions are accurate.
5. You SHOULD propose totally new reasoning questions in various areas, including but not limited to history, law, medical. math, science, computer science, psychology, AI, politics, economies, etc.
6. NOTE that the fact should be an implicit explanation for obtaining the true answer, which means the fact SHOULD NOT appear explicitly in the questions or the options.
7. Only one option is the correct answer, the other options should be much less plausible than the correct option or they are just wrong options.
8. Therefore is explanation in the reference examples, you SHOULD generate explanation
first and then give the answer for your generated new questions.
9. The question could be in any form, such "Why, What, How, Which" etc. You also add
a premise to form a question.
10. The created examples cannot be different just in nouns.
Reference:
Knowledge: {
}
Examples: {
You MUST generate
new examples. The examples MUST be totally different from each other and the reference examples. Please return your response in the form:
Question: [QUESTION]
Options: [CANDIDATE OPTIONS]
Answer: (The option index of the answer such as (B))
Explanation: [A concise explanation for the answer]
Question: [QUESTION]
Options: [CANDIDATE OPTIONS]
Answer: [The option index of the answer such (B))
Explanation: [A concise explanation for the answer]
16

---

Published as a conference paper at ICLR 2026
LaMer: GSM8K
You are an expert in creating math questions. Your goal to generate set of new math questions based on the reference fact and examples.
Here are the requirements:
1. The example should contain question, a solution to derive the answer to the question, several options (i=3), and an answer.
2. Don't make any commonsense mistakes and ensure that your solutions are accurate.
3. You SHOULD propose totally new reasoning questions.
4. NOTE that the fact should be implicit guidance for obtaining the true answer, which means
the fact SHOULD NOT appear explicitly in the questions, solutions, or options.
5. Only option is the correct answer.
6. There are no solutions in the reference examples, you SHOULD generate solutions first and
then give the answer for your generated new questions.
7. The question could be in any form. You can also add premise to form a question.
Reference:
Knowledge: {
}
Examples: {
You MUST generate
examples. The examples MUST be totally different from each other and the reference examples. Please return your responses the form:
Question: [QUESTION]
Solution: [A CONCISE step-by-step SOLUTION to DERIVE the ANSWER to the Question]
Options: ICANDIDATE OPTIONS containing the answer]
Answer: [The option of the answer such as (B) $15]
Question: [QUESTION]
Solution: [A CONCISE step-by-step SOLUTION to DERIVE the ANSWER to the Question]
Options: ICANDIDATE OPTIONS containing the answer]
Answer: The option of the answer such (B) $15]
D PROMPT FOR EVALUATION
Mistral
[NSST]Questio::
Options:
H/INS]]
LLaMA-3 and Qwen2
Question: {
}
Options:
assistant
17

---

Published as a conference paper at ICLR 2026
Table Overall performance of LaMer, LLM2LLM, and baselines. Bold numbers denote the best performance among all methods. Average denotes the average performance across all benchmarks.
L.LMs
Size
Methods
Label-free
Comm.
AGlEval
ARC
MMLUU
BBH
CRASS
GSM-Plus
Average
Base
67.56
32.82
74.15
49.75
28.47
71.67
29.91
50.62
AngGPT
52.56
Mistral
Naive
Yos
67.42
33.60
72.66
50.58
32.22
80.00
31.19
52.52
7B
Single
69.47
34.87
74.85
51.44
29.53
80.00
29.03
52.74
LLM2LLM
74.59
34.44
78.66
55.03
34.58
83.33
16.33
54.00
LaMer (Ours)
Yes
75.10
34.50
77.22
54,52
33.72
88.33
29.23
$6.09
Base
74.84
39.45
86,29
59.72
39.85
76.67
61.93
62.82
AugGPT
76.75
38.13
86.04
60.29
36.60
76.67
14.92
55.63
LLaMA-3
Naive
71.28
33.86
84,50
57.86
40.93
83.33
61.56
61.90
Single
76.36
39.72
84.72
58.97
36.53
76.67
55.85
61.26
M.
75.86
36.92
87.36
60.24
39.95
88.33
56.58
63.61
LaMer (Ours)
Yos
78.15
40.85
86,41
60.24
39.69
88.33
59.22
64.70
Base
71.88
42. 10
85.48
59.53
37.58
85.00
61.79
63.34
AugGPT
Yes
79.41
42.71
89.78
63.66
40.00
90.00
14.75
60.04
Qwen2
Naive
Yes
75.62
42.80
88.40
62.50
39.34
86.67
61.82
65.31
7B
Single
Yes
76.41
44.30
87.64
61.98
39.69 69
80.00
56.62
63.81
LLM2LLM
43.74
63.21
88
60.09
LaMer (Ours)
Yes
78.13
45.02
88.62
62.48
40.56
86.67
61.44
66.13
Base
53.26
25. 80
49.97
33.73
23.29
38.33
7.08
33.07
AugGPT
Yes
53.42
25.76
47.73
33.69
24.31
36.67
3.95
32.50
Gemma-I.1
Naive
Yes
55.38
25.78
47.79
32.23
24.24
36.67
5.99
32.58
28
Single
Yes
52.85
24.51
46.67
34,00
25.03
31.67
5.86
31.51
LLM2LLM
35.77
LaMer (Ours)
Yes
55.81
25.81
48.91
34.40
25.26
41.67
6.69
34.08
Table 6: The size selected knowledge deficiencies.
LLMs
Easy
Normal
Hard
Unfair
Mistral
375
375
375
375
LLaMA-3
125
125
125
125
Qwen2
375
375
375
375
Gemma-1.1
375
375
375
375
Gemma- 1.1
user
Question:
{
Options: model
E SIZE OF SETECTED KNOWLEDGE DEFICIENCIES
The size of selected knowledge deficiencies of each LLM each group can refer to Table
Table 7: Statistics of e-CARE
Dataset
Train
Dev
Test
Total
c-CARE
14,928
2.132
4.264
21,324
18

---

Published as a conference paper at ICLR 2026
Table 8: Ablation study results of the effect of the curricular deficiency remedy, LaMer* denotes that we train corresponding LLMs with randomly shuffled data. Bold numbers denote better performance between LaMer and LaMer".
LLMs
Size
Methods
Comm.
AG1Eval
ARC
MMLU
BBH
CRASS
GSM-Plus
Average
LaMer LaMer"
75.10
34.50
77.22
54.52
33.72
88.33
29.23
56.09
Mistral
7B
74.90
34.07
77.48
54.30
33.66
86.67
28.42
55.64
LaMer
78.15
40.85
86.41
60.24
39.69
88.33
59.22
64.70
LLaMA-3
8B
LaMer"
77.67
40.93
86.28
60.07
38.47
88.33
59.55
64,47
LaMer
78.13
45.02
88.62
62.48
40.56
86.67
61.44
66.13
Qwen2
7B
LaMer*
77.51
44.33
88.36
62. 12
39.80
85.00
61.36
65,50
LaMer LaMer"
55.81
25,81
48.91
34.40
25.26
41.67
6.69
34.08
Gemma-1.1 2B
55.05
25.32
48.63
34.16
25.18
41.67
6.60
33,80
F
FURTHER ANALYSIS
F.1 DEFICIENCY DIAGNOSIS
We formally describe the baselines follows:
Golden Label adopt the labels judge the response of specific LLM, this LLM would answer each question in a chain-of-thoughh (Wei et al. 2022) way. If the LLM gives the wrong answers according to the labels on some examples, then the examples are treated as the deficiencies of this LLM. We treat this method as the golden standard for diagnosing the knowledge deficiencies of LLMs.
Perplexity computes the perplexity of each option based on a specific LLM, the option with the lowest perplexity is treated as the answer of the LLM, and then labels are introduced the judge the correctness of the L.M. Similar to Golden Label, wrongly answered examples are treated as the deficiencies of this LLM.
Random method randomly samples the examples from a dataset.
Relative Entropy is the method proposed in this paper.
We choose e-CARE (Du et al. 2022) as the dataset for experiments, which is an explainable causal
reasoning dataset with two options in each example. The whole set of e-CARE (train, dev, and test) is adopted for experiments. Statistics of e-CARE can refer to Table The LLM we used is
Mistral-7B-Innsrretvvd. (Jiang et al. 2023)
F.2 FULL RESULTS OF LLM2LLM
We provide the full results of LaMer, all baselines, and LLM2LLM in Table which demonstrates the advantages our proposed LaMer.
F.3 FULL RESULTS OF THE EFFECT OF CURRICULAR DEFICIENCY REMEDY
provide the full results of the effect curricular deficiency remedy Table which demonstrates the curricular deficiency remedy strategy.
F.4
VISUALIZATION OF GENERATED EXAMPLES
F.5 CAUSES FOR THE ADVANTAGES OF LAMER
To explore the potential causes for the advantages of LaMer, we visualize the data synthesized by baselines and LaMer into 2D space. Specifically. utilize FlagEmbedding (Zhang al. 2023a) to represent each example in the synthesized data, and then reduce the dimensionality to 2 with the help of t-SNE (Van der Maaten & Hinton! 2008). Finally, employ DBSCAN (Ester al. 1996)
19

---

Published as a conference paper at ICLR 2026
(a) Mistral
Nake / AugGPT
LaMer (Ouns)
t-SNE Feature
t-SNE Feature (b) LLaMA-3
C-SNE Feature
Naive AugGPP
LIMPLE
Figure 7: The distribution of synthesized data by baselines and LaMer on (a) Mistral and (b) LLaMA-
to discover clusters and remove the noise data points. The eps and min samples of DBSCAN are 1.5 and 3, respectively. We also adopt Mistral and LLaMA-3 for experiments. The visualization is shown in Figure 7 from which higher proportion at the outer edges of the whole distribution (LaMer and basclines). This might potential cause for the advancement of LaMer, since LaMer utilize relative entropy to effectively discover more deficiencies.
infer that, data synthesized by LaMer has significantly
20


Reviews:
[{"review_id": "RXsidSYRgJ", "reviewer": "Reviewer_jb7t", "summary": "This paper introduces a label-free framework for identifying and improving knowledge gaps in large language models (LLMs), which does not rely on costly human annotations. Specifically, the authors propose the relative-entropy-based diagnostic method that quantifies how much additional information external knowledge contributes to the LLM, which leads to detecting the areas of weakness. Then, the authors design the remedy process based on the curricular learning: synthesizing examples in proportion to deficiency severity and training the model from easier to harder deficiencies. The authors validate the proposed approach on four LLMs and multiple reasoning benchmarks, showing that it achieves consistent performance gains across them.", "questions": "Please see Weaknesses above.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The processes to identify and remedy deficiencies in LLMs are convincing. \n* The proposed approach clearly outperforms existing relevant baselines.", "weaknesses": "* In extracting the knowledge (needed to check and remedy deficiencies in LLMs), the assumption that, for each query, there should be relevant knowledge from an external knowledge base is very strong. In other words, what if the external knowledge base does not contain the relevant knowledge for each query? Additionally, the process of checking and remedying knowledge deficiencies can be done only for knowledge within the knowledge base, which seems a clear limitation of the proposed approach. Lastly, I am a bit confused whether the proposed approach is truly label-free: it requires the knowledge that is related and associated with the query, which may be considered as the label for the query. \n* There are relevant papers [A, B, C] that the authors should discuss and potentially compare with, especially [A] (which seems highly relevant). \n* The authors could more explicitly justify the advantage of the proposed approach over the unsupervised learning (or SFT) with experiments (i.e., the current setup does not fully justify its advantage over them, despite the claims in the paper). For example, the authors could train the LLMs with all the knowledge in the whole knowledge base and compare the proposed approach against it (i.e., the unsupervised setup) in both effectiveness and efficiency. \n* The performance of the baseline approaches on the Gemma-1.1 (2B) is inferior to the most basic setup (called Base), which may warrant more discussions. \n\n---\n\n[A] Structural Entropy Guided Agent for Detecting and Repairing Knowledge Deficiencies in LLMs, 2025.\n\n[B] R-Zero: Self-Evolving Reasoning LLM from Zero Data, 2025.\n\n[C] Self-Error-Instruct: Generalizing from Errors for LLMs Mathematical Reasoning, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n7F782o3vG", "reviewer": "Reviewer_85W6", "summary": "They propose a method to first find the knowledge deficiency in LLMs which, based on the literature, is the source of mistakes in reasoning. To do so they use user queries and using an external knowledge base, they get the relevant info for each query. Then using relative entropy, they detect the knowledge deficiency. Afterwards, using curricular meaningful learning, they propose a method to remedy this knowledge deficiency. This method is label free and does not require human supervision.\nThey show that this method achieves comparable results to baselines with only 40% training data.", "questions": "1. in the paper you mention that: \"Subsequently, we adopt ChatGPT (Achiam et al., 2023) to synthesize the specified number of examples for the deficiencies in each group.\". How do you make sure that chatGPT has enough and correct information for that knowledge?\n2. why in Gemma-1.1 your method beats others in most of the cases but with Qwen2, it does not?\n3. \"We only keep the examples that possess valid answers for evaluation.\" what percentage of answers did you throw away? and how do you define “valid” here?\n4. \"we use ChatGPT to generate m= 4 pieces of knowledge for GSM8K\". What if ChatGPT makes a mistake? how do you guarantee ChatGPT is correct?\n5. Are you sure numbers is Table 6 are correct?\n6. \"Finally, we synthesize 3,750 examples to enhance Mistral, Qwen2, and Gemma-1.1, while 1,250 examples are synthesized to enhance LLaMA-3 due to denser knowledge in it.\". Why do you use different numbers for LLaMA-3?\n7. \"Therefore, Single enhances Mistral, Qwen2, and Gemma-1.1 with 1,500 examples, and it utilizes 600 examples to enhance LLaMA-3.\". Is it fair to use different number of examples for LLaMA?\n8. \"Naive and Single could supplement some knowledge to them but cause them to forget more useful knowledge.\" Your method also supplies some knowledge but it does not hurt the numbers. why single hurts? how do you select the example for single one?\n9. In section 3.6, item (2), you claim that “Naive and Single could supplement some knowledge to them but cause them to forget more useful knowledge.”. why this forgetting does not happen in LaMer?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "No need for human annotation or labels.\nRich experiments\nClearly written and easy to understand", "weaknesses": "This method is limited to adding knowledge from GenericsKB.\nThey do not show how much of knowledge in GenericsKB are actually missing in the LLM they study (maybe for some of them the knowledge is already there but fine tuning only makes that knowledge sharp.)\nHeavy reliance on ChatGPT (if ChatGPT makes errors, their method will as well).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OqAzHWM1jK", "reviewer": "Reviewer_4XWw", "summary": "This paper introduces LaMer, a novel framework for diagnosing and remedying knowledge deficiencies in LLM without relying on labeled data. The core idea is to use relative entropy to identify knowledge gaps by measuring the change in an LLM's output distribution before and after being provided with relevant external knowledge facts. Based on the severity of the diagnosed deficiency, LaMer employs a curricular learning strategy to synthesize a varying number of diverse examples to progressively fine-tune the model. The empirical results are strong and consistent across four different LLMs and seven OOD benchmarks, demonstrating the effectiveness and efficiency of the proposed method.", "questions": "1. Regarding the knowledge retrieval process, how do you ensure the correctness of the knowledge recalled from the KB? Given that embedding-based matching can introduce noise, have you conducted any analysis on the impact of incorrectly retrieved or irrelevant knowledge on the RE calculation? How robust is the diagnosis mechanism to this noise?\n2. In the Section 2.2, you refer to \"The first situation suggests L might not grasp this knowledge or cannot properly apply this knowledge to problem-solving, while the second situation indicates that L does understand this knowledge but is easily misled by it.\", providing a specific interpretation for two scenarios of knowledge impact. Can you provide the reasoning behind the claim that when knowledge has a negative impact (misleading), it indicates \"L does understand this knowledge but is easily misled by it\"? Why does this scenario not also suggest a failure to properly integrate or contextualize new information, which could be seen as a form of \"not grasping\" the knowledge in a given context?\n3. Can you elaborate on the sensitivity of LaMer to the quality of its external components? Specifically, how would performance degrade if a less comprehensive knowledge base were used, or if a weaker, open-source model was used for data synthesis instead of ChatGPT?\n4. Regarding the RE thresholds in Table 1, have you performed any experiments to analyze their sensitivity? How were these specific values (0.1, 0.4, 0.7, 1.0) determined, and how critical are they to the overall performance of the curricular remedy strategy?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The primary contribution (using relative entropy for label-free diagnosis of knowledge deficiencies) is well-motivated and novel. It addresses a critical and practical challenge in the continuous improvement of LLMs: how to perform targeted enhancements without expensive human annotation.\n2. The integration of curricular and meaningful learning is well-executed. The paper clearly demonstrates that diagnosing deficiencies first and then applying a targeted, progressive remedy (from easy to hard) is more effective than naive data augmentation or random-order training.", "weaknesses": "1. The framework's effectiveness is highly dependent on two external components: a comprehensive knowledge base (GenericsKB) and a powerful teacher model (ChatGPT) for data synthesis. This reliance may limit its applicability in scenarios where a high-quality KB is unavailable or the cost of using a powerful synthesis model is prohibitive. The paper does not sufficiently discuss the impact of these dependencies.\n2. The RE thresholds used to categorize deficiencies into \"Easy,\" \"Normal,\" \"Hard,\" and \"Unfair\" are presented as heuristics. The paper lacks a rigorous justification or a sensitivity analysis for these values, making it unclear how robust the method is to these choices. \n3.  The diagnosis step relies on knowledge retrieved via embedding similarity, which can be noisy and sometimes irrelevant. The paper does not address how such noisy knowledge might affect the stability of the posterior distribution Q and the resulting RE calculation. This raises concerns about whether the RE score is always a reliable indicator of a true knowledge deficiency.\n4. The methodology description contains ambiguities that hinder clarity. For instance, the paper refers to the \"negative log-likelihood (NLL) of each response oi conditioned on x,\"(Section 2.2) but 'x' is not defined in the context.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0alzCnbGzb", "reviewer": "Reviewer_oqoY", "summary": "The paper proposes LaMer, a label-free pipeline that (1) retrieves generic facts for each unlabeled query from an external knowledge base, (2) diagnoses “knowledge deficiencies” by comparing a model’s predictions with and without the retrieved fact, and (3) remedies those deficiencies by synthesizing training examples whose quantity scales with the diagnosed severity and training in an easy-to-hard curriculum. Using open-weight models, the method reports consistent gains over several augmentation baselines on seven out-of-distribution reasoning benchmarks, often matching or surpassing baselines with about 40% of the training data.", "questions": "- I am a bit curious about the retrieval robustness, how sensitive is the deficiency diagnosis to the retrieval pipeline (embedding model choice, number of facts, KB domain coverage)? What happens if retrieved facts are partially wrong or overly generic?\n- How did you select the deficiency threshold, and how stable are results across plausible ranges? Could you replace hard thresholding with an adaptive percentile per domain or model?\n- How would you adapt LaMer when queries are private or streaming (e.g., enterprise chat), and when external KBs are domain-specific or scarce?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper ties together retrieval, a simple distribution-shift diagnostic, and a curriculum that scales data to severity, turning unlabeled user queries into targeted training data without manual labels. The case study and “remedied examples” analysis make the mechanism concrete and inspectable.\n- The paper contrasts label-free deficiency detection against perplexity and a label-reliant data-mining baseline, studies the curriculum order (vs. shuffling), and separates “helpful” vs. “misleading” retrieved facts—showing both expose repairable deficiencies. \n- It specifies the KB (GenericsKB with confidence filtering), retrieval embedding (FlagEmbedding), the number of matched facts per query, the synthesis protocol, and PEFT training settings, which lowers the barrier to trying LaMer in practice.", "weaknesses": "- The diagnosis hinges on retrieved facts being relevant and on a chosen threshold to flag deficiencies. If retrieval drifts (spurious or overly generic facts) or if the threshold is mis-set, the pipeline may teach to noise. \n- The severity buckets and the number of synthesized examples per bucket are fixed heuristics. While effective, it maybe interesting to explore the direction of an adaptive curricula (e.g., stopping rules per deficiency) or data-budget trade-offs across tasks and models. \n- While “remedied examples” are counted, there’s no probing of whether the method inadvertently forgets useful adjacent knowledge or whether it changes calibration/uncertainty properties.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)