Reliable and Diverse Evaluation of LLM Medical Knowledge Mastery
Authors: Yuxuan Zhou
Xien Liu
Chen Ning
Xiao Zhang
Ji Wu
Venue: ICLR 2025 Poster (2025)
Keywords: LLM Evaluation
Medical Evaluation
Large Language Model
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose a reliable and diverse evaluation method, aiming to probe the medical knowledge mastery of LLMs.

Abstract: Mastering medical knowledge is crucial for medical-specific LLMs. However, despite the existence of medical benchmarks like MedQA, a unified framework that fully leverages existing knowledge bases to evaluate LLMs' mastery of medical knowledge is still lacking. We propose PretexEval, a novel framework that dynamically generates reliable and diverse test samples to evaluate LLMs for any given medical knowledge base. We notice that test samples produced directly from knowledge bases by templates or LLMs may introduce factual errors and also lack diversity. To address these issues, our framework employs predicate equivalence transformations to produce a series of variants for any given medical knowledge point. Finally, these produced predicate variants are converted into textual language, resulting in a series of reliable and diverse test samples. Here, we use our proposed framework to systematically investigate the mastery of medical factual knowledge of 12 well-known LLMs, based on two knowledge bases that are crucial for clinical diagnosis and treatment. The evaluation results illustrate that current LLMs still exhibit significant deficiencies in fully mastering medical knowledge, despite achieving considerable success on some famous public benchmarks. These new findings provide valuable insights for developing medical-specific LLMs, highlighting that current LLMs urgently need to strengthen their comprehensive and in-depth mastery of medical knowledge before being applied to real-world medical scenarios.

Published as a conference paper at ICLR 2025
RELIABLE AND DIVERSE EVALUATION OF LLM MEDI- CAL KNOWLEDGE MASTERY
Yuxuan Zhou', Xien Liu"!, Chen Ning', Xiao Zhang', Wu 1 Department of Flectronic Engineering, Tsinghua University, Beijing, China BBRist, Beijing, China
2 College of AI, Tsinghua University, Beijing. China
ABSTRACT
Mastering medical knowledge is crucial for medical-specific LLMs. However, despite the existence of medical benchmarks like MedQA, a unified framework that fully leverages existing knowledge bases to evaluate LLMs' mastery of med- ical knowledge is still lacking. We propose PretexEval, novel framework that dynamically generates reliable and diverse test samples to evaluate LLMs for any given medical knowledge base. We notice that test samples produced directly from knowledge bases by templates or LLMs may introduce factual errors and also lack diversity. To address these issues, our framework employs predicate equivalence transformations to produce a series of variants for any given med- ical knowledge point. Finally, these produced predicate variants are converted into textual language, resulting in series of reliable and diverse test samples. Here, we use our proposed framework to systematically investigate the mastery of medical factual knowledge of 12 well-known LLMs, based on two knowl- edge bases that are crucial for clinical diagnosis and treatment. The evaluation results illustrate that current LLMs still exhibit significant deficiencies in fully mastering medical knowledge, despite achieving considerable success on some famous public benchmarks. These new findings provide valuable insights for de- veloping medical-specifie LLMs, highlighting that current LLMs urgently need to strengthen their comprehensive and in-depth mastery of medical knowledge before being applied to real-world medical scenarios.
INTRODUCTION
Recent years have witnessed the rapid advancement of large language models (LLMs). which have exhibited potential across various domains (Brown et al. 2020 Ouyang et al. [022] Touvron et al., 2023 APenAl 2023 Madani al. 2023 Boiko et al.) 2023), meluding medicine. Solving medical problems requires LLMs to master medical factual knowledge comprehensively and in-depth. Re- cent studies (Singhal et al. 2023: Nori et al. 2023 Pal Sankarasubbu (024) showed that some LLMs (e.g. GPT-4) encode medical factual know knowledge, achieving outstanding performance across multiple medical benchmarks (Jin et al. 2019 2021 Pal at 2022 Singhal et al. 2023 (ung et al.)
2021 Meng et al. 2022), such as MedQA. Constructed through expert annotation, these benchmarks have long been effective tools for evaluating LLMs' medical capabilities. However, they may face challenges such as becoming outdated or being possibly leaked to LLMs, which could lead to eval- uations that lack reliability. Meanwhile, medical databases such as UMLS (Bodenreider) (004)) are regularly updated and contain extensive medical knowledge, but there is currently no unified frame- work that fully leverages these knowledge bases to evaluate LLMs' mastery of medical knowledge. Therefore, we aim to bridge this gap this study by proposing an evaluation framework that inves- tigates LLMs' medical knowledge mastery based on any given medical knowledge base.
Evaluating LLMs using medical knowledge bases requires generating textual test samples from structured knowledge. A straightforward method is to prompt L.MMs to directly generate test samples based on specific knowledge points. However, this method has two drawbacks as illustrated in Figure [ (1) insufficient factuality: factual errors (e.g. incorrect relations) may be introduced during LLM generation process, affecting the reliability of evaluation; and (2) low structure diversity: samples
"Correspondigg author.

---

Published as a conference paper at ICLR 2025
Insufficient Factuality:
Test variants dynamically generated by PretexEval Predicate Equiveient Transtormi
Oynamic
infordt
FFocal inferciion is Prequentyy assomparied microiefarcte se?
Textual variants
Merphology
Wrong
Low Structure Diversity:
Flulamédo commoniy preseried bed nirthess Iisutssm", Hue of filse?
Plllaidee
Dynamio Conecation
VS: Diversity & Reliability
"The medication Futamikdo la Indictted for the treementt rsuisum Fue of fasse?
reshicectrrs
Prompts
"Fullaniid is known for It efteacy in mansignng Nisutiam,, sue false?
Hirsutiom
Knowladge
Generated Questions
Test variants directly generated by LLMs
Figure I: Drawbacks of test samples produced di- rectly by LLMs: (1) LLMs may introduce factual errors into generated samples; (2) Samples directly generated by LLMs exhibit low diversity.
Figure 2: Schema of the proposed Predicate- to-text evaluation method (Top) compared with directly generating test variants by LLMs (Bottom).
generated from the same knowledge point differ primarily in wording (e.g. synonym replacement) rather than in expression structure, compromising the diversity of evaluation.
The purpose of this study is to develop a unified evaluation framework that dynamically generates reliable and diverse test samples from medical knowledge bases to probe LLMs' mastery of medical factual knowledge. Given that medical factual knowledge primarily involves relationships between medical entities, it can be effectively expressed through predicates. Inspired by this, we propose a predicate-to-tex. Evaluation method (PretexEval) that dynamically generates reliable and struc- turally diverse test samples based on the medical knowledge points from knowledge bases. Figure 2 presents the schema of our method. Specifically, we first express each knowledge point using a predicate expression. Then, we derive a series of predicate variants from this predicate expression through predicate equivalent/mmplictioon transformatior[] Such predicate transformation process enhances the structural diversity of generated test samples and also effectively prevents the intro- duction of factual errors. Finally, we use prototype-basee method to convert these variants back into the textual space to create the test samples. This method first transforms the predicate variants into prototype samples using templates to ensure reliability, and then rephrases these prototypes with LLMS to enhance the readability and lexical diversity of the generated samples. It is worth noting that the proposed evaluation method is highly versatile and can be applied any medical knowledge base with minimal adjustments to evaluate LLMs' mastery of the knowledge it contains.
In our study, we employ the proposed evaluation framework to systematically investigate the mas- tery of medical knowledge among 12 well-known LLMs, using two medical knowledge bases that contain essential information for clinical diagnosis and treatment. Experimental results indicate that the performance of current LLMs on test samples generated by our method is significantly lower than on samples directly generated by handcrafted templates or by prompting an LLM. Further- more, these evaluated LLMs exhibit notable inconsistency in handling test samples derived from the same knowledge point. These findings indicate that, despite their impressive performance on sev- eral medical benchmarks, current LLMs have not fully mastered the medical knowledge essential for real-world clinical tasks. Therefore, they may require additional training before being applied in real-world medical scenarios to further enhance their mastery of medical knowledge. We release the codes and datasets to facilitate future study?! Our contributions are summarized as follows:
We introduce PretexEval, a predicate-to-text method that dynamically generates reliable and structurally diverse test samples based on any given medical knowledge base.
Using the proposed method, we systematically investigate the medical factual knowledge mastery of 12 well-known LLMs based on two knowledge bases closely related to clinical diagnosis and treatment.
'For the sake convenience, we refer to both equivalent and implication transformations collectively as predicate equivalent transformations in the following sections, without making a distinction between them. https: // thub.
2

---

Published as a conference paper at ICLR 2025
Our findings reveal that current LLMs have not yet comprehensively and deeply mastered medical knowledge, underscoring the urgent need to improve their medical knowledge mas- tery before applying them to real-world medical tasks.
2 RELATED WORK
LLM Medical Evaluation Current medical evaluation benchmarks for LLMs can be divided into Two categories: (1) QA datasets that evaluate L.MSs' comprehensive medical capabilities with ques- tions collected from medical literature (Jin et al. 2019). exams (Jin et al., [202]] ; Pal et al. 20022). or online websites (Singhal al. (023);: (2) datasets for testing the mastery of LLM medical knowl- edge (Sung et al.J [022 Meng et al. 2022). These static benchmarks are meticulously created by medical experts and possess high reliability. However, they may face problems such as becoming outdated or leaked to LLMs, affecting the comprehensiveness of evaluation. While constructing new benchmarks can alleviate these problems, they will also become obsolete over time.
Dynamic Evaluation Schema Several studies have proposed dynamic evaluation methods that automatically generate new test samples, effectively addressing the issue of unreliable evaluation caused by LLMs memorizing test samples (benchmark leakage). Some works leverage algorithms to dynamically generate test samples for specific tasks, such as mathematics (Zhu et al. (0244) and SQL execution (Lei et al. (023). Others (Zhu et al. 2024c b) generate test samples by paraphrasing existing benchmarks. However, currently there is no related work that uses dynamic evaluation methods evaluate LLMs based knowledge bases. By dynamically generating test samples from regularly updated medical knowledge bases, our framework can effectively address issues associated with static evaluation benchmarks (becoming outdated, being leaked L.L.Ms).
METHOD
3.1 EVALUATION SCHEMA
In this section, we introduce the schema of our PretexEval method, which generates structural di- verse and reliable test samples for LLM factual knowledge evaluation. Given knowledge point P, a straightforward idea to directly generate a test sample using an LLM:
(1) where GLLM denotes the LLM generation process, and S refers to the generated test sample. As introduced above, GLLM may create samples that lack diversity and reliability. In contrast, our method first expresses the knowledge point using predicate expression and then derives a series of variants via predicate equivalent transformation:
$ (P)
q T 1exe (P)
(2)
[v1, V2. .ml = (xy
(3)
where Treathpre denotes mapping that projects the original knowledge point P into the predicate expression TEy refers to the predicate equivalent transformation, and {v are the variants derived from the original expression q. The property predicate equivalent transformation ensures the reliability of these variants, provided that the original expression q is true:
(q True) + (v, = True), 1 1 m
(4)
Finally, convert each predicate variant back to textual test sample for evaluation:
S = T Tpe) 1 i m
(5)
where Tpre2tent maps each predicate variant v, into a corresponding test sample (textual variant). Since these samples derived from predicate variants with diverse structures, the predicate-text duality ensures they exhibit substantial diversity while maintaining reliability.
3.2 EVALUATION FRAMEWORK
Building the proposed evaluation schema, develop novel evaluation framework compre- hensively evaluate LLMs' mastery of medical factual knowledge. B presents an overview of this framework.
3

---

Published as a conference paper at ICLR 2025
Predicate Equivalent Transformation
Prototype-baser Sample Generation
C( MT (A,
cannot treat
Als incapable ra reging B. Thue False
P
treal
True False
MT (A 80
(BLA)
MT (x:00
y may be Frooed by
Ii may be treated by
may possibly be managed by True False
(A, may tear,
Predicate
...
True False
...
Rephrase
LLM
...
D(x MC(A.x)
MCCK,2)
may cure
i potfent hos B.A may cure them True Fatso
Fa patient suffere from Amy cure them. True False Tast Samples
Knowledge Base
Predicato Voriants
Prototype Pool
Prototype Samples
An overview of the proposed PretexEval framework, which dynamically generates test samples from any medical knowledge base for evaluating LLMs' medical knowledge mastery.
Figure 3:
3.2.1 PREDICATE VARIANT GENERATION
A single knowledge point (i.e. knowledge triplet in knowledge bases) be denoted as P (,,r,t). where h, r, and t refer to the head entity, the relation, and the tail entity, respectively. In predicate logic, such relation can be effectively presented by:
R(h,t)
(6)
where R(x,y) y) is predicate derived from the relation т. representing the statement "x has the relation r with y", where q represents its value at the point (h, 1). Next, the framework employs three types of equivalent transformations that are widely employed in practical medical applications, including:
Inversion (R (R, h))): The inverse expression presents the original expression from an- other direction. For example, if the statement "Drug A may treat disease B" holds, then "Disease may be treated by drug A" also holds.
Instantiation (P(h, x) = Q(x, t)): This type of transformation applies a general knowl- edge point to a specific case. For example, the statement "Drug A may treat disease B" can instantiated as "If a patient has disease B, drug A may cure them." Such transformation is commonly used in disease diagnosis and treatment.
The double negation rule is widely utilized to obtain log- ically equivalent expressions. In our framework, this rule applied to construct negative expressions. For example, if "Drug A may treat disease B" holds, then "Drug A cannot treat disease B" must be false.
Double Negation (-(-R(b, t))):
noteworthy that these three types of transformation can be further combined produce ad- ditional expressions based on the transitive property of predicate equivalent transformation. As result, a total of m variants are generated in this process:
Vi = (R(h,) 1 ssism
where Th denotes the 7th predicate cquivalent transformation.
3.2.2 TEXTUAL SAMPLE GENERATION
Finally, the predicate variants are converted back into textual samples for LLM evaluation. A straightforward method is template-bas based rephrasing, but the resulting samples may lack fluency, potentially impacting the model's performance. Another approach is to prompt LLMs to generate test samples directly from predicates. However, considering that LLMs may not fully understand predicate forms, this method can also introduce factual errors. To address this issue, we designed (R((,,
a prototype-basee based sample generation method. Specifically, for each predicate variant (R (h, t)), initially retrieve the corresponding prototype from pre-constructed prototype pool based on the predicate T RR For example, considering the predicate variant MT (y, x) in Figure 3 we trieve the corresponding prototype "y may be treated by x". For predicate variants obtained through double negation, we retrieve prototypes based their negated form (i.e., single negation form) to
4

---

Published as a conference paper at ICLR 2025
generate negated samples for LLM evaluation. Subsequently, the prototype is instantiated by the arguments (h, t). The instantiated prototype precisely conveys the predicate variant in the textual space. Finally, the prototype is further rephrased by an LLM to obtain the final test sample S, Since current L.LMs possess strong language capabilities and seldom make mistakes in sentence rephras- ing, the proposed sample generation method can ensure the reliability and diversity of the generated samples.
3.2.3 EVALUATION METRICS
In our framework, we mainly evaluate LLMs using statement verification tasks, asking them to determine whether a given statement is true or false:
(8) where M is the evaluated LLM, S; is the textual variant (statement) generated by our framework, and M(S.) E {T.F) denotes LLM's prediction for S,. 4, {T. is the label of S,, and the function 1(.) is a characteristic function that equals 1 when the enclosed expression is true, and otherwise. Additionally, we also adopt 4-option multiple-choice questions to further validate the scalability of PretexEval on various evaluation tasks (see Section 4.2.4).
Score[M, S.) 1 (M(S,) 4.), sis m
For dataset with knowledge points {P,)}=ı, we initially use the metric average accuracy to compute the accuracy across all test samples:
1
Score St)
Gavg
12
where S/ denotes the 7th test sample derived from the 5th knowledge point P,. While this metric is widely applied in various benchmarks, it cannot evaluate the consistency of LLMs in predicting all test samples derived from the same knowledge point, which is crucial for high-risk applications in the medical domain. Therefore, also utilize another metric, joint accuracy, which considers knowledge point as mastered if all the related samples are predicted correctly:
Ajoint II
Score(M, S()
(10)
==1=1
4 EXPERIMENTS
4.1 EXPERIMENT SETUP
Datasets Introduction
To validate the effectiveness of our proposed framework PretexEval, we conduct systematic evaluation on LLMs' medical knowledge mastery with PretexEval using two knowledge bases: biomedical knowledge base MedLAMA (Meng et al. 2022) and a clinical knowledge base DiseK (Zhou et al., 2024). MedLAMA a large-scale biomedical knowledge base consisting of 39,053 knowledge triplets that encompass 19 relations among medical entities such as discases, genes, cells, and tissues, all meticulously selected from the UMLS Metathesaurus (Bodenreider 2004 ensure high quality. DiseK is clinical knowledge base that contains 24,413 triplets, covering 1,000 high-frequency diseases across four key relations related to disease diagnosis and treatment. Mastering the knowledge contained within these databases is essential for LLMs to be effectively applied in real medical scenarios. It is important to highlight that our framework can also be applied to other medical knowledge bases with minimal adjustments to evaluate medical knowledge of other types (e.g, drug- related knowledge); we leave this as future work.
Considering computational costs and dataset size, we select a subset from each dataset for evalua- tion. Specifically, we randomly select a single entity from the corresponding tail entities for each pair of a head entity and a relation. This approach aims to reduce the evaluation scale while max- imizing the diversity of the evaluated knowledge. We also excluded two relations in MedLAMA, which are the inversion of the other two relations in MedLAMA. Furthermore, for each head-relation pair (h, r), we randomly sample a negative entity that satisfies - (R,, c) to create negative triplet (h, r, c). Test samples generated from this negative triplet possess a similar structure to those gen- erated from the positive triplet but with opposite labels. By introducing negative triplets, we can further evaluate the ability of LLMs to discern knowleeee,d which is also essential for practical application. Appendix A provides detailed statistics of knowledge bases and relation types within.
5

---

Published as a conference paper at ICLR 2025
MedLAMA
DiseK
Model
Direct
LLMEval
PretexEval
Direct
LLMEval
PretexEval
Llama2-78 7B
+6.4
+8.3729.7%
+5.1%
+11.7
+2.7
+2.8176.3%
Vicuna-7B
+26.4
+18.0g31.7%
171.5%
+9.9
+10.919.7%
10.5% (0.5%
Vicuna-133 13B
+27.0
+19.3
+ +10.7
+12.5
+7.4 140.2%
+5.7 15399%
Gemma-7B
+23.3
+11.1152.3%
+9.4..59.5%
+9.0
146.5%
+5.0
Llama3-88
+28.5
+39.1
+ +16.6141.8%
+17.9
+15.3114.35
+9.3
Llama2-70B
+32.0
139.2139.9%
+ +1..8 56.9%
+20.5
117.3115.7%
+9.0
ClinicalCamel-70B
+34.8
+23.7
+21.9 137.2%
+24.5
+20.6
+ +16.
Meditron-70B
+29.4
432.0132.14
+ +14.7 (49.8%
+21.1
+12.8139.4%
+ +10.2.51.59
Med42-70B
+31.8
+19.3.39.3
+20.0
+23.3
+19.1118.198
+14.8 +14.8.J36.69 (36.66%
Llama3- 7OB
+36.6
+26.9
126.9% 126.5%
+26.9 226.6%
+29.7
+28.2
+20.9 229.7% 12.79.
GPT-3.5-turbo
+32.1
+26.7
+26.71166995
116.9%
+16.2
49.7%
+23.5
+17.6125.49
+10.3, 156.4%
GPT-4o"
+35.8
+34.0
14.9%
431.7 11.5%%
+31.3
+29.9
14.3%
+26.7 (14.5%
Table I: Performance (average accuracy) of LLMs evaluated on datasets directly generated by template paraphrasing (Direct), datasets directly generated by LLM (LLMEval), and datasets gen- crated by our framework (PretexEval). We report the gain relative to random guessing (50%) and the relative performance degradation compared the Direct results. Bold: Best performance under the same evaluation method; Underline: LLM achieved the lowest performance on this evaluation method. +GPT-4 is evaluated on sampled subsets for cost considerations.
Method Setting To ensure the diversity of evaluation, we combined the three types of predicate transformation and generated m 8 expressions (variants) for each knowledge point, including the original expression. We crafted prototype for each combination of relation and predicate transformation type to generate test samples. Moreover, we utilize Llama3-70B-Instrun (AI@ Meta 2024) to rephrase the instantiated prototypes because of its strong performance. We have also tried other rephrasing LLMs and obtained similar evaluation results (see Appendix BB. More details of the predicate transformation, prototypes, and the prompt format provided in Appendix C
For LLM evaluation, we employ the popular 5-shot in-context learning setting (Brown et al. 2020 where five examples presented before the test sample, guiding LLMs to produce answers sistent format with the provided examples. We calculate the average and joint accuracies (introduced for each LLM. Appendix provides more details, including the prompt format.
in See 3.2.3)
Baselines We initially compare our method with the method that directly generates test samples by paraphrasing the knowledge with templates (denoted Direct). We also implemented a dynamic evaluation baseline (named as LLMEval) that directly generates test samples from triplets using an L.M.. Specifically, we prompt Alama3-70B-Instruu
to generate m 8 statements, presenting the given triplet in different ways. We carefully crafted the prompt to ensure maximum diversity in generated samples. Appendix provides more details of these two baselines.
Evaluated LLMs
In our study, evaluate 12 well-know LLMs: (1) general LLMs: Gemma- -B
(Tea et al. 2024), Llama2 (7B,70) 70B) (Touvron et al. 2023), Llama3 (8B,70B) (AI@ 2024).
Vieuna (7B, 13B) (Zheng et al. 2023) GPT-3.5-turbo (Oyyang et al.) 2022). and the latest GPT-4o
OpenA 2024); (2) Med42-70B (Christophe et al., (023). For considerations, we evaluate
LLMs: ClinicalCamel-700 (Toma et al. 2023). Meditron-
Chen al.J [2023) and
GPT-4 on sampled subset containing 200 knowledge triplets for each dataset.
4.2
RESULTS
4.2.1
COMPARISON STUDY
We first conduct a comparison study across different evaluation methods and LLMs. Table [ lists LLMss performance (average accuracy) the MedLAMA and DiseK knowledge bases evaluated by different methods. We also conduct fine- Ene-grained analysis on LLMs performance across knowl- edge types, which is provided in Appendix F due the space limit. The results demonstrate that all
We choose the I.IM utilized in our framework make fair comparison.

---

Published as a conference paper at ICLR 2025
MedLAMA
DiseK
0.8
0.8
LLMEval PretexEval
LLMEval PretexEval
Joint Accuracy
0.6
Joint Accuracy
0.6
0.4
0.4
0.2
0.2
0.0
0.0
2
3
6
7
8
1
2
5
6
0.8
0.8
Joint Accuracy
Joint Accuracy
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
2
3
2
5
5
Number of Expressions
Number of Expressions
LLMEval:
Llama2-78
Gemma-78 Uamm33888
Llama2-708
ConnicalCamal.788
Laama37788 -o GGT-4o
PretexEval:
Llama2-70
70
Ulmma3-0
Llama2-700 70n
ClinicalCamel-7nn
Llama3-700
GPT-to
Figure 4: Performance (joint accuracy) of typical LLMs evaluated by increasing the number of expressions per knowledge point. Top: overall performance trend averaged across LLMs; bottom: detailed performance for each LLM. To eliminate the impact of sample addition orders, we enumer- ate all possible orders and averaged the results, where the value at corresponds to the expected joint accuracy evaluated with any i samples.
evaluated LLMs achieve much lower performance on datasets generated by PretexEval compared to the original datasets. This suggests that dynamically generating multiple samples for each knowl- edge point can significantly enhance the comprehensiveness of evaluation. Moreover, compared to datasets directly generated by an L.M (LLMEval), almost all LLMs achieve lower performance on datasets created by PretexEval, with some models (e.g. Llama2-7B and Llama2-70B) experienc- ing over 50% degradation. These findings suggest that PretexEval is capable of generating test samples that are more diverse than those directly generated by LLMs.
Among all the evaluated LLMs, GPT-4o outperforms the others across almost all datasets and eval- uation methods, achieving performance gains (relative to random guessing (50%)) of 31.7 and 26.7 evaluated by PvetexEval. On open-sourced LLMss, Llama3-70B and Llama3-8B performs best on -regeneevll datasets compared to L.MS with similar parameter scales. It is worth not- ing that Llama3-8B even slightly surpassing the 10x larger Llama2-70B. These results indicate that Llama3 model series encodes significantly more medical knowledge than other evaluated LLMs. Additionally, while some medical-specific LLMs (ClinicalCamel, Med42) perform simi- larly to their backbone model (Llama2-70B) on original datasets, they notably outperform the latter by around 7% on PretexEval- -eenrratd datasets. This suggests that training on medical corpora can notably improve the depth of medical knowledge mastery.
We also study the joint accuracies of LLMs evaluated by increasing numbers of expressions per knowledge point. The results of seven typical LLMs are illustrated in Figure with the full results provided in Appendix G.I We observe that the results from LLMEval and PretexEval are quite close when using single sample for evaluation. However, as the number of test samples increases, the difference between the results from the two methods grows notably larger. This phenomenon indicates that current LLMs generally exhibit significant lower consistency when confronted with structurally diverse test samples generated by our method compared to samples directly generated by LLMss. Moreover, as the number of expressions increases, GPT-4o and Llama3-70B
7

---

Published as a conference paper at ICLR 2025
Knowledge Base Method
ClinCamel-70B
Llama3-70B
GPT-40
PretexEval (Ours)
+21.9
+26.9
+31.7
MedLAMA
w/o Predicate Transformation
+30.6
+33.0
+36.0
w/o LLM Rephrasing
+22.8
+30.4
+33.8
PretexEval (Ours)
+16.1
+20.9
+26.7
DiseK
w/o Predicate Transformation
+23.1
+27.8
+29.3
w/o LLM Rephrasing
+18.0
+24.0
+30.4
Table 2: Ablation results of three typical LLMs for key components of the proposed PretexE- framework. Predicate Transformation: the predicate equivalent transformation module; LLM Rephrasing: the LLM rephrasing module in the aetotype-based generation module.
Knowledge Base Transformation Type
ClinCamel-70B
Llama3-70B
GPT-4o
Direct
+30.6
+33.0
+36.0
+ +Inversion
+30.3
+31.8
+34.3
MedLAMA
+nverriionDDobble Negation
+23.2
+28.6
+33.6
+All
+ +14.7
+26.9
+31.7
Direct
+23.1
+27.8
+29.3
+ +Iversion
+22.4
+27.5
+29.8
DiseK
+nversion+Double Negation
+17.9
+22.3
+26.8
+All
+16.1
+20.9
+26.7
Table 3: Ablation results of three typical LLMs for different predicate transformations in PretexEval.
Each row represents a cumulative experiment, adding one transformation type a time, with "All" indicating the combination of instantiation, inversion, and double negation.
exhibits slower decline in performance compared to other LLMs, indicating a more consistent understanding of diverse expression structures from the same knowledge points. Nevertheless, there still room for improvement in current LLMs'' mastery of medical knowledge.
4.2.2 EFFECTIVENESS ANALYSIS
Effeet of framework components First, we conduct an ablation study to analyze the contribution of each component in our proposed framework. Table 2 presents the ablation results of three typical
LLMs, and the full results are listed in Appendix G.2 Here, we focus on the predicate equivalent transformation and the LLM rephrasing process in the prototype-based generation module that designed to increase the diversity of test samples. We observe that removing these two modules results in higher evaluation performance, especially when the predicate equivalent transformation module was removed (around 7% on Llama3-70B) These results indicate that the predicate equiv- alent transformation contributes most to the evaluation diversity in the proposed framework.
Effeet of Predicate Transformation Types We further conduct fine-grained analysis of the predicate transformation types applied our framework, with results presented in Table Exper- imental results show that LLM performance continually declines types are added, indicating their effectiveness. Furthermore, the inclusion double negation (+DN) leads to significant performance degradation (around 5%) than other implication types. This suggests that current LLMs exhibit relatively less proficiency in understanding negated expres- sions compared to instantiated and inverted statements of medical knowledge.
predicate transformation
Reliability & Diversity of Generated Samples We further conduct a human analysis to investi- gate the reliability and diversity of samples generated by different methods. Specifically, we ran- domly sample 50 knowledge triplets from MedLAMA and have three experienced doctors to score the test samples regarding their lexical diversity, structural diversity, and reliability by comparing with the original knowledge triplet. The analysis results and examples in different grades are illus- trated in Figure with more details (scoring criteria) of this analysis provided Appendix H We observe that, before the rephrasing process, the prototype samples generated by PretexEval exhibit high structural diversity and reliability but have lower lexical diversity compared to other methods. Although the samples generated by L.LMEval achieve relatively high lexical diversity, they signifi-
8

---

Published as a conference paper at ICLR 2025
5.0
Diversity Type
Knowledge Point
Refractory
present
presence
5.0
Levical Divenity
may have findings
HMB 46 Postive
Neoplastic Cells
anama
Nacpiealic Calla
Rail auuforc Non-
pattents
Melanoma.
Outanoous
finding
Reiractory Non-
Fresent
associate
Cutanaous
Melanoma
mecitant nen-ouleeeeus
melanoma that does not
45-cosilven
respond treatment.
may
development
Tubular Breast
Tuteder Anstomic sites
Levical Diversity
Structural
Relabilty
Diverity
cardiomm
disorder
Tubular
affacted
Cardinomaa patient
presence Tubuiar
LLLMEval
PretexEval
PreteeEval (wio Rephrasing)
Breast
breast
Figure 5: Left: Results of the human analysis on the reliability and diversity (lexical, structural) of samples generated by different methods; Right: Text examples in different grades of diversity.
cantly lack structural diversity and are also less reliable than the prototype samples. After rephras- ing, the PretexEvva-geeneratee samples maintain high structural diversity and reliability, while also achieving much higher lexical diversity. Our findings indicate that the proposed PretexEval method capable of generating reliable and diverse test samples based on knowledge bases.
Knowledge Point: (ribollavin tetrabulyrate, may treal, riboflavin deficiency)
PretexEval (Ours)
L.MEval
Question (Instantiate]:
Question (Instantiate Inverse]: "In the event rioflavin deficienzy
Question:
Question::
indministraiion rtoollewin totrabutyrate
"Riboflanin tetrabutyrote has been shown
"The administration of riborlavin tetrabutyrare yoonee may help alleviate
onables patients address rbollaxin ceficiency. true talse?
diagnosi, supplomentation with ribollavin lot tarrabuyrate recommenced for the palien", true faise?
be effeative treating riboflavin doficioncy, true or faise?
riboftavin daficiency true or faise?
Label: True
Label: True
X
Label: True
Label: True
Llama3-708 Answer: True
Llama3-708 Answer: False
Liema3-70B Answer: True
Llama3-70B Answer: True
Figure 6: A case of evaluating LLMs using PretexEval compared with the LLMEval method.
4.2.3 CASE STUDY
We also conduct case study on our evaluation. Figure 6 iillustrates the text samples generated by PretexEval in comparison with those generated by the LLMEval method, along with the LLMs' responses. The case shows that the proposed PretexEval framework generates test samples that are much diverse than those directly generated by LLMs, enabling a more comprehensive evaluation.
4.2.4 GENERALIZABILIT STUDY
Scalability Across Evaluation Tasks To validate the scalability of PretexEval for different types of evaluation tasks, we also generated multiple-choice questions (which is widely adopted in cur- rent benchmarks) using PretexEval for evaluation. The experimental results (Figure (7) show trends similar to those observed in the statement verification evaluation, demonstrating that PretexEval can integrate with various task types to accurately evaluate LLMs' medical knowledge mastery.
Applicability for Model Training Finally, conducted preliminary study explore the po- tential of improving LLMs' medical knowledge mastery through training on PretexEval-genenaa samples. Here, we mainly focus two research questions: RQI: Can training PretexEval- generated samples improve LLMs' understanding of k''oolledgg outside the training set? RQ2: Can training on a few types of PretexEval -eeeraa-ee samples improve LLMs' understanding of other unseen expressions? For RQ1, we selected 200 knowledge triples as the training set and another 200 triples as the test set. We finetune Llama3-8B using all PretexEval-geenratee from the training set, and apply PretexEval for evaluation on the test set. Experimental results in Figure &h show that training on PretexEval-generaan performance (~20%) on all types of expressions derived from knowledge outside the training set. For RQ2, we randomly selected 4 out of types of cestexEval-generatee expressions for training, and apply the rest 4 types for evaluation. Figure 8p demonstrates that training PretexEval- -eeerrttd samples could largely improve LLMs' performance (~~30% on all of the un- seen expressions. These results suggest that training with PretexFival samples may help
samples derived
samples could significantly improve models'
few types of

---

Published as a conference paper at ICLR 2025
MedLAMA (Multiple-choice Question-answwrrngg)
1.0
1.0
LLMEval
PretexEval
0.8
0.8
Joint Accuracy
Joint Accuracy
0.6
0.6
0.4
0.4
0.2
0.2
0.0
1
2
0.0
1
2
Number of Expressions
Number of Expressions
LLMEval:
Llama2-78
Gemma-78
Liama3-88
Liama2-708
ClinicalCamel.708
Ulma3-708
GPT-4o
PretexEval:
Uama2-78
Gemma-78
Llama3-08
Liama2-708
C0lnicakCamel-708
Uama3-708
Figure 7: Performance (joint accuracy) of typical LLMs evaluated using 4-option multiple- choice questions. Left: averaged performance trend; Right: detailed performance for each LLM.
a. 1.0
b. 1.0
Average Accuracy
0.9
Average Accuracy
0.9
Expr4
Expr5
Expr6
Expr1
Expr3
Expr4
nBefore Training
After Training
Before Training
After Training
Figure 8: Comparison of LLMs before/after trained on PretexEval-generatee samples. Left: perfor- mance on knowledge outside the training set; Right: performance on unseen expressions.
to enhance LLMs' consistency in mastering medical knowledge. More details of this study provided in Appendix!!
CONCLUSION AND DISCUSSION
In this paper, we propose PretexEval, a novel evaluation framework that dynamically generates reliable and diverse test samples based medical knowledge bases to probe LLMs' mastery of medical factual knowledge. The proposed framework is highly generalizable and can be applied to any medical knowledge base with minor adjustments. We validated the effectiveness of PretexEval by conducting a systematic evaluation based on two medical knowledge bases. The experimental results show that the performance of current LLMSs evaluated by the proposed framework is much worse than their performance on public medical benchmarks. Furthermore, these LLMs exhibit in- consistency understanding diverse expressions derived from the same medical knowledge point. These findings suggest that current LLMs have not fully mastered medical knowledge, which may be one of the potential reasons for their insufficient performance on real-worle medical scenarios. We further explored the scalability of PretexEval across various evaluation tasks and its potential use for model training. Although PvetexEval could facilitate research on medical L.Mss, also has the following limitations: (1) while PretexEval can be incorporated with different tasks to evaluate LLMs' medical capabilities, it may not well-suited to integrate with some particularly complex med- ical tasks, such clinical diagnosis; (2) while the LLM rephrasing module in PretexEval effectively improves the readability of generated samples, it may also potentially introduce some uncertainty. We plan to further expand our evaluation framework in the future address these limitations.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This research was supported by Noncommunicable Chronic Diseases Science and Tech- nology Major Project (2023DD000650) and the Research Project on Large Language Model Long- Text Memory Processing Technology (funded by iFLYTEK Co., Ltd.).
REFERENCES
Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. ArXiv preprint, abs/2404.14119,, 2024.
URL httts:://axxii..ttt
abs 14219
AI@Metu. Llama 3 model card. 2024.
URL h.tpss//gsttubb..co///ttaaa--__
CARD . md
Olivier Bodenreider. The unified medical language system (umls): integrating biomedical terminol- ogy. Nucleic acids research, $2(suppL.11:D267-DD777 2004.
Daniil Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models. Nature, 624(7992):570-578 2023.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.). Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL http:://ppccocee.ss....o neurips wwc.ipps.cccpppe wttps/ccpppp////////aaae/ 1447c0ddbfcbb 96741 42
html
Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, et al. Meditron- 70b: Scaling medical pretraining for large language models. ArXiv preprint, 2023. Utt.htttss://axxt
htrgg//bs//// 16099
Clément Christophe, Avani Gupta, Nasir Hayat, Praveen Kanithi, Ahmed Al-Mahrooqi, Prateek Munjal, Marco Pimentel, Tathagata Raha, Ronnie Rajan, and Shadab Khan. Med42 a clinical large language model. 2023.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth Inter- national Conference on Leurning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. net wtt.://eoooomdddddeeeeeee...ee
OpenReview.ned 2022.
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What dis- ease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 1114)):42,, 2021.
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A dataset for biomedical research question answering. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- IJCNLP), pp. 2567-2577, Hong Kong, China, 2019. Association for Computational Linguistics. doi: 10.18653/v1/DD 12t-D19991199., URL https: http:::://aaaaathooooooooo
wwwp/ogggg...... 1259
Fangyu Lei, Qian Liu, Yiming Huang, Shizhu He. Jun Zhao, and Kang Liu. S3eval: A synthetic, scalable, systematic evaluation suite for large language models. ArXiv preprint, abs/2310.151477, 2023. URL.httpa.
/tatps:///rxiuue http://://aawww..o////////// abaaaaaaa11110 /231 13117.111111177
11

---

Published as a conference paper at ICLR 2025
Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos Jr, Caiming Xiong, Zachary Z Sun, Richard Socher, et al. Large language models generate functional protein sequences across diverse families. Nature Biotechnology. pp- 1-8, 2023.
Zaiqiao Meng, Fangyu Liu, Ehsan Shareghi, Yixuan Su, Charlotte Collins, and Nigel Collier. newire-then-prokee
contrastive recipe for probing biomedical knowledge of pre-trained lan- guage models. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I; Long Papers), pp. 4798-4810, Dublin, Ireland, 2022. Association for Computational Linguis- tics. doi: 10. 10.18853/1/2022.aac-22 12tp:22.caalloog..22222 URL https: http://///alaaatoooooooooooo sol-long.o. 329
Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carignan, Richard Edgar, Nicolo Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu, et Can generalist foundation models outcompete special-purpose tuning? case study in medicine. ArXiv preprint, abs/2311.164551 2023. URL
https://arxiv org/ wtt.ggg//////goao/a 112.2.6..222
OpenAl. Gpt-4 technical report, 2023.
OpenAl. Gpt-4o mini: advancing cost-efficient intelligence, 2024. URL https:///pennt
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models follow instructions with human feed- back. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Informa- tion Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022. URL http: http:::/ppppeeeeeee..pppppp http://appprreee ni http:///pppprroonppoccpaaeeeeeee ff wttpeerr_fllleeepppeeeeeee 2002:/22222ahaaa /hash/ ht.l
2lefde53be:3 64a 35t.:///2....4444444400000003330
Ankit Pal and Malaikannan Sankarasubbu. Gemini goes to Med school: Exploring the capabil- ities of multimodal large language models on medical challenge problems & hallucinations. In Tristan Naumann, Asma Ben Abacha, Steven Bethard, Kirk Roberts, and Danielle Bitter- man (eds.), Proceedings of the 6th Clinical Natural Language Processing Workshop, pp. 21- 46, Mexico City, Mexico, 2024. Association for Computational Linguistics. URL https: //aclanthologe org/2024 cwwiinclnnpppp111
Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Gerardo Flores, George H Chen, Tom Pollard, Joyce C Ho, and Tristan Naumann (eds.). Proceedings of the Con- ference on Health, Inference, and Learning, volume 174 of Proceedings of Machine Learning Research, pp. 248-260. PMLR, 2022. URL pa 22122a.12 html
ngss mlr. press/w1141
Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, pp. 1-9, 2023.
Mujeen Sung, Jinhyuk Lee, Sean Yi, Minji Jeon, Sungdong Kim, and Jaewoo Kang. Can language models be biomedical knowledge bases? In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. pp. 4723 4434, Online and Punta Cana. Dominican Republic, 2021. Association for Computational Linguistics. doi: 10. 18653vv122221.cooo wwwp:211.mmmmpmmm.....mm URI. /tttps://sclaatto// www.anhhwh.o..ooooooooooooooo..e
ogy. org/ 2021 e 388
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, al. Gemma: Open models based on gemini research and technology. ArXiv preprint, abs/2403.08295, 2024. URL
https://arre http//awww..rraaoa/aoo///aaa 2403 08295
12

---

Published as a conference paper at ICLR 2025
Augustin Toma, Patrick R Lawler, Jimmy Ba, Rahul G Krishnan, Barry B Rubin, and Bo Wang. Clinical camel: An open-source expert-level medical language model with dialogue-basee knowl- edge encoding. ArXiv preprint, fbs/2305.12031,, 2023. URL ptttps://arxin org/ /bs/
2305. 12031
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Llama Open foun- dation and fine-tuned chat models. ArXiv preprint, abs/2307.09288, 2023. URL https:
Mlariiv.orge abs/ http://bb/222000939 222770020777288
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang. Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural In- formation Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 wttp://siiii..c...ppppeeeees cc hippppr_fllesppppeeeep
16, 2023, 2023. URL http::/papers..nnnt nips 2est:22ffaa223333333666622222223 Absstttatttt-tttcattttctacacaate
Abstract Datasets_aaaana
Benchmarks. html
Yuxuan Zhou, Xien Liu, Chen Ning, and Ji Wu. Multifaceteval: Multifaceted evaluation to probe Ilms in mastering medical knowledge. In Kate Larson (ed.), Proceedings of the Thirty-Third International Joint Conference Artificial Intelligence, 1JCAI-24, pp. 6669-6677. International Joint Conferences on Artificial Intelligence Organization, 2024. doi: 10.24963/iccui22224473..
URL https://aoi.org: wttp:////.....////.......o. /tt./////aaa222/7733333 Main Track.
Kaijie Zhu, Jiaao Chen, Jindong Wang, Neil Zhenqiang Gong, Diyi Yang, and Xing Xie. Dyval: Dy- namic evaluation of large language models for reasoning tasks. In The Twelfth International Con- ference on Learning Representations, 2024a. URL
.ee.t/ewweeeeeeeeeeeee.eee.eeeee forum?
Kaijie Zhu, Jindong Wang, Qinlin Zhao, Ruochen Xu, and Xing Xie. Dyval 2: Dynamic evaluation of large language models by meta probing agents, ArXiv preprint, abs/2402. 14865, 2024b. URL
https: wrwia.ivv....... www.abs.ss...........ee.......aa httpaa///2222222 2402 14865
Wenhong Zhu, Hongkun Hao, Zhiwei He, Yun-Ze Song. Jiao Yueyang, Yumeng Zhang, Hanxu Hu, Yiran Wei, Rui Wang, and Hongyuan Lu. CLEAN-EVAL: Clean evaluation on contaminated large language models. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings the Association for Computational Linguisties: NAACL 2024, pp. 835 847, Mexico City, Mexico, 2024c. Association for Computational Linguistics. URL /ttps://alaatthooooyy.ooo 2024. 4.
baacl.5 53
13

---

Published as a conference paper at ICLR 2025
A DETAILS OF DATASETS
We validate the proposed framework on two datasets: a biomedical evaluation benchmark, Med- LAMA, and disease-centric clinical knowledge base, DiseK. Given the large scale of these datasets, we sample a subset of knowledge points from each by selecting a single tail entity for each 1-to-N relation. Additionally, we sample negative triplets increase the evaluation difficulty. Table 3 and
5 list the relation types involved in the sampled datasets. The sampled MedLAMA dataset includes 1,000 positive triplets and 1,000 negative triplets for each relation, while the detailed statisties for
DiseK are presented in Table 6
Relation Type
Description
particular morphology (structural feature or form) is associ- ated with another concept, often disease. A disease is characterized by the presence of abnormal cells.
associated morphology of disease has abnormal cell disease has associated anatomic site
A disease occurs has an impact at anatomic site.
disease has normal cell ori- gin
A disease originates from a type of normal cell.
disease has normal tissue
A disease originates from type of normal tissue.
origin
disease mapped to gene disease may have associated
gene associated with a specific disease.
A disease may be associated with another disease.
disease
disease may have finding disease may have molecular abnormality
A possible clinical finding symptom is observed in disease.
potential molecular abnormalities may be present disease.
A particular gene encodes a specific gene product, such as pro- tein.
gene encodes gene product
gene product has associated anatomy
gene product is associated to an anatomical structure.
gene product has biochemi- cal function
gene product is associated to a biochemical function.
gene product plays role in biological process has physiologic effeet
gene product plays role in a biological process.
A substance process has physiological effect on the body.
may prevent
substance may prevent a disease.
may treat
A substance may treat disease.
occurs after
event or condition occurs after another.
Table 4: Relation types in the MedLAMA dataset that involve in our study.
Relation Type
Description
Symptoms Affected sites
Physical or mental feature that indicates the presence of the disease. Specific parts of the body that are impacted or harmed by the disease.
Pharmaceutical substances preseribed to manage, alleviate, or cure the symptoms and effects of the disease.
Therapeutic Drugs
Medical procedures that treat the disease, involving the cutting, repair-
Surgical Procedures
ing, removal of body parts.
Table 5: Relation types involved in the DiseK dataset.
B EFFECT OF REPHRASING LLM SELECTION
Brrtlluu----iinsttt 4k-instruct (Abdin et al. 2024) as the rephrasing model in our prototype-base sample generation module to study the effect of rephrasing LLM selection on the evaluation results. The evaluation results in Table 8 show that L.LMs generally achieve similar
We have also leveraged
14

---

Published as a conference paper at ICLR 2025
Relation Type
Positive
# Negative
#Symptoms #Affected Sites
987
987
745
745
#Therapeutic Drugs
836
836
#Surgical Procedures
599
599
Table Statistics the sampled DiseK dataset. # Positive: the number of positive triplets extracted from DiseK. # Negative: the number of negative triplets sampled from DiseK.
Dataset
MedLAMA Biomedical
DiseK Clinical
Type
# Rel Types
17
# Triplets
34,000
6,348
Table 7: Statistics of the sampled datasets.
performance on datasets generated based on different rephrasing LLMs, indicating that the effect of rephrasing LLM selection is minimal the final evaluation results.
MedLAMA
DiseK
Model
Llama-3 Reph
Phi-3 Reph
Llama-3 Reph
Phi-3 Reph
Llama2-
+3.1
+2.9
+2.8
+2.3
Vicuna-7B
+7.5
+6.3
+3.9
+3.5
Vicuna- 13B
+10.7
49.8
+5.7
+5.3
Gemma-7B
+9.4
+9.1
+5.0
+5.3
Llama3-8B
+16.6
+16.4
+9.3
49.3
Llama2-70B
+13.8
+13.3
+9.0
+7.0
Clinicalcamel-70B
+21.9
+22.4
+16.1
+14.6
Meditron-70B
+14.7
+15.8
+10.2
+8.2
Med42-70B
+20.0
+20.4
+14.8
+13.9
Llama3-70B
+26.9
+27.4
+20.9
+19.3
GPT-3.5-turbo
+16.2
+17.9
+10.3
+8.2
GPT-4
+31.7
+32.3
+26.7
+26.1
Table 8: Performance of LLMs datasets generated by PretexEval using different rephrasing LLMs.
C DETAILS OF METHOD SETTING
Details of Predicate Equivalent Transformation An example of the predicate equivalent trans- formation procedure applied in this study is illustrated in Figure 9 First, the inversion operation is applied to the original expression to create new expression. Subsequently, these two expressions are instantiated into two additional expressions. Finally, double negation is used to generate four more expressions.
Details of Prototypes-based Generation As introduced before, we designed a prototype-based sample generation strategy to ensure the reliability of the generated samples and crafted proto- type for each combination of relation type and predicate transformation type by discussing with clinicians. We list all the crafted prototypes in Table 9 10 and [ for reproducing our experiments.
prompt the Llama3-70B-- Ins-trrtuctt model with the following instruction: "Please paraphrase the following statement to present the same concept a different way. DO NOT change the basic sentence structure. Directly output the paraphrased statement without other text. Statement: [prototype]'. our experiments, we found that statements rephrased using this method effectively preserve the original meaning of the prototypes.
For LLM rephrasing,
15

---

Published as a conference paper at ICLR 2025
(B,A)
D(xB) MC(A.))
MT (A, B)
C(A,8) MD B)
Original Expression
{ MT -B. (B.A))
(-(D(x B) MC(A.x)))
(-MT (A,
(-(C(())) MD(x
1
Inversion
Instantiation
3 Double Negation
Figure 9: An example of the predicate equivalent transformation procedure implemented in this study.
D DETAILS OF EVALUATION SETTING
In our implementation, for the statement verification task, we form test samples based on the follow- ing format: "[Statement], is the statement above true or false? Please answer True or False." For 4-option multiple-choice questions, we first mask the tail entity in statement to create a masked version and then generate questions in the following format, requiring the LLM to select the correct tail entity from four options to fill in the blank:" [Masked statement). Which of the following options most likely to fill in the blank above? Options: [Options]." Negative options domly sampling negative entities of the same type as the tail entity from the medical knowledge base. For double-negation-typ statements, modify the prompt to require the LLM to select the least likely option.
generated by ran-
Josamycin propionate has the potential to inhibit the development bacterial infections.. statement above false? Please true/false.
Statement:
Answer: True
"Ceftolozane sulfate has the potential to inhibit the development of bacterial infections.", is the statement above true false? Please answer true/false.
Statement:
Five Demonstrative Examples
Answer: False
Statement:
"Infection with hepatitis virus confer immunity against Hepalitis B.", the statement above true or false? Please answer true/false. Answer: True
Statement:
"Benzphetamine may be used manage weight loss in individuals Obesity." statement above false? Please trueffalse.
Test Question
Answer: True
Figure 10: An example of the five-shot in-context learning process applied in our evaluation.
Regarding the five-shot setting, we randomly select five question-anss answer pairs for each combina- tion of relation and predicate transformation type to create demonstrative examples, as depicted in Figure Complex prompting strategies such as -f-thuggtt are not applied in our study, as the evaluation statements are crafted to be straightforward and easily understandable, allowing for verification without the need for complex logical reasoning. In the inference process, we use greedy search for most of LLMs. However, commercial LLMs like GPT-3.5-turbe do not support greedy search, and we use their default generation setting to make a relative fair comparison across LLMs. We extract the prediction from models' response based on keywords since the words/phrases used to express True and False are limited. We listed all of the keywords applied to recognize answers in Table
16

---

Published as a conference paper at ICLR 2025
E DETAILS OF BASELINES
The "Direct" baseline directly generates a statement for a knowledge triplet using pre-constructed template. For example, for for the relation type "may-prevent", the "Direct" baseline first manually creates a template based on this relation type:
may-prevent + [head] may be able to prevent [tail].
Next, for specific knowledge triplet (Oprelvekin, may-prevent, Thrombocyoopenia), "Direct" base- line replaces the head and tail entities in the template with those from the triplet to generate the textual expression of the knowledge point:
[head] may be able prevent [tail] Oprelvekin may be able prevent Thrombocytopenia.
This template-based paraphrasing approach ensures the reliability of the generated knowledge rep- resentations. However, since each generation uses a fixed template, the diversity of the gener- ated knowledge expressions is limited, making it difficult to achieve comprehensive evaluation of L.LMs' medical knowledge mastery.
For LLMEval baseline, implement this method by directly generating diverse statements using Llama3-70B: -0--tttuttt Specifically, we prompt the LLM with the following instruction: "Based on the given knowledge triplet, generate 8 statement to express the underlying knowledge different ways. Output one statement per line. Directly oulput the statements without other text. Knowledge the greedy search for the can follow the instruction, generating samples
triplet: [Iriplet]." To ensure the quality of generated samples, decoding process. We find that Llama3-70B-Instre in separated lines. Compared the "Direct" baseline, LLMEval generates test samples with much higher lexical diversity, while it may potentially introduce factual errors into the generated samples.
F PERFORMANCE OF LLMS ACROSS KNOWLEDGE TYPES
We also analysis the fine- ene-grained performance of LLMs across different types of clinical knowledge stored in DiseK. The analysis results are presented Table [3] Based on this analysis, we can draw the following conclusions: (I) GPT-4o outperforms the rest of LLMs on 3 out of types of clinical knowledge, exhibiting more comprehensive mastery of medical knowledge than other LLMs; (2) With the same model parameter scale, Llama3-70B achieved the best performance across all four relation types, possibly due to its significantly large training data volume (7.5 times that of Llama2-70B); (3) The three medical-specifie 70B models (ClinicalCamel. Meditron, Med42) are all developed based on Ilamm2-700 through finetuning on medical corpora, and they show a notable improvement in medical knowledge mastery compared to Llama2-70B. In particular, ClinicalCamel 70B enhanced accuracy on "Affected Sites" from 72.2 with Llama2-700 AOB to 84.1, while Med42-70B improved performance on "Surgical Procedures," " raising it from 62.6 with Llama2-70B to 71.7.
G FULL EXPERIMENTAL RESULTS
G.1 JOINT ACCURACY
We illustrate the joint accuracy of all LLMs evaluated by PretexEval and LLMEval in Figure 11 and respectively. The experimental results support our conclusions: the evaluated LLMs generally perform worse on datasets generated by PretexEval. Moreover, LLMs' performance decline faster when evaluated by PretexEval compared with evaluated by L.MEval, indicating that current LLMs lack consistency in understanding medical knowledge presented in various structures.
12
G.2 ABLATION STUDY
We present the ablation results of all evaluated LLMs regarding key components and predicate trans- formation types Table [nd [5 respectively. We also conducted an ablation on DiseK regarding different transformation types in Table Toble[o]and found that current LLMs generally struggle to handle the negated expressions. These results are consistent with our findings in the paper, demonstrating the effectiveness of our framework.
17

---

Published as a conference paper at ICLR 2025
MedLAMA
DiseK
0.9
0.9
0.8
0.8
Joint Accuracy
0.7
Joint Accuracy
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.2
0.2
0.1
0.1
0.0
0.0
1
2
3
8
1
2
3
4
5
8
Number of Questions
Number of Questions
Vicuna-138
ClinicalCamel-70s
Vicune-70 7B
Gemma-70
Liama2-708
Meditron-708
GPT-3.5-turbo
GPT-4o
Figure 11: Performance (joint accuracy) of all LLMs evaluated by the proposed PretexEval frame- work.
MedLAMA
DiseK
0.9
0.9
0.8
0.8
0.7
0.7
Joint Accuracy
loint Accuracy
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0.0
0.0
1
2
7
8
1
2
3
4
5
8
Number of Questions
Number of Questions
Llema2-78
Vicune-13B
Llema3-00
clinicalCemel-700 Meditron-70B
Meda2-708
Liama3-708
Gemma-78
Figure 12: Performance (joint accuracy) of all LLMs evaluated by the LLMEval method.
18

---

Published as a conference paper at ICLR 2025
Predicate Transtormation Type
Relation Type
Nonc Inv
Ins
Inv+Ins
[X] is the
If patient exhibits mor-
If patient suffers from
associated
associated morphology
[Y] often accom-
phological change of [X],
[У], then he/she is exhibit-
morphology of
panied by the mor- phology of [X].
then he/she may suffer from [Y].
ing a morphological change
of [Y]..
of [X].
[X] has the
The abnormal cell
a patient suffers from [X],
a patient has the ahnormal
disease has
abnormal cell
type [Y] is detected within [X].
then he/she has the abnormal
cell [Y], then he/she may
abnormal cell
[Y]
cell [Y].
suffer from [X].
The disease
[X] stem
Anatomical site [Y] associated with the development of dis- ease [X].
disease has
If a patient suffers from [X], then he/she has lesions in [Y).
If a patient hus lesions in [Y], then hefshe may suffer from [X].
from the
associated
associated
anatomic site
anatomic site
[Y]-
The discase
Normal cell [Y] is associaated with the development of dis- case [X].
disease
has
[X] stems
If a patient suffers from [X], then he/she has lesions in [Y].
If a patient has lesions in [Y]. then he/she may suffer from [X].
normal cell
from the
origin
normal cell
[Y]-
The disease
Normal tissue [Y] is associated with the development dis-
disease has
[X] stems
If patient suffers from [X], then he/she has lesions in [Y].
If patient has lesions in
normal tissuc
from the
[Y]. then he/she may suffer from [X].
origin
normal tissue
case [X].
[Y]
disease
The discase
Gene [Y] is associ-
If a patient suffers from [X]. then he/she has lesions in
If a patient has lesions in [Y], then hefshe may suffer from [X].
mapped
to
[X] is mapped
ated with the disease [X].
gene
to gene (Y].
[Y].
The disease
disease may
[X] might
The disease [Y]
If a patient suffers from [X], then the likelihood of he/she
If a patient suffers from [Y], then the likelihood of he/she suffering from [X] is higher.
have associ- ated discase
have the asso-
might have the asso- ciated discase [X].
ciated discase
suffering from [Y] is higher.
[Y]
disease may
X may have
[Y] may be associate with [X]
If patient suffers from [X], then he/she has [Y].
If a patient has [Y]. then he/she may suffer from [X].
have finding
YY]
The discase
disease
Molecular abnormal- ity [Y] may be asso- ciuted with the dis- ease [X].
[X] may have
If a patient suffers from [X]. then he/she may has molec- ular abnormality [Y].
If a patient has molecular abnormality [Y]. then he/she may suffer from [X].
may have
molecular
molecular
abnormality [Y].
abnormality
If the expression level of [X] decreases, it may lead to reduction in the production or activity of [Y].
If the production activ- ity of [Y] decreases, it may caused by the reduction in the expression level of (X].
The gene [X]
The gene product [Y] is encoded by the gene [X].
gene encodes
encodes gene product [Y).
gene product
The
gene
gene product
product
The unatomy [Y] is associated with the gene product [X].
The gene product [X] plays role in anatomy [Y].
Anatomy [Y] where [X] functions.
has associated
[X] has the
anatomy
associated
anatomy [Y].
If the functionality IY] decreases, may caused by the reduction in the produc- tion of [X].
gene product has biochemi-
[X] has bio-
If the production of [X] de- creases, the functionality of [V] may decrease.
[Y] is biochemical function of [X].
chemical
cal function
function [Y].
The gene
gene product plays role in biological
Biological process
product [X]
If the production of [X] de- creases. the process of [Y] may be influenced.
If [Y] is affected, it may caused by the reduction in the production of [X].
[Y] associated
plays a role
in biological with the gene prod-
process
process [Y].
uet [X]
[X] has phys-
If a patient takes [X]. he/she may have physiologic effect of [Y].
If patient has physiologic effect of [Y], he/she may have taken [X].
has physio-
iologic effect
[Y] can be caused by [X].
logic effect
of [Y].
[X] may be
[Y] may be prevented
If a patient takes [X]. he/she can prevent [Y].
If a patient wishes to prevent [Y], he/she should take [X].
may prevent
able to pre-
by [X]
vent [Y].
[X]
might
[Y] may be treated by [X]
If a patient takes [X], he/she can treat [Y].
If a patient suffers from [Y], he/she should take [X].
may treat
treat [Y].
If a patient occurs [X], he/she may occur [Y] be- fore.
If patient occurs [Y].
occurs af- ter [Y].
[Y] may occur before
occurs after
he/she may occur [X] after- wards.
[X].
Table 9: Prototypes crafted for the MedLAMA dataset (1/2). Inv: inversion; Ins: instantiation.
19

---

Published as a conference paper at ICLR 2025
Predicate Transformation Type
Relation Type
DN
Inv+DN
Ins+DN
Inv-Ins+DN
[X] is not the
[Y] is accompunied
A patient that exhibits
A patient that suffers from [Y]
associated mor-
ussociated mor-
by the morphology of
phologicul change of [X] does
does not exhibit a morphologi-
phology of
phology of [Y].
[X].
not sulfer from [Y].
change [X].
[X] does not
The abnormal cell type
A patient that suffers from [X]
A patient that has the abnormal
discase has ab-
has the abnor-
[Y] is not detected
does not have the abnormal cell [Y].
cell [Y] does not suffer from [X].
normal cell
[Y].
within [X].
The disease
Anatomical site [Y] is not associated with the development of disease [X].
discase
has
[X] is not stem
A patient that suffers from [X] does not have lesions in [Y).
A patient that has lesions in [Y]
associated
from the asso-
does not suffer from [X].
anatomic site
ciated anatomic
site [Y].
The discase [X]
Normal cell [Y]
discase has nor- mal cell origin
does not stem
associaated with the de- velopment disease [X].
A patient that suffers from [X]
A patient that has lesions in [Y]
from the nor-
does not have lesions in [Y].
does not suffer from [X].
mal cell [Y].
The disease
Normal tissue [Y] is
disease has mal tissuc ori-
[X] not stem
not associated with
A patient suffers from [X]
A patient that has lesions in [Y]
from the nor-
development of disease
does not have lesions (Y).
does not suffer from [X].
gin
mal tissue [Y].
[X].
disease [X]
Gene [Y] is asso-
discase mapped gene
A patient that suffers from [X]
A patient that has lesions in [Y]
is not mapped
ciuted with the discase
does have lesions (Y). does not suffer from [X].
to the gene [Y).
[X].
The discase [X]
have associated discase
discase
may
is not associ-
The disease [Y] is not associated with discase
then the likelihood of bolshe suffering from (Y) not higher.
If patient suffers from [X],
If patient suffers from (Y),
then the likelihood helshe
ated with dis-
[X]
suffering from [X] is not higher.
[Y].
discase
[X] does not
[Y] is not associated
A patient that suffers from [X] does not have [Y).
A patient that has [Y] does not suffer from [X].
have finding
have [Y].
with [X]
The discase
discase may
[X] does not
Molecular abuormality [Y] not associated with the discase (X).
A patient that suffers from [X]
A patient that has molecular ab- normality [Y] does not suffer from [X].
have molecular
have molecular
does not have molecular abnor- mality [Y].
abnormality
abnormality
[Y].
gene [X] does not
The gene product [Y] is not encoded by the gene [X]
A decrease in the expression
decrease in the production or activity of [Y) is not caused by the reduction in the expression level of (X].
gene encodes
level [X] does not affect production and activity of [Y].
gene product
encode
gene
product [Y].
The gene prod-
has associated
gene product
uct [X] does not
have the asso-
not associated with the gene product [X].
The anatomy [Y] is
The gene product [X] does not play a role in anatomy [Y].
Anatomy [Y] is not where [X] functions.
anatomy
ciated anatomy
[Y].
[X] does not
A decrease in the functional-
gene product
have biochem-
[Y] is not biochemical function of [X].
A decrease in the production of [X] does not affect the function- ality of [Y].
ity of [Y] is not caused by the
has biochemi-
ical
function
reduction in the production of [X].
cal function
[Y].
The gene prod-
gene product
uct [X] does not
Biological process [Y]
A decrease in the production of [X] does not affect the process of [Y].
A change of [Y] is not caused by the reduction in the produc- tion of [X].
plays role
play mole in
is not associated with the gene product [X]
biological pro-
biological pro-
cess
cess [Y].
[X] does not
has physiologic
have physio-
[Y] cannot be caused by
A patient that takes [X] does not have physiologic effect of [Y]
A patient that has physiologic
effect
logic effect of [X]. [Y]
effect of [Y] has taken [X].
[X] is not able
[Y] cannot be prevented
Taking [X] have no effect on preventing [Y].
A patient wishes to prevent [Y]
may prevent
prevent [Y]
by [X]
has need take [X].
[X] is not able
[Y] cannot be treated by
Taking [X] have no effect on
A patient that sufters from [Y]
may treat
treat [Y],
[X]
treating [Y].
no need to take [X].
[X] does not
[Y] cannot occur before
A patient occurs [X] will not oceur [Y] before.
A paticat occurs [Y] will not occur [X] afterwards.
occurs after
occur after [X]
[X].
Table 10: Prototypes crafted for the MedLAMA dataset (2/2). Inv: inversion; Ins: instantiation; DN: double negation.
20

---

Published as a conference paper at ICLR 2025
Relation Type
Implication Type
Symptoms
Affected Sites
Therapeutic Drugs
Surgical Procedures
[Y] is common
[Y] is the affected
[Y] is a common
iY] a common
None
symptom of [X].
site for [X].
medication for [X].
cedure for [X].
Common medica-
Common procedures
Common symptoms
Affected sites for [X]
Inv
tions for treating [X]
for treating [X] in-
[X] include (Y].
include (Y).
include [Y].
clude [Y].
If patient has [X], they are very likely to have symptoms of [Y].
If patient has [X],
a patient has [X], [Y] can be used to treat their condition.
a patient has [X],
their [Y] site is very likely to show lc- sions.
[Y] can be used treat their condition.
If patient has symp- toms of [Y], they are very likely to have [X].
If patient shows le- sions in their [Y] site, they ure very likely to have [X].
If [Y] can be used to treat patient's con-
If [Y] can be used to treat patient's con- dition, they may have
Inv+Ins
dition, they may have [X].
[X].
is] common symptom of [X].
is] not the affected
[Y] is not common
[Y] is common procedure for [X].
DN
site for [X].
medication for [X].
Common symptoms of [X] do not include
Common medica-
Common procedures for treating [X] do not include [Y].
Affected sites for [X] do not include [Y].
Inv+DN
tions for treating [X]
[Y].
do not include [Y].
Patients with [X]
Patients with [X] unlikely to show le- sions in their [Y] site.
Patients with [X] do not commonly
Patients with [X] do
Ins+DN
unlikely have symptoms [Y].
commonly use for treatment.
[Y] for treatment.
Patients showing le-
Patients with symp- toms of [Y] are un-
Patients who can be treated with [Y] ure unlikely to have [X].
Patients who can be treated with [Y] are unlikely to have [X].
sions in their [Y] site
IN
unlikely have
likely to have [X].
[X].
Table 11: Prototypes crafted for the DiseK dataset. Inv: inversion; Ins: instantiation; DN: double negation.
Categories
Keywords
True
True, Entailed, Correct, Yes
False
False, Contradicted, Wrong, No
Table 12: The keywords utilize to extract answers from LLMs' responses.
Model
Symptoms
Affected Sites
Therapeutic Drugs
Surgical Procedures
Llama2-7B
+1.9
+10.7
+-0.4
+-1.1
Vicuna-7B
+0.2
+8.1
+1.3
+8.6
Vieuna- 13B
+3.3
+14.2
+1.1
+5.9
Gemma-78 7B
+3.2
+11.2
+0.7
+6.1
Llama3-8B
+6.2
+17.4
+3.8
+11.8
Llama2-70B
+4.6
+22.2
+0.1
+12.6
ClinicalCamel.
+9.5
+34.1
+5.1
+19.6
Meditron-70B
+6.9
+18.3
+3.0
+15.9
Med42-70B
+7.2
+29.9
+5.4
+21.7
Llama3-70B
+15.1
+37.7
+11.6
+22.4
GPT-3.5-turbo
+5.5
+19.4
+5.6
+13.1
GPT-4o
+23.4
+41.5
+20.9
+19.0
Average
+7.1
+21.2
+4.8
+12.4
Table 13: Performance of LLMs on the four types of discase-related knowledge contained in the DiseK knowledge base.
21

---

Published as a conference paper at ICLR 2025
Med
DiseK
Model
PretexEval
w/o PreEq Trans
wio
LLM Rephrasing
PretexEval
wo TranqTrans
w/o LLM Rephrasing
Llama2-7B
+3.1
+7. 47.4959
+1. +199ssss
+2.8
+7.51189.5%
+2.6p
Vicuna-7B
+22. 419510
43.713438
+9.5
+2.5g58
Vicuna 13B
+10.7
+20.3 3-198
+1 +11.04336
+5.7
+9.2
+5.9
Gemma-7B
-16.2-72.%
+12.8-35u
+7.2744ir
+6.9r30.99
Llama3-8B
+16.6
+24.1-455%
+18.5-11.9
+9.3
+18. 100.59
+10.2
Llama2-700
+13.8
+18.27104.74
+14.6153s
+18.4 100.5%
+7.811418
ClinicalCamel-088
+21.9
+30.6-a0.2
+22.8t4.55.
+16.1
+23.1-4400
+18.0-120%
Meditrom-70B
+14.7
+25.7-74m
+15.8pp.rx
+10.2
+18.1,17.5
+11.5-12775
Med42-708 70B
+20.0
+25.2-u178
+20.4
+14.8
+20.4-31.3%
+17.9-21.88
GPT:3.5-turbo
+16.7
+9. 29.4-70u8
+17.94p3s
+10.3
+17. +17.1 1665%
+11.8-15.4
Llama3-708
+26.9
+33.0 0-228
+30.4-1334
+20.9
+27.8:33.6%
+21.0
GPT-4o
+31.7
+36.0
+33.8
+26.7
+29.2r9.9%
+30.4-1189
Table 14: Ablation results of all evaluated LLMs for key components of the proposed PretexEval framework. Pranq: Predicate Equivalence Transformation; LLM Rephrasing: Prototype-base Sample Generation.
MedLAMA
Model
None
+lnv
+DN+Inv
+AIl
Origin
+Inv
+DN+Inv
+All
Llama2-7B
47.4
+7.1
+3.615188
+3.1,5008
+7.5
+7. +7.9748
+3.9
+2. 12.8162.08
Vicuna-7B
+22.1
+21.8
+8.2 +8.2
+7.5 +7.5
+9.5
+11. 111.3
+ +4.7 150
+3. 93.9 198.89
Vicuna-
+20.3
119.911.9%
+11.6
+10.774735
+9.2
+10.0y90rs
+5.8
+5.7 157.2%
Gemma-7B
+16.2
+15.9
+10.813355%
+9.4141.9
+7.2
+10. 4141o
+5. 45.2g27005
+5.0 130.9%
Llama3-8B
+24.1
+23.3
+18.5
+16.6gm1.9
+18.9
+18.61188
+10.1 140.1166....
-9.3151.14
Llama2-70B
+28.2
+27 4...%
+15.8
+13.819122%
+18.4
+18.8p2.9
+9.7 +9.7
+9.0
ClinicalCamel-700
+30.6
+30.311118
+23.2
+21.912K77
+23.1
+22.413.9
+17.9121.5%
+16.1 130.9
Mcditron-70B
+25.7
+25.4
+15.8138.68
+14.Tg4276
+18.1
+19.5y1.58
+11.1138.99
+10.2,44.5%
Mcd42-70B
+28.2
+27.9
+21.9172.54
+20.011898
+20.4
+20.211.18
+15.7123.19
+14.812r.7%
GPT--3.-tturoo
+29.4
+27.6
+18.2
+16.714322
+17.1
+18.1
+9.6
+10.3
Llama3-70B
+33.0
+31.811.8
+28.6,132%
+26.9Q1R6R
+27.8
+27. 51144
+22.3
+20.9)2st
GPT-4o
+36.0
+34.2
+33.61699
+31.711218
+29.2
+29.8
+26.81833%
+26.7 18.7%
Table 15: Ablation results of all evaluated LLMs for types of predicate transformation in the pro- posed framework.
Model
Direct
+Double Negation
+Inversion
+Instantiation
Llama2-7B
+7.5
+4.3
+7.9
+6.2
Vicuna-7B
+9.5
+4.0
+11.4
+10.3
Vicuna- 13B
+9.2
+3.8
+10.0
+10.2
Gemma-7B
+7.2
+3.6
+10.4
+7.8
Llama3-8B
+18.9
+10.9
+18.6
+18.8
Llama2-70B
+18.4
+11.0
+18.8
+19.6
ClinicalCamel-7000
+23.1
+18.9
+22.4
+23.8
Meditron-70B
+18.1
+10.2
+19.5
+17.9
Med42-70B
+20.4
+14.1
+20.2
+22.0
Llama3-70B
+27.8
+22.3
+27.5
+29.4
GPT--35-tturb
+17.1
+9.0
+18.1
+19.6
GPT-4o
+29.3
+25.8
+29.8
+30.8
Table 16: Ablation results of different types of transformation on the evaluated L.MMs.
22

---

Published as a conference paper at ICLR 2025
H DETAILS OF HUMAN ANALYSIS ON GENERATED SAMPLE QUALITY
We conduct human analysis on the quality of generated samples, regarding the reliability, lexical diversity, and structural diversity. As mentioned above, randomly sampled 50 knowledge triplets from MedLAMA and collect the corresponding test samples generated by LLMEval and PretexEval before/after rephrasing. We then engaged three experienced doctors (all holding medical licenses, with two having 3-6 years of experience and one senior doctor with years of experience) to grade the test samples based the following criteria:
Reliability:
0 (Poor): Many inaccuracies present, leading to significant misunderstandings misinterpretations of the knowledge presented.
(Good): Most information is correct, but minor inaccuracies or ambiguities are present that do not affect the overall meaning.
5 (Excellent); All information presented is factually correct, with clear and precise explanations. No errors or ambiguities are detected.
2. Lexical Diversity:
0 (Poor):
The text shows very limited vocabulary diversity compared to the original knowledge triplet.
(Good): There is moderate variety of vocabulary compared to the original in the non-medical lexicon, while the medical terms remaining unchanged.
5 (Excellent): The text uses diverse vocabulary compared to the original knowledge triplet, including both medical terms and non-medical lexicon.
3. Structural Diversity:
(Poor): The sentence structure remains unchanged, fully replicating the original
order of the knowledge triplet.
3 (Good): The sentence structure has been slightly adjusted, such by changing word order or modifying certain phrase combinations, while the main grammatical structure remains unchanged.
5 (Excellent): The sentence structure has been thoroughly reconstructed, significantly altering the way information is presented, while conveying the same content with a completely new syntax and grammatical logic.
For each test sample, the doctors are presented with the original knowledge triplet for reference (see Figure [3). We hide the source of each text sample to ensure the fairesss of the evaluation. Finally, we average the scores of samples generated by the same method to derive the final scores. We also measured the inter-annotator agreement coefficient across the three doctors the three
Knowledge: (nusinersen, may treat, Muscular Atrophy, Spinal)
Text Sample:
Treatment with nusinersen enables patients to manage Spinal Muscular Atrophy.
Lexical Diversity:
(Poor) 3 (Good)
5 (Excellent)
Structural Diversity: (Poor) 3 (Good)
5 (Excellent)
Reliability:
(Poor) (Good) 5 (Excellent)
Figure 13: A grading sample presented to human doctors.
evaluation dimensions. Specifically, we leverage the Intraclass Correlation Coefficient (ICC) as the consistency metric, where ICC 0.9 indicates excellent agreement). The measurement results in Table [I7 show that the three doctors achieved excellent consistency scores across all evaluation dimensions, indicating that manual validation process highly robust.
23

---

Published as a conference paper at ICLR 2025
Dimensions Reliability
Inter-annotator Agreement Coefficient (ICC2k)
Confidence Interval (95%)
0.912
[0.88. 0.93]
Lexical Diversity
0.938
[0.92,0.95]
Structural Diversity
0.956
[0.94.0.97]
Table 17: Inter-annotator agreement metrics (ICC2k) of the human validation conducted in this study.
I APPLICABILITY OF PRETEXÉVAL FOR MODEL TRAINING
In out study, we have conducted a preliminary study to explore the potential of training with PretexEval-2e -eeerratd samples. For this study, we selected LLaMA3-8B as the backbone model, applying LoRA finetuning (Hu et al. (2022) the training method. We apply grid search on the learning rate (1e-4, 5e-5, 2e-5} and batch size {4, 8, 16) to find the best hyperparameters. We train each model for 10 epochs. This study aims to investigate two research questions:
RQ1: Can training on PretexEval -eeeevaaee samples improve LLMs' understanding of knowledge outside the training set?
RQ2: Can training few types PretexEval-genenan samples improve LLMs' under- standing of other unseen expressions?
For RQ1, we selected 200 knowledge triples as the training set and an additional 200 triples as the training set. We use all the types of expressions generated by PretexEval for training. We apply PretexEval on the test set for evaluation. For RQ2, we select four types (Direct, Double Negation (DN), Inversion (Inv), Instantiation (Inst)) for training and utilize the remaining four types of pressions for evaluation. We use triplets from the test set of RQ1 for this study. Our experiments demonstrate that training on BettexEval-generatee derstanding of knowledge outside the training set and their understanding of unseen expressions as well. This suggests the potential of leveraging TretexEval-generate samples as effective resources for training.
samples could potentially improve LLMs' un-
While the results of this preliminary study show promise in enhancing LLMs' medical knowledge consistency, future work is needed to make this approach practical. We leave this for future work.
24


Reviews:
[{"review_id": "AjVijah4oA", "reviewer": "Reviewer_fVf4", "summary": "The paper presents PretexEval, an innovative evaluation framework designed to assess medical LLMs' knowledge mastery with a focus on generating reliable, diverse test samples. By transforming medical knowledge points through predicate equivalence transformations, PretexEval produces varied yet accurate textual representations, allowing for a richer evaluation compared to traditional benchmarks like MedQA. The framework not only highlights areas where LLMs are deficient but also introduces joint accuracy as a critical metric to measure the consistency of LLMs’ responses—an important yet often overlooked requirement in healthcare settings.", "questions": "- How do you plan to extend PretexEval beyond true/false verification tasks? Adding context-sensitive or reasoning-based tasks could expand its applicability to more complex clinical scenarios.\n\n- Prototype Automation: Are there plans to automate prototype creation to reduce potential subjectivity and improve scalability? If not, what criteria ensure prototype reliability and consistency across varied medical contexts?\n\n- Reliability Validation: Given the small sample size and limited number of annotators for reliability measurement, how would expanding the annotation set or incorporating inter-annotator agreement metrics influence reliability scores? Would this improve the robustness of results?\n\n- Could PretexEval-generated samples serve a role in training, not just evaluation? If incorporated into training, would these samples likely boost LLMs’ consistency on diverse medical knowledge?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Novel Evaluation Method: PretexEval introduces a fresh approach by transforming knowledge points into diverse predicates, effectively improving both the reliability and comprehensiveness of LLM evaluation. This is particularly relevant in fields like healthcare, where factual consistency is crucial.\n- Focus on Consistency: The framework’s use of joint accuracy to measure consistency across expressions of the same knowledge point is a valuable contribution. In healthcare, where consistency is key, this metric allows for a deeper look at whether a model can reliably interpret nuanced variations of medical knowledge.\n- Broad Evaluation Scope: The study evaluates 12 leading LLMs across both general and medical-specific models, demonstrating PretexEval’s versatility and highlighting critical insights into current LLM limitations.", "weaknesses": "- Scope of Evaluation Tasks: The framework is limited to true/false verification tasks, which could constrain its applicability in complex medical scenarios where contextual understanding and reasoning are required.\n- Prototype Dependence: Manually crafted prototypes, while improving reliability, may limit scalability and introduce potential subjectivity. Refining this step or automating parts of it could enhance PretexEval’s flexibility. Reliability here is measured by two annotators on a 50-sample subset, introducing a skewed potential for bias. Expanding this evaluation to at least three annotators and including an inter-annotator agreement measure would add robustness.\n- Applicability for Model Training: While it’s a promising evaluation tool, exploring the potential of PretexEval-generated samples for training might add another layer of impact, possibly making it a resource for enhancing models’ medical knowledge consistency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7UWSZvj5sb", "reviewer": "Reviewer_jTNo", "summary": "PretexEval is a new way to assess how well large language models (LLMs) understand medical knowledge by creating dynamic test samples from medical databases. The framework uses simple logic transformations (like flipping statements, modifying them, and using double negatives) to produce a variety of reliable test questions, going beyond traditional fixed tests. The authors evaluate 12 well-known LLMs, including some designed specifically for medicine, using two databases focused on clinical diagnosis and treatment. They show that while these models perform well on existing medical benchmarks, they still struggle to consistently grasp medical knowledge when it’s presented in different ways. The main contribution is a flexible method for testing LLMs’ understanding of medical knowledge, backed by detailed results that reveal significant gaps between their performance on tests and their genuine understanding of medical concepts. This research offers further insights into methods for setting up a systematic way to scale the evaluation of LLM readiness for use in clinical settings.", "questions": "In the case study shown in Figure 6, the PretexEval method includes a rewording that shifts from \"may treat\" to \"is recommended.\" This raises concerns about whether this transformation validly rephrases the original medical relationship. In medical contexts, \"may treat\" and \"is recommended\" carry different implications:\n\"May treat\" suggests a possibility or potential for treatment, leaving room for other options or uncertainties.\n\"Is recommended\" implies a stronger clinical endorsement, suggesting that the treatment is advised or endorsed by clinical guidelines or experts.\nGiven this distinction, the shift from \"may treat\" to \"is recommended\" could be problematic if it alters the strength of the claim. This transformation might lead to incorrect evaluations, as LLMs could be unfairly penalized for failing to recognize a relationship that has been shifted beyond its original meaning.\nTherefore an important question arises:\n- Clarification on Predicate Transformation Boundaries: Could you clarify how you ensure that predicate transformations do not alter the underlying meaning of medical relationships? Specifically, how do you prevent shifts in meaning from \"may treat\" (a possibility) to \"is recommended\" (a stronger clinical assertion)? I understand some validation in the initial setting was completed, but this may be lost during rephrasing- so I just want to clarify what of the final questions were validated and by what grade physicians.\n- Handling Ambiguities in Medical Relationships: How does PretexEval handle nuanced medical relationships where different levels of certainty or recommendation are involved? For example, how do you ensure that transformations like \"may treat\" vs. \"is recommended\" retain their original intent without introducing stronger or weaker claims?\n- Validation of Rephrased Statements: Could you describe any validation steps taken by medical experts to confirm that reworded statements retain their original intent and do not introduce factual inaccuracies or misleading implications? \n- Addressing these concerns would help ensure that the transformations generated by PretexEval maintain clinical accuracy and do not inadvertently alter the meaning of medical knowledge points during evaluation.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- PretexEval introduces a new approach to testing medical knowledge in language models. Rather than using fixed test questions, it dynamically generates diverse test cases that probe deeper understanding. This represents a significant methodological advancement in AI evaluation.\n- The authors create a reliable yet flexible testing framework by combining predicate transformations with natural language generation. The method is both principled and practical, with clear steps for reproduction.\n- The empirical results reveal important insights about current AI capabilities in medicine. The authors demonstrate that even state-of-the-art models struggle with consistent understanding by testing 12 different models with varying forms of the same medical knowledge. This finding has direct implications for clinical applications.\n- The framework's generalizability is a key strength. While demonstrated on two medical knowledge bases, the approach can extend to other medical domains and potentially beyond medicine. This broad applicability makes the contribution particularly valuable for the field.\n- LLMs across sizes and even API based models were evaluated and compared against simple LLM rephrasing", "weaknesses": "The study's methodology, while strong, has a few areas that could be strengthened. \n- Though clearly explained for simple relationships, the predicate transformation concept lacks a formal specification for handling complex medical relationships that don't cleanly map to simple predicates. \n- Further, the evaluation is limited to one relationship test in a multiple-choice setting, which may not fully capture the complexity of medical knowledge application. Exploring the handling of multiple variations, such as incorporating distractors or relationship swapping similar to GSM symbolic approaches, could provide a more comprehensive assessment of LLM capabilities in medical contexts. \n- Currently, the framework is only tested for binary settings; it would be useful to know how this setting can be applied to even a simple 4-class mcq or, ideally, an open-ended setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ifBJ4YkTFj", "reviewer": "Reviewer_hmcp", "summary": "The paper presents PretexEval, a novel evaluation framework designed to assess the mastery of medical knowledge by large language models (LLMs). Recognizing the limitations of existing evaluation methods, which often produce test samples with factual errors and insufficient diversity, PretexEval introduces a new schema based on predicate equivalence transformations. This approach generates a series of variant samples from a given medical knowledge point, ensuring both reliability and diversity in the test samples created. This framework was applied to evaluate 12 well-known LLMs using two medical knowledge bases. The findings reveal that, despite notable successes on standard QA benchmarks, current LLMs exhibit significant deficiencies in thoroughly understanding medical knowledge.", "questions": "- In comparison with the previous subsections, section 3.2.2 is harder to follow. To facilitate understanding, this section would greatly benefit from including examples (particularly lines 211-213).", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: The concept of dynamically generating test samples from medical knowledge bases using predicate equivalence transformations is innovative.\n\nQuality: The experimental design is robust, involving a comprehensive evaluation of 12 well-known LLMs across two distinct medical knowledge bases. The methodology for transforming predicates into diverse textual samples ensures that the evaluations are rigorous. The detailed ablation study and the use of two evaluation metrics (average accuracy and joint accuracy) further substantiate the quality of the research.\n\nClarity: The paper is well-organized and articulates ideas clearly and logically.\n\nSignificance: The findings highlight significant deficiencies in the current LLMs' ability to fully master medical knowledge, which is critical for their deployment in real-world medical scenarios.", "weaknesses": "1) The limitations of the proposed approach should be discussed in the paper:\n- The PretexEval framework heavily relies on predicate transformations to generate test samples. While this approach contributes to sample diversity, it may not adequately capture the complexity or the nuances of medical reasoning that goes beyond simple factual recall.\n- The current evaluation metrics, while useful, focus predominantly on binary true/false assessments of knowledge mastery. This binary approach might oversimplify the evaluation of medical knowledge, where a more nuanced understanding might be necessary.\n\n2) The code and datasets should be made publicly available for reproducibility purposes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GpfwgkPPM2", "reviewer": "Reviewer_D3So", "summary": "The paper introduces PretexEval, a framework for generating test samples from medical knowledge bases by generating variants of predicate expressions. This is done in three steps: \n\n(1) A knowledge point is converted to a predicate expression. \n\n(2) A predicate expression is converted to multiple predicate variants. \n\n(3) Predicate variants are converted back to a textual format.  \n\nAuthors prompt a number of current LLMs to verify the correctness of each statement (True/False), and find that current LLMs display subpar performance on these newly generated test samples, leading to the conclusion that they have not yet mastered medical knowledge.", "questions": "- In the Related Work section, authors point out that existing benchmarks crafted from medical experts may “become outdated or leaked to LLMs, affecting the comprehensiveness of evaluation”. How is the dataset created from the suggested framework any different? Further clarification regarding the authors’ intentions would be helpful.\n- How are the test samples for the “Direct” baseline constructed? More clarification on the template paraphrasing process would be helpful, as readers may not be familiar with related works that adopt template paraphrasing. Please provide the example of a template and how it is used to create a test sample for the \"Direct\" baseline.\n- Can you elaborate more on the qualifications of the “two experienced doctors” that have manually evaluated the test samples? Such as providing the years of experience or the doctors' specialities, or any relevant certifications that qualify them to evaluate these medical knowledge samples.\n- When analyzing the “Effect of Predicate Transformation Types”, have the authors conducted the “Direct + Double Negation” setting as well? Experiments in this setting seems to be important for the claim that “current LLMs exhibit relatively less proficiency in understanding negated expressions”.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The motivation of generating reliable test samples without factual errors, which is especially important in the medical domain, is clear and concise.\n- The proposed framework can be applied to any medical knowledge base.", "weaknesses": "- Opposed to the overall impression that the proposed framework enables diverse evaluation of LLMs’ medical knowledge, the paper only utilizes a single metric (true/false verification) for evaluation. A reviewer think that addressing multiple choice question answering or generating through (subject, relation, object) triplet makes more beneficial to elicit LLMs' medical knowledge. Although the authors do involve joint accuracy as a measure to account for all related samples that stem from a specific knowledge point, defining the knowledge span of current LLMs without diverse evaluation metrics (e.g. Single/Multiple Choice QA, Extractive/Abstractive QA) seems insufficient.\n- Further consideration on baseline selection seems necessary to support the efficacy of the proposed framework. Based on the initial problem statement that prompting LLMs to “directly generate test samples based on specific knowledge points” may lead to insufficient factuality, the LLMEval baseline from Table 1 seems to be unreliable. Could you provide more justification details why you added the LLMEval baseline?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)