Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models (2024)

Abstract: Clinical natural language processing faces challenges like complex medical terminology and clinical contexts. Recently, large language models (LLMs) have shown promise in this domain. Yet, their direct deployment can lead to privacy issues and are constrained by resources. To address this challenge, we delve into synthetic clinical text generation with LLMs for clinical NLP tasks. We propose an innovative, resource-efficient approach, CLINGEN, which infuses knowledge into the process. Our model involves clinical knowledge extraction and context-informed LLM prompting. Both clinical topics and writing styles are drawn from external domain-specific knowledge graphs and LLMs to guide data generation. Our extensive empirical study across 8 clinical NLP tasks and 18 datasets reveals that CLINGEN consistently enhances performance across various tasks by 7.7%-8.7% on average, effectively aligning the distribution of real datasets and enriching the diversity of generated training instances. Our code is available at https://github.com/ritaranx/ClinGen.

heading: 1 Introduction
text: Clinical Natural Language Processing (NLP) emerges as a distinct subfield including the extraction, analysis, and interpretation of unstructured clinical text (Wornow et al., 2023). Despite its significance, unique challenges exist for methodology development in clinical NLP. For example, clinical texts are often dense with abbreviations and specialized medical terminologies can be perplexing to standard NLP models (Lee et al., 2023). Fortunately, recent advances in Large Language Models (LLMs) (Brown et al., 2020; Chung et al., 2022; Ouyang et al., 2022; OpenAI, 2023b,a) provide a promising way to resolve these issues, as they contain billions of parameters and have been pretrained on massive corpora, thus inherently capture a significant amount of clinical knowledge (Agrawal et al.,
2022; Singhal et al., 2023). These progresses inspire the need for designing specialized approaches for adapting LLMs to clinical settings, which both address the terminology complexities and improve models through clinical data finetuning (Tu et al., 2023; Liu et al., 2023).
Despite the strong capacity of general LLMs, directly applying them to infer over clinical text data is often undesired in practice. Firstly, these LLMs often have billions of parameters that translate to significant computational resources even for inference, leading to increased infrastructure costs and long inference time. Furthermore, the sensitive patient information in the clinical text naturally raises privacy and regulatory compliance concerns (Meskó and Topol, 2023). To combat these challenges, generating synthetic training data using LLMs serves as a promising solution, as it leverages the capability of LLMs in a resourceefficient and privacy-centric way. When trained with synthetic data mimicking real-world clinical data, models can achieve high performance while obeying data protection regulations.
Synthetic data generation with LLMs is a popular research area in NLP (Meng et al., 2022; Ye et al., 2022a,b; Wang et al., 2023), with a focus on gemeral-domain data. However, adapting LLMs trained on general texts for generating high-quality clinical data poses distinct challenges. To assess the quality of data generated by existing methods, we carry out an evaluation centered on distribution and diversity, detailed in Section 3, which indicate a noteworthy data distribution shift. We further examine the clinically-related entity quantities and frequencies in synthetic data, where a notable decline is observed when contrasting synthetic data with ground truth data. While some research has delved into clinical data generation with language models, many of these efforts are tailored to specific tasks. Examples include medical dialogues (Chintagunta et al., 2021), clinical
15496
notes (Giorgi et al., 2023), and electronic health records (Ive et al., 2020; Wang and Sun, 2022). These studies often directly adopt language models for text generation, and sometimes on excessive training data. Till now, a unified principle to better adapt LLMs for generating synthetic text for facilitating clinical downstream applications is still missing.
Motivated by the above analysis, we propose CLINGEN, a clinical knowledge-infused framework for high-quality clinical text generation in few-shot scenarios. Our ultimate goal is to bridge the gap between synthetic and real data while enhancing topic diversity. Towards this end, we propose to utilize clinical knowledge extraction to contextualize the prompts. This includes generating clinical topics on entity and relation information from both KGs and LLMs and deriving writing style suggestions from LLMs. By doing this, CLINGEN integrates both non-parametric insights from external clinical knowledge graphs with the intrinsic parametric knowledge encoded in LLMs and enjoys higher diversity via dynamically composing different topics and writing styles together during the data generation process. It is worth noting that, CLINGEN only relies on minimal additional human efforts, and can be readily applied to a wide array of core tasks in clinical NLP.
Our contributions can be summarized as follows:
• We propose CLINGEN, a generic clinical knowledge-infused framework for clinical text data generation in few-shot settings. It can be readily applied to a wide range of tasks in clinical NLP.
• We present an analysis of the pitfall of existing data generation approaches for clinical text data, and propose a simple yet effective strategy to extract clinical knowledge and customize the prompts toward target clinical NLP tasks. This includes generating clinical topics from both KGs and LLMs and deriving writing style suggestions from LLMs.
• We conduct an exhaustive evaluation of synthetic clinical data generation across 8 clinical NLP tasks and 18 datasets. Empirical findings demonstrate that CLINGEN not only aligns more closely with the distribution of the original data but also amplifies the diversity of the generated training samples. The empirical performance gains are consistent across various tasks with different LLMs and classifiers (8.7% for PubMedBERTBase and 7.7% for PubMedBERTLarge).

heading: 2 Related Work
text: Generating additional training data enables a more precise analysis of medical text, and has gained more attention in the past years. Earlier research has employed data augmentation techniques to generate similar samples to existing instances with word substitution (Kang et al., 2021), back translation (Xie et al., 2020), pretrained transformers (Kumar et al., 2020; Zhou et al., 2022). But they often yield rigid transformations and the quality of the augmented text cannot be always guaranteed.
The emergence of LLMs has presented new possibilities for synthetic data generation (Meng et al., 2022, 2023; Ye et al., 2022a; Li et al., 2023). However, these methods often use generic and simple prompts that may not fully capture domain-specific knowledge, thus potentially limiting the quality of the generated data. Liu et al. (2022a); Chung et al. (2023); Yu et al. (2023) employ interactive learning to generate instances, at the cost of additional human efforts. Several recent studies explore LLMbased synthetic data generation for clinical NLP. Tang et al. (2023) rely on a much larger training set to generate candidate entities, which disregards the practical low-resource setting (Perez et al., 2021). Moreover, these studies often concentrate on specific target tasks, thus lacking generality for diverse clinical NLP scenarios.
On the other hand, several works aimed at optimizing prompts using LLMs (Zhou et al., 2023; Wang et al., 2024) or knowledge graphs (Liu et al., 2022b; Chen et al., 2022b), yet they mainly focus on refining prompts to obtain the answer for the given input, and the prompt template often remains unchanged. Instead, we focus on the different task of generating training instances. By composing different topics and styles together, we can generate diverse templates for prompting LLMs to improve the quality of the synthetic data.

heading: 3 Preliminary Study
text: This section first presents the foundational setup of synthetic data generation. Then, we provide an in-depth investigation into the pitfalls of existing synthetic data generation methods.

heading: 3.1 Problem Setup
text: In this paper, we study synthetic data generation under the few-shot setting. The input consists of a training set D = {(xi, yi)}Ki=1, where (xi, yi) represents an input text and its corresponding label
yi ∈ Y for the i-th example. K denotes the total number of training samples, which is kept at a very small value (5-shot per label). The primary objective is to harness the LLM M to generate a synthetic dataset, denoted as D̃ = {(x̃i, ỹi)}Ni=1, where N is the number of generated samples (N ≫ K). We use ρ(·) to denote the generation process from the LLM. For each downstream task, we fine-tune a classifier Cθ (a moderate-size pretrained language model) parameterized by θ on the synthetic dataset D̃ for evaluating its quality.1

heading: 3.2 Limitations of Existing Methods
text: Denote the task-specific prompts for class label name j as pj , we take a closer look at the synthetic text data generated by two representative approaches: ZeroGen (Ye et al., 2022a), which directly instructs LLMs for data generation as D̃Zero ∼ ρj∼Y(·; pj), and DemoGen (Yoo et al., 2021; Meng et al., 2023), which augments the prompt with few-shot demonstrations D as D̃Demo ∼ ρj∼Y (·; [pj ,D]). The prompt format of ZeroGen and DemoGen are in Appendix E.3. We observe that these methods often introduce distribution shifts and exhibit limited diversity, which can lead to suboptimal downstream performance. Distribution Shift. An inherent issue when adapting LLMs to specific domains for text generation is the distribution shift, given that LLMs are primarily trained on vast amounts of web text in general domains. To quantify the data distribution shift, we employ Central Moment Discrepancy (CMD) (Zellinger et al., 2017) to measure the gap between synthetic and real data across six clinical NLP datasets — a high CMD value indicates a large gap between two distributions2. Figure 1(a) illustrates that both ZeroGen and DemoGen exhibit
1While In-context Learning (Brown et al., 2020) can also be utilized, it is often hard to fit all generated instances into the context window, especially for datasets with high cardinality.
2Details of calculating CMD is in Appendix A.
elevated CMD scores. Despite the inclusion of few-shot demonstrations in DemoGen, this limitation remains evident, indicating a notable disparity between the ground-truth and synthetic data. Limited Diversity. Clinical datasets in real-world scenarios often include rich domain knowledge that can be challenging to replicate in synthetic data. We evaluate synthetic dataset diversity by using both entity quantity and their normalized frequencies. The results are illustrated in Figures 1(b) and 1(c). Our analysis reveals that datasets generated by ZeroGen and DemoGen exhibit a limited number of clinical entities, having a substantial discrepancy with the ground truth. Furthermore, it is highlighted that only a minority of potential entities and relations are frequently referenced across instances, while the majority are generated infrequently.
To explicitly illustrate the limitations, we present a case study in Figure 9, Appendix B. The comparison reveals that samples generated by ZeroGen and DemoGen lack sufficient details present in the ground truth data. Besides, the generated samples adhere to a more uniform style, while the ground truth encompasses various situations and writing styles, including urgent and informal inquiries.

heading: 4 Knowledge Infused Data Generation
text: Section 3 highlights the necessity of domaintailored knowledge for clinical synthetic data generation. In pursuit of this, we present CLINGEN, a knowledge-informed framework for clinical data generation. The overview of CLINGEN is shown in Figure 2. This two-step methodology harnesses the emergent capabilities of LLMs and external knowledge from KGs to facilitate the synthesis of clinical data, even with few-shot examples only.

heading: 4.1 Clinical knowledge extraction
text: Contrary to previous studies (Ye et al., 2022a,b; Meng et al., 2023) which employ generic queries pj
to prompt LLMs for text generation, CLINGEN emphasizes refining clinically informed prompts. This approach aims to extract rich clinically relevant knowledge from parametric (e.g. LLMs) or nonparametric sources (e.g. knowledge graphs) and tailor it to clinical NLP tasks. To realize this, our modeling contains two dimensions including clinical topics T and writing styles W , which are integrated into the original prompts to infuse domain-specific knowledge. The Clinical topic refers to a clinical entity (e.g., disease) or relation (e.g., the relationship between diseases and medications), which is usually a phrase, while the writing style is a phrase that depicts the tone, and overall presentation of the text. By composing different topics and writing styles together, CLINGEN provide a diverse suite of prompts, resulting in a wider spectrum of text produced from the LLM M. For details of prompt formats across various tasks, please see Appendix E.

heading: 4.1.1 Clinical Topics Generation
text: We provide two choices to generate clinical topics T – one is to sample related entities or relations from external KG, and the other is to query relevant knowledge from LLM. Topics TKG sampled from Non-Parametric KGs. Healthcare KGs offer a rich collection of medical concepts and their complex relationships, which organizes medical knowledge in a structured way (Li et al., 2022). In our study, we employ the integrative biomedical knowledge hub (iBKH) as the KG (Su et al., 2023) G to generate topics TKG ∼ query(G) due to its broad coverage over clinical entities. To illustrate, for the Disease Recognition task (NCBI, Dogan et al. (2014)), we extract all disease nodes e from the iBKH to bolster the medical information as T NCBIKG ∼ query(Gdisease),
Gdisease = {e ∈ G| type(e) = disease}. As another example, we retrieve links between chemicals c and diseases d for the chemical and disease relation extraction (CDR, Wei et al. (2016)) as T CDRKG ∼ query(Grelation_cd), Grelation_cd = {⟨c, r, d⟩ ∈ G| type(r) = has_relation}. By injecting information from the KG into the data generation step, we ensure the generated samples are more contextually accurate and semantically rich. Topics TLLM queried from Parametric LLMs. Pre-trained on extensive text corpora such as medical literature, LLMs provide an alternative method for acquiring domain knowledge. Specifically, we aim to harness the rich clinical domain knowledge encoded in ChatGPT (gpt-3.5-turbo-0301) to augment the prompt. The incorporated prior knowledge from LLMs focus on entity types that hold relevance within clinical text datasets, including diseases, drugs, symptoms, and side effects. For each of entity types ei, we prompt the LLMs by formulating inquiries q(ei), e.g., “Suppose you are a clinician and want to collect a set of <Entity Type>. Could you list 300 entities about <Entity Type>?”. These crafted conversational cues serve as effective prompts to retrieve clinically significant entities from the rich domain knowledge within LLMs as TLLM ∼ ρ (·; q(ei)). For each entity type, we generate 300 entities for synthetic data generation.

heading: 4.1.2 Clinical Writing Styles Suggestion
text: Styles suggested by LLMs. To address the limitations mentioned in Sec 3.2 and introduce a diverse range of writing styles W for synthetic samples, we leverage the powerful LLM to suggest candidate writing styles for each task. Specifically, for the task i, we incorporate task names ni into our prompts pstylei (e.g., disease entity recognition, recognizing text entailment) and integrate few-shot
demonstrations dstylei . We then engage LLM in suggesting several potential sources, speakers, or authors of the sentences as W ∼ ρ ( ·; [pstylei , d style i ] ) . Responses such as “medical literature" or “patientdoctor dialogues" are augmented into the prompts to imitate the writing styles found in real datasets.

heading: 4.2 Knowledge-infused Data Generation
text: With the generated topics and styles, the key challenge becomes how to leverage them to extract rich clinical information from the LLM for improving synthetic data quality. Directly putting all the elements to enrich the prompt is often infeasible due to the massive size of entities. To balance informativeness as well as diversity, we propose a knowledge-infused strategy, where for each class label name j ∈ Y , the collected clinical topics and writing styles serve as the base unit. In each step, we randomly sample a topic t ∈ T and a writing style w ∈ W from the candidate set to augment the prompt for class j ∈ Y as pClinj (t, w) = [pj , t, w]. Then, we use the augmented prompt pClinj (t, w) together with the few-shot demonstrations D to generate the synthetic dataset D̃Clin as
D̃Clin ∼ ρj∼Y,t∼T ,w∼W (·; [pj , t, w] ,D) . Despite its simplicity, this strategy enjoys several merits: (1) Clinical infusion: the clinical context is incorporated into the prompts to directly guide data generation; (2) Diversity: it encourages data diversity via dynamically composing different entities and writing styles into prompts; (3) Flexibility: it is compatible with different sources of T and W without reliance on specific knowledge formats. Consequently, the quality and clinical relevance of the generated synthetic data are enhanced. While some works focus on prompt optimization for data generation or other NLP tasks, they typically utilize a fixed prompt and optimize this prompt format, which is orthogonal to CLINGEN.

heading: 4.3 Language Model Fine-tuning
text: After generating synthetic data D̃, we fine-tune a pre-trained classifier Cθ for each downstream task. Following Meng et al. (2023), we first fine-tune Cθ on D with standard supervised training objectives on few-shot examples (denoted as ℓ(·)) in Stage 1, then on synthetic data D̃ in Stage 2 as
θ(1) = min θ E(x,y)∼Dℓ (f(x; θ), y) , θ(2) = min θ E (x̃,ỹ)∼D̃ℓ (f(x̃; θ), ỹ) , θinit = θ (1).
It’s important to highlight that we strictly follow a standard fine-tuning process and avoid using any extra techniques: (1) for standard classification tasks, ℓ(·) is the cross-entropy loss; (2) for multilabel classification tasks, ℓ(·) is the binary crossentropy loss; (3) for token-level classification tasks, we stack an additional linear layer as the classification head and ℓ(·) is the token-level cross-entropy loss. The design of advanced learning objectives as well as data mixing strategies, while important, are orthogonal to the scope of this paper.

heading: 5 Empirical Evaluation
text: Given our focus on data generation, our major interest lies in faithfully evaluating different synthetic text generation approaches under few-shot scenarios, rather than competing in a “state-of-the-art" race with general few-shot NLP methods. The following questions particularly intrigue us: RQ1: How does CLINGEN perform when compared with baselines on different downstream tasks? RQ2: What impact do factors like LLM generators and synthetic data size have on the performance of CLINGEN? RQ3: How is the quality of the synthetic data generated by CLINGEN and baselines?

heading: 5.1 Experiment Setup
text: We conduct experiments in the few-shot settings with 5 examples for each class. We employ ChatGPT (OpenAI, 2023b) (gpt-3.5-turbo-0301) as the LLM generator M3 and maintain the same amount of synthetic training data for both CLINGEN and baselines for a fair comparison. The pre-trained PubMedBERT (Gu et al., 2021) is then applied to fine-tune on the synthetic data for both CLINGEN and baselines, where we consider both the Base and Large model. Datasets and Tasks. We undertake a comprehensive evaluation of 18 datasets across a diverse array of tasks in clinical NLP benchmarks (Peng et al., 2019; Fries et al., 2022): 2 text classification, 3 relation extraction (RE), 3 natural language inference (NLI), 2 fact verification, 2 question answering (QA), 1 sentence similarity (STS), 4 Named Entity Recognition (NER), and 1 attribute extraction datasets. Please see Appendix C for descriptions and the statistics of each dataset. Baselines. We compare CLINGEN with 10 baselines in total, including 6 data augmentation and
3Studies on using Medical LLMs are in Appendix J.
4 LLM-based data generation techniques. See Appendix D for their descriptions. Implementation Details. For implementation, we use PyTorch (Paszke et al., 2019) and HuggingFace (Wolf et al., 2019). For each dataset, we randomly sample 5 examples from each class to provide few-shot demonstrations and keep a validation set of the same size. During the data generation process when we call the ChatGPT APIs (OpenAI, 2023b), we set the parameter top_p = 1.0 and temperature t = 1.0 to balance between the quality of the generated text as well as diversity (Chung et al., 2023; Yu et al., 2023)4. In the experiments, We generate 5000 synthetic training data for both CLINGEN and the baselines and report the average performance over 3 random seeds for all the results. With the generated synthetic dataset, we follow the common few-shot learning setting (Perez et al., 2021) to train all the models for 6 epochs and use the model with the best performance on the validation set for evaluation. During the PubMed-
4We do not further increase t, as previous analysis (Chung et al., 2023; Yu et al., 2023) has shown that increasing t to larger value does not help with additional performance gain.
BERT fine-tuning, we adopt AdamW (Loshchilov and Hutter, 2019) for optimization with a linear warmup of the first 5% steps and linear learning rate decay. The learning rate is set to 2e-5 for Base and 1e-5 for Large, and the maximum number of tokens per sequence is 256.

heading: 5.2 Model Performance with Synthetic Data
text: Table 1 summarizes the experimental results. Due to space limits, we report the average performance over all datasets for each task, but provide the detailed results for each dataset in Tables 7, 8, 9 in Appendix F. Based on the experimental results, we have the following findings: ⋄ Our approach, CLINGEN, consistently outperforms the baselines across all tasks. The average performance gain over all main metrics is 8.7% at Base scale and 7.7% at Large scale. LLM-based methods outperform traditional DA techniques, showcasing their ability to capture task-specific information from a few examples. DemoGen and ProGen’s gains over ZeroGen highlight the positive impact of few-shot examples. Despite being one of the most powerful data generation approaches,
S3’s gains are marginal in the few-shot setting due to its reliance on large validation sets. ⋄ In token classification tasks, CLINGEN performs better with KG compared to LLM due to the better alignment between the task’s target and the generated domain knowledge, where the extracted topics serve as direct labels. Conversely, single-sentence and sentence-pair tasks favor LLM-based knowledge extraction. This could be because (1) These tasks prioritize sentence comprehension over specific terminologies, and some specialized terms might even impede LLM comprehension. (2) KGs may not always contain the required information, e.g., certain relations in chemical/protein relation extraction tasks, limiting performance gains. ⋄ Some DA methods are task-specific, limiting their generalizability. For example, LightNER and KGPC are designed for NER. It is also non-trivial to apply Back Translation to NER or RE, as it requires locating related entities in the generated sentence accurately. In contrast, CLINGEN is flexible and can be readily applied to various tasks.

heading: 5.3 Ablation and Parameter Studies
text: Effect of Different LLM Generators. To investigate the impact of various LLMs on CLINGEN, we utilize InstructGPT (text-curie-001) (Ouyang et al., 2022) and GPT-4 (OpenAI, 2023a). Note that we only generate 500 samples in the GPT-4 setting due to budget constraints, but we provide the results of GPT-3.5 with same amount of synthetic samples for a fair comparison. From Figure 3 we observe that CLINGEN generally outperforms the best baseline in all settings. Additionally, we observe generally improved performance with larger models, as they often have better capabilities to fol-
low our designed instructions for the given prompts. See Appendix G for more results. Effect of Size of Synthetic Data. In Figure 4 (and more in Appendix G), we study the effect of the size of synthetic data. The result shows that CLINGEN consistently outperforms the best baseline, using only around 10% of the synthetic examples. This illustrates that incorporating domain knowledge and increasing the diversity of the prompts could be an effective way to improve the sample efficiency and narrow the gap between the performance of synthetic and ground-truth datasets. Comparison with few-shot inference via prompting LLM. We also evaluate the performance of 5- shot in-context learning with ChatGPT and 3 medical LLMs, namely PMC-LLaMa-13b (Wu et al., 2023), MedAlpaca-13b (Han et al., 2023). Due to budget limits, we run experiments on datasets with few testing samples for each task. As presented in Table 2, CLINGEN at PubMedBERTLarge scale achieves better results on 5 out of 6 datasets than ChatGPT few-shot learning, which uses ∼ 530× more parameters. One exception is for PUBHEALTH, as it requires complex reasoning abilities that PubMedBERTLarge may not fully possess. Three medical LLMs, on the other hand, perform less effectively than both CLINGEN and GPT-3.5 due to fewer parameters, limited reasoning capabilities, and training on a general medical corpus
unsuited for the tasks. Overall, CLINGEN offers cost-effective and time-efficient advantages. While it entails a one-time investment in both money and time for synthetic training data generation, subsequent prediction relying on a moderate-sized model is much more efficient. Besides, the continued use of ChatGPT for inference on new testing data incurs ongoing time and financial costs, while our model requires zero additional costs for new data. Effect of Topic Extraction and Style Suggestion. We inspect different components of CLINGEN in Table 3. It is observed that both Topics Extraction and Style Suggestion contribute to model performance as they enhance the relevance of generated samples to domain knowledge and introduce greater diversity. Different from the other datasets, MEDIQA-RQE shows more performance gain incorporating writing style than topics. It is because NLI tasks focus on capturing the relationships between two sentences while incorporating additional knowledge entities does not directly help the model improve the reasoning ability.

heading: 6 Quality Analysis of the Synthetic Data
text: Data Distribution Measures. Figure 5(a) shows the t-SNE plot of data generated by CLINGEN and baselines compared with the ground truth. This visualization demonstrates that CLINGEN exhibits
a greater overlap with the ground truth, indicating a similar distribution as the original dataset. In addition, as depicted in Figure 6(a), the embedding of CLINGEN aligns more closely with the ground truth distribution than other baselines across all six datasets, further justifying the efficacy of CLINGEN for mitigating the distribution shift issue. Diversity Measures. Table 4 calculates the average cosine similarity for sample pairs using SentenceBERT embeddings. Compared to baselines, the dataset generated with CLINGEN exhibits lower cosine similarity and the average similarity is close to that of the ground truth training data, which shows CLINGEN could render more diverse data.
Moreover, Figure 6(b) highlights CLINGEN covers a broader range of entities than baselines, with CLINGEN w/ KG capturing more entities due to KGs’ extensive knowledge. Figure 6(c) reflects CLINGEN has a more balanced entity frequency distribution aligned with ground truth, ensuring diverse topic coverage. Case Study. In Figure 5(b), we present a case
study of examples generated by CLINGEN with LLM on MEDIQA-RQE dataset, which consists of consumer health queries. The examples reveal that the sentences generated by CLINGEN include more extensive contextual information compared with the baseline. These sentences closely resemble the queries people might pose in real-life scenarios. Study on Factual Consistency. A human evaluation was carried out to assess the factual accuracy of the generated outputs across six representative tasks: LitCovid, CDR, Mediqa-RQE, MQP, PubHealth, and BC5CDR. For each task, a sample of 100 examples per class was randomly selected. Medical students then examine the generated text and evaluate its factuality. The findings from this rigorous human study revealed no instances of misinformation or hallucinated content in the randomly sampled examples, verifying the system’s reliability in generating factually sound outputs. Monetary Cost We display the monetary cost of CLINGEN for calling the OpenAI APIs, with a comparison with prompting GPT-3.5 for direct inference and DemoGen. From the values shown in Table 5, we observe that inference via GPT-3.5 generally has a higher cost, as it needs to input all the testing samples for prompting. In contrast, DemoGen has a relatively lower cost, because it does not include the topics and writing styles to the prompts as CLINGEN does.

heading: 7 Conclusion
text: In this work, we study clinical text data generation using LLMs. We thoroughly assess existing methods for clinical data generation and identify issues including distribution shifts and limited diversity. To tackle these challenges, we introduce CLINGEN, a framework that leverages clinical knowledge from non-parametric KGs and parametric LLMs. This empowers data generation by utilizing clinical topic knowledge and real-world writing styles in domain-specific prompts. Our extensive empirical evaluations across 8 clinical NLP tasks and 18 datasets, compared to 10 baseline methods, consistently show that CLINGEN improves task
performance, aligns closely with real data, and enhances data diversity. We expect CLINGEN can be seamlessly incorporated into a broad suite of clinical text tasks to advance clinical NLP research.

heading: Acknowledgement
text: We thank the anonymous reviewers and area chairs for valuable feedbacks. This research was partially supported by the Emory Global Diabetes Center of the Woodruff Sciences Center, Emory University. Research reported in this publication was supported by the National Institute Of Diabetes And Digestive And Kidney Diseases of the National Institutes of Health under Award Number K25DK135913. The research also receives partial support by the National Science Foundation under Award Number IIS-2145411. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. We also thank Microsoft for providing research credits under the Accelerating Foundation Models Research Program.
Limitation
In this work, we propose CLINGEN to better harness the LLM for synthetic text data generation. Despite its strong performance, we mainly verify their efficacy from their empirical performance, sample diversity, and distribution gaps. There are still some limitations to this work: Factuality of LLM-generated Text. One issue with LLM-based synthetic data generation is the phenomenon of hallucination, wherein the model generates information that does not ground in reality (Zhang et al., 2023). This can lead to the propagation of misinformation, which may have negative impacts on the clinical domain. However, we have conducted a human study to justify that our generated synthetic data does not suffer from the issue of misinformation. Application to other type of clinical data. Apart from text, there are other types of clinical data: For example, EHR data falls within a distinct
modality (i.e. tabular data) from textual data, which may require different methodologies and approaches (Wornow et al., 2023).
Ethics Consideration
On specific issue is about patient privacy. To eliminate this concern, we carefully select the five fewshot demonstrations to ensure they are fully free from any Protected Health Information (PHI) related to patients. We also make a deliberate effort to avoid any instructions that can potentially extract sensitive patient information within the prompts. Lastly, we conduct rigorous inspections of the generated synthetic data across all covered tasks to affirm that no such private information exists in the synthetic data generated by our method. In addition, we have opted out of human review for the data by completing the Azure OpenAI Additional Use Case Form5. This allows us to use the Azure OpenAI service while ensuring Microsoft does not have access to patient data.

heading: A Details on the Calculation of CMD
text: We introduce the Central Moment Discrepancy (CMD) (Zellinger et al., 2017), which is a widely used metric to measure the domain shift in the area of domain-invariant representation learning. Let X = (x1, . . . , xn) and Y = (y1, . . . , yn) be bounded feature vectors independent and identically distributed from two probability distributions p and q. The central moment discrepancy metric (CMD) is defined by
CMD(p, q) = 1
|b− a|∥E(X)− E(Y )∥2
+ ∞∑
k=2
1
|b− a|k ∥ck(X)− ck(Y )∥2
where E(X) is the expectation of X , and
ck(X) =
( E ( N∏
i=1
(Xi − E (Xi))ri ))
r1+...+rN=k r1,...,rn≥0
is the central moment vector of order k. To estimate the CMD efficiently without infinite-order calculation, we follow (Zellinger et al., 2017) and use a K-order approximation of CMD as
CMDk(p, q) = 1
|b− a|∥E(X)−E(Y )∥2
+ K∑
k=2
1
|b− a|k ∥Ck(X)− Ck(Y )∥2
where E(X) = 1|X| ∑
x∈X x is the empirical expectation vector computed on the sample X and Ck(X) = E ( (x−E(X))k ) is the vector of all kth order sample central moments of the coordinates of X6. To adapt CMD in our work, we set K = 5, and use the embedding from SentenceBERT (Reimers and Gurevych, 2019) to calculate the embedding X,Y for instances.

heading: B Additional Preliminary Studies
text: We present additional preliminary studies of the t-SNE plots in Figure 7 and the regularized entity frequencies in Figure 8. In Figure 7, we visualize the embeddings7 of both the ground truth training
6The implementation of CMD is available at https://gist.github.com/yusuke0519/ 724aa68fc431afadb0cc7280168da17b
7We employ SentenceBERT (Reimers and Gurevych, 2019) as the text encoder.
data and synthetic datasets generated via two representative methods. Overall, these methods use generic prompts (see Appendix E.3 for details) with minimal domain-specific constraints. These results further justify the distribution shift issue mentioned in section 3.2, demonstrating that the limited diversity as well as the distribution shift issue generally exists for a broad range of clinical NLP tasks.
Figure 9 shows a case study, where we randomly select one sample from each class within the training set generated by ZeroGen and DemoGen. These selected samples are compared with the ground truth data from the MEDIQARQE dataset, which aims to predict whether a consumer health query can entail an existing Frequently Asked Question (FAQ). It is evident that the samples generated by ZeroGen and DemoGen exhibit a limited range of writing styles and tend to follow a specific template, whereas the ground truth sample contains more contextual elements that are typically encountered in real-life scenarios.

heading: C Dataset Description
text: The evaluation tasks and datasets are summarized in Table 6. Note that the number of training samples indicates the size of the original training set. Specifically, we consider the following datasets:
• Single-Sentence Tasks
◦ Text Classification: * The LitCovid dataset (Chen et al., 2021)
consists of COVID-19-related publications from PubMed. The task is to predict the topics of the sentences, including “Epidemic Forecasting", “Treatment", “Prevention", “Mechanism", “Case Report", “Transmission", and “Diagnosis".
* The HOC dataset (Baker et al., 2015) also extracts sentences from PubMed articles, each annotated at the sentence level. The task is to predict the topics of the sentences, including “evading growth suppressors", “tumor promoting inflammation", “enabling replicative immortality", “cellular energetics", “resisting cell death", “activating invasion and metastasis", genomic instability and mutation", “inducing angiogenesis", “sustaining proliferative signaling", and “avoiding immune destruction".
◦ Relation Extraction:
* The GAD (Bravo et al., 2015) dataset is to predict whether there is a relation between the given disease and gene in the sentences. Note that the original annotation for this dataset is Noisy. To remedy this issue, we relabel 350 examples from the original test set to form a clean subset for faithful evaluation.
* The CDR (Wei et al., 2016) dataset is to predict whether the provided chemical can induce the disease in the sentences.
* The ChemProt (Taboureau et al., 2010) dataset focuses on the chemical-protein relations, and the labels include “Upregulator", “Downregulator", “Agonist", “Antagonist", “Product_of" and “No relation".
• Sentence-Pair Tasks
◦ Natural Language Inference (NLI): * The MedNLI (Shivade, 2017) dateset con-
sists of sentences pairs derived from MIMIC-III, where we predict the relations between the sentences. The labels include “entailment", “neutral" and “contradiction".
* The MEDIQA-NLI (Ben Abacha et al., 2019) dataset comprises text-hypothesis pairs. Their relations include “entailment", “neutral" and “contradiction".
* The MEDIQA-RQE (Abacha and DemnerFushman, 2016) dataset contains NIH consumer health question pairs, and the task is to recognize if the first question can entail the second one. ◦ Fact Verification: * The PUBHEALTH (Kotonya and Toni,
2020) encompasses claims paired with journalist-crafted explanations. The task is to predict the relations between the claim and evidence, including “Refute", “Unproven", “Support", and “Mixture".
* The HealthVer (Sarrouti et al., 2021) contains evidence-claim pairs from search engine snippets regarding COVID-19 questions. The relations between claims and evidences are chosen from “Refute", “Unproven", and “Support". ◦ Question Answering (QA): * The PubmedQA task (Jin et al., 2019) en-
tails responding to inquiries regarding the abstracts of biomedical research papers.
* The BioASQ task (Tsatsaronis et al., 2015) spans multiple question types, including factoid, list, summary, and yes/no questions derived from expert-reviewed biomedical research papers. ◦ Sentence Similarity (STS): * the MQP (McCreery et al., 2020) dataset
comprises a collection of medical question pairs designed for identifying semantically similar questions. The task is to predict
whether the two questions are equivalent or not.
• Token Classification Tasks
◦ Named Entity Recognition (NER): * The BC5CDR-Disease (Li et al., 2016) is
to recognize diseases in the sentences.
* The BC5CDR-Chemical (Li et al., 2016) is to recognize chemicals in the sentences.
* The NCBI-Disease (Dogan et al., 2014) is to recognize diseases in the sentences.
* The CHEMDNER (Krallinger et al., 2015) is to recognize chemicals in the sentences. ◦ Attribute Extraction (MedAttr): * The CASI dataset (Agrawal et al., 2022;
Moon et al., 2014) aims to identify interventions including medication, dosage, route, freq, reason, duration

heading: D Baseline Details
text: In this section, we give a detailed introduction for all baselines used in this study.
Data Augmentation Methods:
• DA-Word Sub (Ribeiro et al., 2020): It performs word substitution for few-shot demonstrations to create new training sample. Specifically, we follow Checklist (Ribeiro et al., 2020) and maintain a word list to generate new examples.
• DA-Back Translation (Xie et al., 2020): It employ back translation to augment the training data (Xie et al., 2020), including translating text from the target language to the source language and then back to the target language.
• DA-Mixup (Chen et al., 2020; Zhang et al., 2020): It adds interpolation on the embedding space of the training examples to create virtual augmented examples.
• DA-Transformer (MELM) (Kumar et al., 2020; Zhou et al., 2022): It introduces a conditional data augmentation technique that prepends class labels to text sequences for pretrained transformer-based models. Specifically, it leverage the sequence to sequence transformer to perform conditional text generation based on the seed examples.
• LightNER (Chen et al., 2022a): It adopts a seq2seq framework, generating the entity span sequence and entity categories under the guidance of a self-attention-based prompting module. It is designed specifically for NER tasks.
• KGPC (Chen et al., 2023): It injects the semantic relations of the knowledge graph to sequence to text generation models to perform knowledge-guided instance generation for fewshot biomedical NER. It also only applies to NER tasks.
LLM-based Generation Methods.
• ZeroGen (Ye et al., 2022a): It generates a dataset using simple class-conditional prompts and then trains a tiny task-specific model for zero-shot inference. We follow the prompting
method mentioned in their original paper as implementation, which does not consider any style information as well as domain knowledge.
• DemoGen (Meng et al., 2023; Yoo et al., 2021): It leverages LLMs to synthesize novel training data by feeding few-shot samples as demonstrations to guide the data generation process. Note that we focus on using the black-box LLM as the generator, thus we do not tune the LLM as (Meng et al., 2023).
• ProGen (Ye et al., 2022b): It first identifies the most important examples from the generated synthetic data using the influence function, then adds these examples as demonstrations to generate new training instances. To ensure fair comparison, we also add the few-shot demonstrations for data generation.
• S3 (Wang et al., 2023): It is a synthetic data generation method that iteratively extrapolates errors made by the classifier model trained on synthetic data leveraging a large language model. To adapt it in our few-shot setting, we use fewshot demonstrations D as the validation set.

heading: E Prompt Format
text: E.1 The prompts for Writing Styles Suggestion with CLINGEN
Listing 1: Prompt Format for writing styles suggestion with CLINGEN.
Suppose you need to generate a synthetic clinical text dataset on [task] tasks. Here are a few examples from the original training set: [demonstrations] Please write three potential sources , speakers or authors of the sentences.
[task]: The task names for each specific task. [demonstrations]: The few-shot demonstrations from the original training set.
E.2 The prompts for Data Generation with CLINGEN
In the following prompt format, [topic] and [style] are randomly sampled from the topics candidate set and styles candidate set we formulate in the knowledge extraction step, respectively.
Named entity recognition tasks:
Listing 2: Prompt Format for NER tasks with CLINGEN.
Suppose you need to create a dataset for [domain] recognition. Your task is to:

heading: 1. generate a sentence about [domain],

heading: 2. output a list of named entity about [domain] only ,

heading: 3. the sentence should mimic the style of [style],

heading: 4. the sentence should mention the [domain] named [topic].
text: [domain]: “disease" for BC5CDR-Disease and NCBI-Disease; “chemical" for BC5CDR-Chemical and CHEMDNER.
Medication attributes tasks: Listing 3: Prompt Format for medication attributes tasks with CLINGEN.
Suppose you need to create a dataset for clinical attributes recognition. Your task is to:

heading: 1. generate a sentence about clinical attributes , The Clinical
text: Attributes you need to extract
include "Medication", "Dosage", " Route", "Frequency", "Reason", " Duration ". For each attribute class , please return a list of attributes within the class that occurs in the Sentence.

heading: 2. the sentence should mimic the style of [style],

heading: 3. the sentence should be relevant to [topic].
text: Text classification tasks: Listing 4: Prompt Format for text classification tasks with CLINGEN.
Suppose you need to create a dataset for [domain]. Your task is to: 1. generate a sentence about [domain]. 2. the sentence should mimic the style of [style].
3. the sentence should be relevant to the subtopic of [topic] for [class_name].
[domain]: “COVID-19 Literature" for LitCovid and “Cancer Document" for HOC. [class_name]: the label name for this generated sample, listed in Appendix C. Relation extraction tasks:
Listing 5: Prompt Format for relation extraction tasks with CLINGEN.
Suppose you need to generate synthetic data for the biomedical [domain] task. Your task is to:

heading: 1. give a sentence about [class_name] relation between [entity0] and [entity1]
text: 2. the sentence should discuss the [entity0]: [topic0] and [entity1]: [topic1] with the relation [label_desc].

heading: 3. the sentence should mimic the style of [style].
text: [domain]: “Disease Gene Relation" for GAD, “Chemical Disease Relation" for CDR, and “Chemical Protein Relation" for ChemProt. [entity0] and [entity1]: “disease" and “gene" for GAD, “chemical" and “disease: for CDR, and “chemical" and “protein" for ChemProt. [class_name]: the label name for this generated sample, listed in Appendix C. [label_desc]: the description of the selected label. For example, the label “upregulator" in ChemProt has a description of “the chemical activates expression of the protein."
Natural language inference tasks:
Listing 6: Prompt Format for generating the first sentence in NLI tasks with CLINGEN.
Suppose you need to create a set of [content]. Your task is to:

heading: 1. generate one sentence for a [content].

heading: 2. the [content] should be relevant to [topic],

heading: 3. The [content] should mimic the style of [style].
text: [content]: “health question" for MEDIQA-RQE, “claim" for MEDIQA-NLI, MedNLI and MQP, and “health news" for PUBHEALTH and HealthVer.
Listing 7: Prompt Format for generating the second sentence in NLI tasks with CLINGEN.
Suppose you need to create a pair of sentences for the [domain]
task with the label '[class_name]'. Given the [content]: '
[first_sentence]', Your task is to:

heading: 1. generate one short [content] about [topic] so that [label_desc].

heading: 2. The [content] should mimic the style of the first sentence.
text: [domain]: “Question Entailment" for MEDIQARQE, “Natural Language Entailment" for MEDIQA-NLI and MedNLI, “Fact Verification" for PUBHEALTH and HealthVer, and “Sentence Similarity Calculation" for MQP. [content]: “health question" for MEDIQARQE, “hypothesis" for MEDIQA-NLI, MedNLI, “evidence" for PUBHEALTH and HealthVer, and “sentence" for MQP.
[class_name]: the label name for this generated sample, listed in Appendix C. [label_desc]: the description of the selected label. For "entailment", the description is "we can infer the [content] from the given sentence". For "neutral", the description is "there is no clear relation between the [content] from the given sentence". For "contradict", the description is "we can refute the [content] from the given sentence". [first_sentence]: the first sentence we generate
E.3 Prompts for ZeroGen, DemoGen, ProGen We use the same set of prompts for ZeroGen, DemoGen and ProGen, while DemoGen and ProGen have additional demonstrations augmented to the prompts. DemoGen uses the few-shot examples in the training set as demonstrations, and ProGen leverages feedbacks from previous rounds to iteratively guide the generation.
Named entity recognition tasks:
Listing 8: Prompt Format for NER tasks with baselines.
Suppose you need to create a dataset for [domain] recognition. Your task is to generate a sentence about [domain] and output a list of named entity about
[domain] only.
[domain]: “disease" for BC5CDR-Disease and NCBI-Disease; “chemical" for BC5CDR-Chemical and CHEMDNER.
Medication attributes tasks: Listing 9: Prompt Format for medication attributes tasks with baselines.
Suppose you need to create a dataset for clinical attributes recognition. Your task is to generate a sentence about clinical attributes , The Clinical Attributes you need to extract
include "Medication", "Dosage", " Route", "Frequency", "Reason", " Duration ". For each attribute class , please return a list of attributes within the class that occurs in the Sentence.
Text classification tasks:
Listing 10: Prompt Format for text classification tasks with baselines.
Suppose you are a writer for [domain]. Your task is to give a synthetic [domain] about [class_name].
[domain]: “COVID-19 Literature" for LitCovid and “Cancer Document" for HOC. [class_name]: the label name for this generated sample, listed in Appendix C. Relation extraction tasks:
Listing 11: Prompt Format for relation extraction tasks with baselines.
Suppose you need to generate synthetic data for the biomedical [domain] task. Your task is to
give a sentence about [class_name] relation between [entity0] and [entity1] so that [label_desc].
[domain]: “Disease Gene Relation" for GAD, “Chemical Disease Relation" for CDR, and “Chemical Protein Relation" for ChemProt. [entity0] and [entity1]: “disease" and “gene" for GAD, “chemical" and “disease: for CDR, and “chemical" and “protein" for ChemProt. [class_name]: the label name for this generated sample, listed in Appendix C. [label_desc]: the description of the selected label. For example, the label “upregulator" in ChemProt has a description of “the chemical activates expression of the protein."
Natural language inference tasks:
Listing 12: Prompt Format for generating the first sentence in NLI tasks with baselines.
Suppose you need to create a set of [content]. Your task is to generate one sentence for a [content].
[content]: “health question" for MEDIQA-RQE, “claim" for MEDIQA-NLI, MedNLI and MQP, and “health news" for PUBHEALTH and HealthVer.
Listing 13: Prompt Format for generating the second sentence in NLI tasks with baselines.
Suppose you need to create a pair of sentences for the [domain]
task with the label '[class_name]'. Given the [content]: '
[first_sentence]', Your task is to generate one short [content] so that [label_desc].
[domain]: “Question Entailment" for MEDIQARQE, “Natural Language Entailment" for MEDIQA-NLI and MedNLI, “Fact Verification" for PUBHEALTH and HealthVer, and “Sentence Similarity Calculation" for MQP. [content]: “health question" for MEDIQARQE, “hypothesis" for MEDIQA-NLI, MedNLI, “evidence" for PUBHEALTH and HealthVer, and “sentence" for MQP.
[class_name]: the label name for this generated sample, listed in Appendix C. [label_desc]: the description of the selected label. For "entailment", the description is "we can infer the [content] from the given sentence". For "neutral", the description is "there is no clear relation between the [content] from the given sentence". For "contradict", the description is "we can refute the [content] from the given sentence". [first_sentence]: the first sentence we generate.

heading: F Detailed Per-task Experimental Results
text: In this section, we present additional experimental results on every dataset in Tables 7, 8, 9. We also include the experimental results combining topic from both KG and LLM, which yields a performance improvement, though not a substantial one. However, note that in practice, it is challenging to tune the ratio in the few-shot setting.
LitCovid HOC CDR GAD ChemProt
F1 F1 P R F1 P R F1 F1
PubMedBERTBase Supervised-Full (SOTA) 73.55 84.35 67.81 76.60 71.96 — — 84.39 77.97 Supervised-Full 71.70 82.32 67.81 76.60 71.96 82.55 85.10 83.81 76.24 Supervised-Few 24.08 13.13 41.62 52.96 46.61 57.71 46.54 51.53 33.54
DA-Word Sub 36.49 44.98 40.50 46.20 43.16 51.15 32.10 39.45 31.82 DA-Back Trans 39.70 54.78 — — — — — — — DA-Mixup 40.82 49.35 41.40 44.80 43.03 55.44 48.30 51.62 35.45 DA-Transformer 39.86 42.18 44.60 61.70 51.77 59.40 46.50 52.16 38.73
BC5CDR-Disease BC5CDR-Chemical NCBI-Disease CHEMDNER CASI
P R F1 P R F1 P R F1 P R F1 P R F1
PubMedBERTBase Supervised-Full (SOTA) — — 86.10 — — 93.33 — — 88.76 — — 92.35 — — — Supervised-Full 83.84 87.92 85.83 92.22 91.74 91.98 87.54 89.92 88.71 91.84 92.45 92.14 — — — Supervised-Few 24.86 39.47 30.51 63.73 46.07 53.48 36.16 39.47 37.74 48.00 28.70 35.92 38.11 43.82 40.77
DA-Word Sub 35.34 39.54 37.32 63.13 52.52 57.34 53.40 36.70 43.50 47.45 33.15 39.03 40.25 47.65 43.64 DA-Mixup 36.13 42.90 39.23 66.43 50.54 57.41 56.57 26.48 36.07 52.40 27.53 36.10 42.37 48.96 45.43 LightNER 39.80 33.20 36.20 — — — 43.70 41.90 42.78 — — — — — — DA-MELM 34.20 41.30 37.42 47.23 72.81 57.29 36.90 48.50 41.91 39.33 45.95 42.38 37.82 44.28 40.80 KGPC 50.80 51.30 51.05 — — — 52.20 52.10 52.15 — — — — — —
ZeroGen 55.60 39.10 45.91 73.20 82.85 77.73 56.25 45.98 50.60 54.34 52.93 53.63 52.80 49.53 51.11 DemoGen 63.10 48.44 54.81 76.40 81.65 78.94 57.65 49.08 53.02 54.00 53.77 53.88 58.15 56.84 57.49 ProGen 61.60 50.50 55.50 77.10 82.02 79.48 56.01 53.50 54.73 51.55 53.00 52.26 57.76 58.57 58.16 S3 58.26 55.96 57.08 77.28 80.80 79.00 56.39 49.34 52.62 48.53 57.79 52.75 56.21 63.60 59.68

heading: G Additional Ablation and Parameter Studies
text: Figure 10 and 11 show the effect of different generators and the effect of the proportion of data on two additional datasets, respectively. Overall, our method generally outperform the best baseline. One interesting finding for the NCBI-Disease dataset is that CLINGEN performs worse than the best on one variant. We hypothesize that it is because this task involves more complex input and output, potentially posing a challenge for moderatesize LLMs to follow the instructions.
Besides, as few-shot sample selection is important for the final performance, we show the performance of different 3 random seeds in Table 10 (with different seed examples/training process), and observe that our method CLINGEN generally outperforms the baselines with non-negligible margins, which indicates the robustness of CLINGEN as it does not rely on a specific subset of few-shot training examples to perform well.

heading: H Additional Quality Analysis
text: We present additional quality analysis of the synthetic dataset with t-SNE plots in Figure 12 and the regularized entity frequencies in Figure 13.

heading: I Comparison with different prompt designs
text: I.1 Model Performance
We carry out an additional analysis with two recent and representative prompt optimization techniques, namely Reframe (Mishra et al., 2022), APE (Zhou et al., 2023) and PromptAgent (Wang et al., 2024).
In our setting, Reframe incorporates several principles (e.g. using low-level patterns, itemizing instructions, etc.) to produce high-quality prompts to enhance text generation, whereas APE and PromptAgent leverage the LLM to optimize the prompts based on the target task information. We demonstrate their performance on various clinical tasks in Table 11. The results indicate that our proposed CLINGEN consistently outperforms both baselines. This performance gain is attributed to the fact that the prompts generated by these baselines do not adequately address the unique challenges for the clinical data generation, i.e. distribution shift and lack of diversity. As a result, although they tend to include some generic task-specific information for guiding LLMs to generate training data, the
performance gains brought by these advanced techniques are limited. One important avenue of future work is to design effective approach to combine these automatic prompt optimization approaches with our extracted clinical-related concepts.
I.2 Prompt Templates We provide the detailed prompt templates we use for Reframe (Mishra et al., 2022), APE (Zhou et al., 2023) and PromptAgent (Wang et al., 2024) in the followings.
Natural Language Inference tasks:
Listing 14: Prompt Format for generating sentences in NLI tasks with Reframe.
Generate a pair of sentences for the [domain] task. Follow these guidelines:

heading: 1. Formulate a medical premise in
text: the first sentence , such as a
clinical observation or a patient 's medical history.

heading: 2. Craft a medical hypothesis or claim related to the premise in the second sentence.

heading: 3. Ensure that the hypothesis logically follows from the premise.

heading: 4. Avoid introducing any unrelated or contradictory information in either sentence.

heading: 5. The length should be in 50 words.
text: Listing 15: Prompt Format for generating sentences in NLI tasks with APE.
Generate a pair of sentences for the [domain] task. The first sentence should be a medical premise , such as a clinical observation or a patient 's medical history. The second sentence should be a medical hypothesis or claim , related to the premise. The goal is to determine whether the hypothesis logically follows from the premise , and you can use various medical scenarios , conditions , or treatments for creating these
sentence pairs.
Listing 16: Prompt Format for generating sentences in NLI tasks with PromptAgent.
You 've been assigned the task of creating a dataset for determining the [domain] in medical text pairs. Ensure that you do not include any irrelevant information. Keep in mind that
the content may involve medical conditions , treatments , and observations in various formats. Your goal is to accurately label the relationships for each medical text pair based on their logical connections.
[domain]: “Question Entailment" for MEDIQA-RQE.
Sentence similarity tasks:
Listing 17: Prompt Format for generating sentences in sentence similarity tasks with Reframe.
Suppose you need to generate two sentences for the [domain] task. Your task is to give a pair of sentences with the following instructions: (1) Generate two sentences that exhibit a clear similarity or dissimilarity in meaning without using complex or specialized terms. (2) express attributes affirmatively. (3) Ensure that both sentences have a common attribute for
comparison. (4) The length should be in 50 words.
Listing 18: Prompt Format for generating sentences in sentence similarity tasks with APE.
Suppose you need to generate two sentences for the [domain] task. The goal is to assess how close or similar the meaning of two sentences is, including ' equivalent ' or 'not equivalent '.
Listing 19: Prompt Format for generating sentences in sentence similarity tasks with PromptAgent.
You 've been assigned the job of creating a dataset for [domain]. Make sure not to include any extraneous details. Keep in mind that sentences can vary in structure and wording while conveying similar meanings. Your task is to calculate the similarity score accurately for each sentence pair.
[domain]: “Sentence Similarity Calculation" for MQP.
Text classification tasks:
Listing 20: Prompt Format for generating sentences in text classification tasks with Reframe.
Suppose you are a writer for [domain]. Your task is to give a synthetic [domain] about
[class_name] with the following instructions: (1) Illustrate points with everyday scenarios related to the [class_name].
(2) about 50 - 100 words.
Listing 21: Prompt Format for generating sentences in text classification tasks with APE.
Suppose you are a writer for [domain]. Generate a clinical article discussing the latest advancements in [domain] with a focus on [class_name]. Please include information on recent clinical trials , emerging research findings , and potential implications for healthcare practitioners and patients.
Listing 22: Prompt Format for generating sentences in text classification tasks with PromptAgent.
You 've been assigned the responsibility of creating a dataset for classifying text related to [domain]. Ensure that you do not include any irrelevant information. Keep in mind that
references to COVID -19 may appear in various forms , including
abbreviations and synonyms. Your objective is to accurately identify and classify text that is relevant to [domain].
[domain]: “COVID-19 Literature" for LitCovid. [class_name]: the label name for this generated sample. Relation extraction tasks:
Listing 23: Prompt Format for generating sentences in relation extraction tasks with Reframe.
Suppose you need to generate a dataset for the biomedical [domain] task where the relationships between entities in biomedical texts need to be
identified. Your task is to give a synthetic example about [class_name] relation with the following instructions:
(1) Provide the sentence or text snippet where the relationship is mentioned.
(2) The length should be in 50 words.
Listing 24: Prompt Format for relation extraction tasks with APE.
Generate a sentence that describes a [class_name] [domain] between [entity0] and [entity1]. The sentence should provide
information about how these terms are related , such as its
potential therapeutic use , side effects , or any relevant research findings.
Listing 25: Prompt Format for relation extraction tasks with PromptAgent.
You 've been assigned the task of creating a [class_name] [domain] dataset for identifying relationships between [entity0] and [entity1] from the provided text. Be sure to exclude any extraneous information. Keep in mind that chemicals and diseases may be referred to using various names , abbreviations , or synonyms . Your goal is to recognize and extract these associations accurately.
[domain]: “Chemical Disease Relation" for CDR. [entity0] and [entity1]: “chemical" and “disease: for CDR. [class_name]: the label name for this generated sample. Named entity recognition tasks:
Listing 26: Prompt Format for generating sentences in NER tasks with Reframe.
Suppose you need to create a dataset for [domain] recognition. Your task is to generate a sentence about [domain] and also output the [domain] name with the following instructions: (1) Generate a sentence that contains a named entity. The named entity should be a
recognizable entity type within the sentence. (2) The named entity must be contextually relevant and correctly labeled with its type. (3) The length should be in 50 words.
Listing 27: Prompt Format for NER tasks with APE.
Suppose you need to create a dataset for [domain] recognition. Generate a sentence or short text passage where you mention a
[domain] entity within a context. The named entity should be clearly identifiable within the text.
Listing 28: Prompt Format for NER tasks with PromptAgent.
You 're tasked with generating a dataset for recognizing [domain] from the given sentence. Remember to avoid incorporating any
associated elements. Consider both specific diseases and broader categories , and remember diseases and conditions can also appear as common abbreviations or variations.
[domain]: “disease" for BC5CDR-Disease; “chemical" for CHEMDNER.

heading: J Using Medical LLMs as Data Generator
text: In this work, we mainly evaluate CLINGEN using GPT-family models as the LLM. However, we are aware that many LLMs have been fine-tuned on additional clinical contexts as well as instructions and achieved superior performance on clinical NLP benchmarks. We select MedAlpaca-13b (Han et al., 2023) as one representative clinical LLM and study the effect of CLINGEN using a medical LLM as the data generator. Many other medical LLMs, such as Med-PALM8, are not open-sourced, thus we cannot run them in our experiments.
From the results shown in Table 12, we observe that using medical LLM as the clinical text data
8https://sites.research.google/med-palm/
generator exhibits lower downstream performance. This could be attributed to the medical LLMs having fewer parameters than ChatGPT, which results in limited instruction-following capabilities.

heading: K Effect of Data Mixing Ratio
text: In this work, we present KGs and LLMs as two alternative and complementary sources for obtaining topics. However, we also consider combining topics from KGs and LLMs as a potential approach to enhance performance. Thus, we conduct experiments to demonstrate the impact of combining topics from KGs and LLMs at various ratios. Note that we still keep a total of 5000 generated synthetic samples to maintain a fair comparison. The experimental results in Table 13 indicate that combining knowledge from KGs and LLMs can yield a performance improvement, though not a substantial one. However, note that in practice, it is challenging to tune the ratio in the few-shot setting due to the limited volume of validation labels (Perez et al., 2021), and thus we only include the 1:1 results in Tables 7, 8, 9 in Appendix F for all the datasets.
KG : LLM LitCovid CDR MEDIQA-RQE BC5CDR-Disease Average

heading: Future Work
text: One important avenue of future work is to design effective approach to combine these automatic prompt optimization approaches with our extracted clinical-related concepts. I. 2 Prompt Templates We provide the detailed prompt templates we use for Reframe (Mishra et al. , 2022), APE (Zhou et al. , 2023) and PromptAgent (Wang et al. , 2024) in the followings. Natural Language Inference tasks:
Listing 14: Prompt Format for generating sentences in NLI tasks with Reframe. Generate a pair of sentences for the [domain] task. Follow these guidelines: