Paper: Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models (ICLR-2024)

Reviewer comment (Evaluation):
The paper mentioned privacy as an issue but I think that the method will not address any privacy issues that are accruing in the original LLM.

Author response:
A: Thanks for the insightful feedback. We have answered this question in the general response for all reviewers. Please refer to the **Patient privacy concerns** section for details.

Reviewer comment (Experiments):
Baselines descriptions could be more elaborated.

Author response:
A: Thank you for the suggestion. We provide the baseline details in **Appendix E** in our original manuscript, and we have expanded these descriptions to provide more comprehensive information in the revision.

Reviewer comment (Reproducibility):
The most important question is what is the additional pre trained classifier.

Author response:
A: As mentioned in W2, we utilize the same pre-trained PubMedBERT as a classifier for our method and baselines on downstream tasks to prevent unfair comparison. Please see “Sec 5.1 Experiment Setup” for reference. The word “additional” is to differentiate the classifier with the previous LLM generator. We do not conduct any additional pre-training. We have deleted the word to avoid confusions.

Reviewer comment (Writing):
It's unclear from the paper.

Author response:
A: As mentioned in W2, we utilize the same pre-trained PubMedBERT as a classifier for our method and baselines on downstream tasks to prevent unfair comparison. Please see “Sec 5.1 Experiment Setup” for reference. The word “additional” is to differentiate the classifier with the previous LLM generator. We do not conduct any additional pre-training. We have deleted the word to avoid confusions.

Reviewer comment (Experiments):
It is unclear how the prompts in Appendix F are determined. Also, the work does not experiment with different prompt designs.

Author response:
A: Thanks for the feedback. We notice that the majority of existing research on synthetic data generation relies on manually crafted, straightforward prompts without intricate designs [1,2,3]. Thus, we design our prompt based on the structures of those existing works and task-specific information mentioned in [4], with additional instructions to further incorporate the selected clinical topics and writing styles. For other existing prompt optimization methods in the literature, we identify two related techniques that may be potentially applicable to our case. Please refer to the **Comparison with different prompt designs** section of the general response to all reviewers for the experimental results and further discussions.

>[1] Ye et al. "Zerogen: Efficient zero-shot learning via dataset generation." EMNLP 2022.
>
>[2] Meng et al. "Tuning language models as training data generators for augmentation-enhanced few-shot learning." ICML 2023.
>
>[3] Ye et al. "ProGen: Progressive zero-shot dataset generation via in-context feedback." EMNLP 2022.
>
>[4] Fries et al. "Bigbio: a framework for data-centric biomedical natural language processing." NeurIPS 2022.

Reviewer comment (Novelty):
One useful and missing reference: Ive J, Viani N, Kam J, Yin L, Verma S, Puntis S, Cardinal RN, Roberts A, Stewart R, Velupillai S. Generation and evaluation of artificial mental health records for natural language processing. NPJ digital medicine. 2020 May 14;3(1):69.

Author response:
A: Thanks for mentioning this relevant paper. We have modified our manuscript and added the reference in **Section 1** and **Appendix A**.

Reviewer comment (Experiments):
The clinical topics/concepts in section 4.1.1 are picked from KG and LLMs. How many topics are from KGs and how many are from LLMs? How is this ratio determined? Would a different ratio impact results? Why not ask a clinician to hand-pick clinical topics?

Author response:
A: Thank you for bringing up this question. In our experiments, we keep the topics from KGs and LLMs **separate**. Specifically, we create a candidate set of approximately 6000 topics from KGs and another candidate set of around 2000 topics from LLMs. We then independently sample topics from these two candidate sets to generate two distinct sets of synthetic training samples. Please refer to **Table 1** for the experimental results for both sets of synthetic data.

The motivation behind this is that, in our paper, we present KGs and LLMs as two alternative and complementary sources for obtaining topics for readers to understand the influence of topic injection from KG and LLMs distinctly. However, we agree with the reviewer that combining topics from KGs and LLMs has the potential to enhance performance. Thus, we have conducted additional experiments to demonstrate the impact of combining topics from KGs and LLMs at various ratios. Note that we still keep a total of 5000 generated synthetic samples to maintain a fair comparison.

| KG : LLM| LitCovid | CDR | MEDIQA-RQE | BC5CDR-D | Average |
|:--:|:--:|:--:|:--:|:--:|:--:|
| PubMedBERT-Base | F1 | F1 | ACC | F1 ||
| 1:0 | 58.01 | 61.75 | 74.85 | 60.75 | 63.84 |
| 2:1 | 56.18 | 62.89 | 73.50 | 60.53 | 63.28 |
| 1:1 | 56.76 | 63.86 | 74.01 | 63.26 | 64.47 |
| 1:2 | 55.49 | 64.33 | 75.10 | 61.62 | 64.14 |
| 0:1 | 59.22 | 63.34 | 72.40 | 61.03 | 64.00 |
| PubMedBERT-Large ||||||||
| 1:0 | 55.81 | 62.66 | 79.92 | 61.21 | 64.90 |
| 2:1 | 54.21 | 64.22 | 76.15 | 62.40 | 64.25 |
| 1:1 | 56.80 | 65.90 | 79.12 | 65.94 | 66.94 |
| 1:2 | 54.41 | 64.68 | 80.77 | 64.55 | 66.10 |
| 0:1 | 57.07 | 64.99 | 77.36 | 63.15 | 65.64 |

The experimental results indicate that combining knowledge from KGs and LLMs can yield a performance improvement, though not a substantial one. However, note that in practice, it is challenging to tune the ratio in the few-shot setting due to the limited volume of validation labels [5], and thus we only include the 1:1 results in **Table 6,7,8** in **Appendix G** for all 16 datasets.

> [5] Perez et al. "True few-shot learning with language models." NeurIPS 2021.

***
> Q4 (continued): Why not ask a clinician to hand-pick clinical topics?

A: Thank you for the suggestion. While we admit clinicians' ability to select clinical topics, it is important to point out that such a manual approach involves significant time and money, thus is less scalable. Furthermore, it can be challenging for clinicians to manually create thousands of clinical topics for synthetic training samples. To this end, we utilize KGs and LLMs as more efficient methods to automatically extract clinical topics.

Reviewer comment (Experiments):
The current SOTA results (regardless of fully supervised or few-shot) should be added to Table 1 as reference points.

Author response:
A: Thanks for the valuable suggestion. In our study, we present the supervised model performance on few-shot demonstrations and the complete original training set, denoted as "Supervised - Few" and "Supervised - Full," respectively. We use PubMedBERT, the same classifier employed for ClinGen and all baseline models, as a reference point. 

Following the reviewer's request, we have tried our best to gather relevant data on SOTA models and their performances. However, this task is nontrivial due to several reasons: (1) Many studies only test their models on a subset of the datasets we used; (2) Original papers for some datasets might report results only for base models (like BERT) and not include more recent models (such as PubMedBERT or LinkBERT); (3) In some cases, datasets report only F1 scores without precision and recall metrics.

Despite our best efforts to compile this information, there are still missing values in the available SOTA performance data. We have documented these findings in Tables 6, 7, and 8 in Appendix G, with references from [5-10]. Note that many SOTA performances are based on different classifiers, so these results should only serve as reference points and cannot be directly comparable with the performance of ClinGen and other baselines.


> [6] Peng et al. "Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets." BioNLP 2019.
>
> [7] Tinn et al. "Fine-tuning large neural language models for biomedical natural language processing." Patterns 4.4 (2023).
>
> [8] Gu et al. "Domain-specific language model pretraining for biomedical natural language processing." Transactions on Computing for Healthcare 2021.
>
> [9] Yasunaga et al. "LinkBERT: Pretraining Language Models with Document Links." ACL 2022.
> 
> [10] BLURB Leaderboard (https://microsoft.github.io/BLURB/leaderboard.html)

Reviewer comment (Experiments):
D_train in Equation 1 has only K (K<=5) samples per label?

Author response:
A: $D_{train}$ has only K samples per label, where we set K=5 in our experiments. We intentionally set K to a very small value to simulate a few-shot learning scenario. Recent work on language model fine-tuning suggests that it can help model to learn a good initialization before fine-tuning on synthetic samples [2].

>[2] Meng et al. "Tuning language models as training data generators for augmentation-enhanced few-shot learning." ICML 2023.

Reviewer comment (Novelty):
The proposed prompting strategy is intuitive but lacking in novelty. Leveraging both KGs and LLMs is expected to help.

Author response:
A: Thanks for the insightful feedback. We have answered this question in the general response for all reviewers. Please refer to the **Lack of novelty** section for details.

***
A: We would like to clarify that we *do not* use “more data” compared to the baselines. In our experiments, all the baseline models are evaluated based on their respective generated synthetic datasets, and we maintain the **same amount of synthetic training data** across all methods, as mentioned in Appendix C. This ensures a fair comparison among all the methods evaluated in our study. In the revised version, we explicitly mention this in **Section 5.1** to avoid confusion.

Moreover, it's essential to highlight that merely increasing the volume of training data **does not guarantee** better results. As shown in Table 1, some standard data augmentation methods cannot even outperform the supervised setting with few-shot examples only. Besides, for LLM-based data generation approaches, Figure 11(a) shows that the best baseline (DemoGen) using 100% of the training data actually shows lower performance compared to using only 50% of the training data. These observations illustrate the importance of the quality of training data over its quantity. Our focus, therefore, has been on developing a methodology to create high-quality synthetic training data. The results from our experiments in Table 1, Figure 5 and Figure 11 confirm that we consistently outperform baselines with the same amount of generated data under different budgets. 

***
A: Thanks for the suggestion. We would like to emphasize that our primary goal is indeed to improve a model’s medical capacity during fine-tuning. Our approach focuses on enriching the training dataset with diverse medical knowledge, which can empower models of varying sizes with improved medical capacity. The generated synthetic data can also be easily and directly applied to improve various existing models without additional design. 

Besides, we believe that our approach (e.g., improving the data quality for fine-tuning task-specific models) is **orthogonal** to those efforts that improve the small models’ medical capacity in the pretraining stage. As shown in Table 1, the average performance of ClinGen with *PubMedBERT-base* (110M Parameters) as the backbone is *better than* the strongest baseline using *PubMedBERT-large* (340M Parameters) on the majority of tasks (e.g. Text Class, Relation Extraction, NLI, Fact Verification, and Medication Attribute Extraction), indicating that better data quality for fine-tuning is also crucial for adapting small model on specific tasks.

Moreover, it is important to note that our strategy of incorporating more diverse training data alleviates rather than exacerbates the overfitting issue. By incorporating diverse training data enriched with medical knowledge, we enhance the ability of models to generalize across various clinical scenarios.

Reviewer comment (Experiments):
To the best of my knowledge, the generated data is still in-domain data, generating the style the same as the training data. So, it is not surprising for me that using more data for training can improve model performance. It makes more sense to improve a small model's overall capacity, instead of overfitting on a specific task with more training data.

Author response:
A: We would like to clarify that we *do not* use “more data” compared to the baselines. In our experiments, all the baseline models are evaluated based on their respective generated synthetic datasets, and we maintain the **same amount of synthetic training data** across all methods, as mentioned in Appendix C. This ensures a fair comparison among all the methods evaluated in our study. In the revised version, we explicitly mention this in **Section 5.1** to avoid confusion.

Moreover, it's essential to highlight that merely increasing the volume of training data **does not guarantee** better results. As shown in Table 1, some standard data augmentation methods cannot even outperform the supervised setting with few-shot examples only. Besides, for LLM-based data generation approaches, Figure 11(a) shows that the best baseline (DemoGen) using 100% of the training data actually shows lower performance compared to using only 50% of the training data. These observations illustrate the importance of the quality of training data over its quantity. Our focus, therefore, has been on developing a methodology to create high-quality synthetic training data. The results from our experiments in Table 1, Figure 5 and Figure 11 confirm that we consistently outperform baselines with the same amount of generated data under different budgets.

Reviewer comment (Experiments):
No rigorous comparison with other prompting optimization methods in the literature.

Author response:
A: Thanks for the constructive feedback. We would like to highlight that the majority of existing research on synthetic data generation relies on manually crafted, straightforward prompts without dedicated designs [4,5,6]. That being said, we have tried our best to identify two closely related prompt optimization techniques that are potentially applicable to our case. Please see the **Comparison with different prompt designs** section of the general response to all reviewers for the experimental results and further discussions.


> [4] Ye et al. "Zerogen: Efficient zero-shot learning via dataset generation." EMNLP 2022.
> 
> [5] Meng et al. "Tuning language models as training data generators for augmentation-enhanced few-shot learning." ICML 2023.
> 
> [6] Ye et al. "ProGen: Progressive zero-shot dataset generation via in-context feedback." EMNLP 2022.

Reviewer comment (Experiments):
I am not sure why the two stages training is conducted. Was the result you mixed training data with generated data together for training?

Author response:
A: Thank you for the question. In Stage I, we perform fine-tuning of the classifier using few-shot examples in the original training set, specifically K samples per label (in our experiments, K=5). Subsequently, in Stage II, we conduct further fine-tuning of the classifier using the synthetic training data that we generate. We employ this two-stage training because it has been empirically demonstrated to be stable and is commonly employed in various works, including synthetic data generation tasks [1] and other semi-supervised learning methods [2,3]. Note that we mix the few-shot training data with the generated synthetic data and apply this two-stage training **consistently** for all the baselines to ensure a fair comparison.

>[1] Meng et al. "Tuning language models as training data generators for augmentation-enhanced few-shot learning." ICML 2023
>
>[2] Laine and Aila. "Temporal ensembling for semi-supervised learning." ICLR 2017
>
>[3] Chen et al. "Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification." ACL 2020

Reviewer comment (Novelty):
The reviewer likes that the paper did a direct comparison with the prompts from GPT-3.5. However, I'm interested to know if the current medical LLMs are proficient in the tasks described. Should they already excel in these areas, it might somewhat diminish the necessity for the proposed approach.

Author response:
Thank you for the suggestion. Per your request, we have included additional experimental results that compare the direct inference performance of two recent and popular medical Large Language Models (LLMs): **PMC-LLaMa** [1] and **MedAlpaca** [2]. The table below illustrates that these medical LLMs perform less effectively than both ClinGen and GPT-3.5. This result can likely be attributed to two reasons:
- The smaller size of the medical LLMs (7B) compared to GPT-3.5, which results in limited language modeling and reasoning capabilities. While we are aware that larger medical LLMs like Med-PaLM exist, they are not publicly accessible and thus cannot be used in our experiments. 
- Furthermore, these medical LLMs are typically trained on general medical knowledge sources, such as Wikipedia and Stack Exchange, as indicated by https://github.com/kbressem/medAlpaca#data-overview. This general knowledge may not translate effectively to specific tasks like chemical-protein interaction prediction or health fact verification, leading to suboptimal performance in few-shot learning scenarios. 

This observation aligns with findings from other research [3], which suggest that for certain clinical Natural Language Processing tasks, the fine-tuning of more compact models may outperform the in-context learning capabilities of larger LLMs. The results and discussions have been updated to **Table 2** in **Section 5.3**.

| | HOC | | GAD | | ChemProt | MEDIQA-RQE | PUBHEALTH | | | NCBI-Disease | | | CASI | |
|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
| | F1 | P | R | F1 | F1 | ACC | ACC | F1 | P | R | F1 | P | R | F1 |
| GPT-3.5 Inference | 68.76 | 84.21 | **97.46** | 90.35 | 49.42 | 74.31 | **69.50** | **52.47** | 46.62 | 52.31 | 49.30 | 48.82 | **74.75** | 59.07 |
| PMC-LLaMa Inference | 32.94 | 90.14 | 90.59 | 90.37 | 13.35 | 52.17 | 14.53 | 2.94 | 61.87 | 37.81 | 46.79 | 59.89 | 37.94 | 45.45 |
| MedAlpaca Inference | 36.44 | 69.95 | 70.29 | 70.12 | 26.29 | 57.67 | 56.51 | 35.71 | 44.69 | 31.16 | 27.85 | 52.51 | 49.16 | 51.64 |
| CLinGen w/ KG | 77.71 | 94.30 | 89.09 | **91.62** | 60.12 | **79.92** | 50.20 | 41.26 | **62.46** | **64.08** | **63.26** | 70.96 | 69.66 | **70.30** |
| CLinGen w/ LLM | **78.14** | **95.08** | 86.14 | 90.39 | **63.05** | 77.36 | 52.96 | 43.31 | 61.12 | 60.16 | 60.64 | **71.61** | 66.86 | 69.15 |


>[1] Wu et al. "Pmc-llama: Further finetuning llama on medical papers." arXiv preprint arXiv:2304.14454 (2023).
>
>[2] Han et al. "MedAlpaca--An Open-Source Collection of Medical Conversational AI Models and Training Data." arXiv preprint arXiv:2304.08247 (2023).
>
>[3] Lehman et al. "Do We Still Need Clinical Language Models?." CHIL 2023.