Enhancing Small Medical Learners with Privacy-preserving Contextual Prompting
Authors: Xinlu Zhang
Shiyang Li
Xianjun Yang
Chenxin Tian
Yao Qin
Linda Ruth Petzold
Venue: ICLR 2024 poster (2024)
Keywords: natural language processing
large language model prompting
application in healthcare
privacy
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Large language models (LLMs) demonstrate remarkable medical expertise, but data privacy concerns impede their direct use in healthcare environments. Although offering improved data privacy protection, domain-specific small language models (SLMs) often underperform LLMs, emphasizing the need for methods that reduce this performance gap while alleviating privacy concerns. In this paper, we present a simple yet effective method that harnesses LLMs' medical proficiency to boost SLM performance in medical tasks under $privacy-restricted$ scenarios. Specifically, we mitigate patient privacy issues by extracting keywords from medical data and prompting the LLM to generate a medical knowledge-intensive context by simulating clinicians' thought processes. This context serves as additional input for SLMs, augmenting their decision-making capabilities. Our method significantly enhances performance in both few-shot and full training settings across three medical knowledge-intensive tasks, achieving up to a 22.57% increase in absolute accuracy compared to SLM fine-tuning without context, and sets new state-of-the-art results in two medical tasks within privacy-restricted scenarios. Further out-of-domain testing and experiments in two general domain datasets showcase its generalizability and broad applicability.

Published as a conference paper at ICLR 2024
ENHANCING SMALL MEDICAL LEARNERS WITH PRIVACY-PRESERVING CONTEXTUAL PROMPTING
Xinlu Zhang's :Shiynng Li', Xianjun Yang', Chenxin Tian', Yao Qin ,inda Ruth Petzold
University of California, Santa Barbara
"Chnese Academy of Medical Sciences and Peking Union Medical College
ABSTRACT
Large language models (LLMs) demonstrate remarkable medical expertise, but data privacy concerns impede their direct use in healthcare environments. Although offering improved data privacy protection, domain-specific small language models (SLMs) often underperform LLMs, emphasizing the need for methods that reduce this performance gap while alleviating privacy concerns. In this paper, we present a simple yet effective method that harnesses LLMs' medical proficiency to boost SLM performance in medical tasks under privacy-restricted scenarios. Specifically, we mitigate patient privacy issues by extracting keywords from medical data and prompting the L.LM to generate a medical knowledge-intensin context by simulating clinicians' thought processes. This context serves additional input for SLMs, augmenting their decision-making capabilities. Our method significantly enhances performance in both few-shot and full training settings across three medical knowledge-intenee intensive tasks, achieving up to a 22.57% increase in absolute accuracy compared to SLM fine-tuning without context, and sets new state-of- the-art results in two medical tasks within privacy-restrieted scenarios. Further out-of-domain testing and experiments two general domain datasets showcase its generalizability and broad applicability. Our code can be found at https: SLM
/ //github. www.cc////...oo httmm//hhaag77666111eeaa
Figure 1: Synthetic medical data for illustration. Though rich in domain-specific knowledge, medi- cal data contains sensitive private information. We extract keywords to mitigate privacy concerns.
Steven Smith 60-vear-old man admitted at Auckland Hospital. 06/01/2008. He has a past medical history significant uncontrolled HTN who presents inguinal hernia. Patient first noticed a right sided bulge in months prior. Every day it slips out and he has to manually push back it. He has had to present to the emergency room twice recently when he was unable to push it back it. He was pending an outpatient repair of his right inguinal hernia.
attended Dr. Edward Jones Date:
non-reducible right
What are the assessment and recommendations for this patient?
INTRODUCTION
Large language models (LLMs) (Brown et al. 2020 Chowdhery et al. 2022; Ouyang et al. 2022 OpenAl 2022 2023) have shown promise in the medical field (Singhal et al. 2022 Nori et al. 2023
Liévin et al. 2023). However, concerns about medical data privacy prevent the direct use of LLMS
medical capabilities in healthcare domain, as illustrated in Figure | Despite data usage policies to safeguard user data, the implementation of these policies varies among L.I.M,, creating inconsistent protection levels. Moreover, medical data usage agreements" are stringent, explicitly forbidding data sharing with third parties, like direct uploads to ChatGPT (OpenAT] 2022). Therefore, developing methods to harness LLMs medical knowledge while balancing data privacy in privacy-restricted scenarios is an urgent and underexplored research area.
are
"Corresponding Author: xinluzhang@ucsh..cm
https://opeaii.cco//// http://ppw....ooo/oo/o////////as http:://wwyymnaee.coooommemmmmmn

---

Published as a conference paper at ICLR 2024
Figure 2: Framework overview. (a) To mitigate privacy leakage, we use keyword extractor to obtain medical keywords. Clinicians then create several contexts based on these keywords and candidate answers, which the LLM uses to produce privacy-restrice contexts. (b) The generated contexts are used as additional input to enhance SLM medical decision-making capacity.
(1) Keywords Extraction
(2) Context Generation
Candidate
Candidate
Question
Keywords
Question
Context
answers
keywords
Extractor
Keywords
Context
Prediction
(a) Privacy-Restriceed Context Prompting in LLMs
(b) Eontext-Enhanced Medical Capability in SLMs
Small language models (SLMs) that are specific to the medical domain Guu et al.
2021 Yasunaga
al. 2022a Lee et al. 2020 Alsentzer et al. 2019; Bolton et al. 2022 Lehman et (023) have
shown superior in-domain task performance compared to general- SLMs (Radford et al. 2019
Raffel et al. 2020 Devlin al. 2019). addressing the vital need for data privacy in the medical field through local training. However, a notable performance gap between SLMs and LLMs in medical tasks remains (Singhal et al. [2022] Liévin et al. 2023). A critical question arises: How can we bridge the performance gap between SLMs and LLMs for medical tasks in privacy-restricet scenarios?
One common strategy to narrow the performance gap between L.LMs and SLMSs involves leveraging generated rationales (Wei et al.) 2023 Wiegreffe et al. (021)) from LLMs to boost the performance of SLMs (Li et al. 2022; Fu et al. 2023] Ho et al. 2022 Shridhar et al. 2022). However, previous research has often required feeding complete data information into LLMs, ignoring privacy concerns. Thus, it is essential to explore alternative methods that can effectively utilize LLM -...-eeerttted context,
which is rich in medical knowledge, while balancing privacy concerns for LLMs and performance enhancement for SLMs in the medical domain.
In this paper, we present simple yet effective pipeline that boosts SLM performance by incorporating medical contexts from L.MSs in privacy-restrieted scenarios. To the best of our knowledge, this is the first work that has utilized LLMs to improve SLM performance in such settings. While our primary focus is on multiple-choice medical QA, our framework can be adapted to other tasks or domains. Figure 2 iillustrates our framework. Specifically, we use existing named-entip -enttyy recognition (NER) models (Nuumann et al. 2019) to extract keyword, FT thereby mitigating privacy risks. Based on these keywords and candidate answers, clinicians generate several medical contexts that mirror their thought processes. These clinician wrrtten contexts serve as demonetisation to generate contexts for the remaining data by the LLM by in-context learning (Brown et al. 2020). Finally, we integrate these contexts into the SLMs to improve their performance medical tasks.
Overall, our main contributions can be summarized as follows:
1. We propose a simple yet effective pipeline that uses keywords and candidate answers to elicit medical knowledge from LLMs, which is then fed into SLMs to enhance medical capabilities.
2. We introduce a crivacy-conscione prompting strategy that mimics clinicians' thinking to generate medical wowledge-intensive contexts from LLMs in
scenarios.
3. Our method notably surpasses standard SLM fine-tuning without context in both full training and few-shot settings, achieving up to 22.57% increase in accuracy, and obtains SOTA results in two medical tasks in privacy-restricted scenarios.
2 RELATED WORK
LLMs in Biomedicine. LLMs excel in NLP tasks, including medical fields (Brown et al.
2020
Chowdhery et al.] 2022; Ouyang et al. 2022
OpenAl 2022 2023 Liévin et al., 2023] Kung ct al.
2023 Singhal et al.) 2022). Kung et al. 2023 used ChatGPT for the US Medical Exam. Liévin et al.
(2023 leverage GP GP3-3.5
models for medical reasoning tasks. MedPaLM, tuned from Flan
FFollowigg Li ct al. 2022). we argue that the definition of small and large models is context context-dependent. Keywords he extracted by other methods, e.g., a manually created dictionary based domain expertise.
2

---

Published as a conference paper at ICLR 2024
(Chung et al. 2022), answered consumer medical questions comparably to clinicians (Singhal et al. 2022).
However, privacy concerns limit LLMs in real-world medical uses, highlighting the need to utilize LLM medical knowledge in privacy-restricter setting.
Biomedical SLMs.
Domain-specitfe SLMs, either extended from general- pretraining 2020) built from scratch biomed- 2021 Yasunaga et al. 2022a: Bolton et al. 2022). surpass general models
(Alsentzer et al.
2019
Lee et al.; 2020 Gururangan et al.
ical corpora (Gu et al.
(Radford al. 2019
Raffel al. |020] Devlin al. 2019) in biomedical tasks (Gu al. 2021
Yasunaga et al.
2022a Lee et al., 2020; Alsentzer et al. 2019 Bolton et al. 2022] Lehman et al.
2023). They also enhance privacy through local training. However, a performance gap remains
between fomainn-specif SLMs and LLMs in medical tasks (Liévin et al. 2023 Singhal et al. 2022)
emphasizing the need for strategies to reduce this gap.
Knowledge Distillation from LLMs. Previous studies have investigated distilling knowledge
(Hinton et al., 2015) from LLMS to enhance smaller models' performance (Ho et al. al.] [202]] [Shridhar et al. [022]] Wang et al., 2022; Li et al.) 2022). Ho et al. (2022) fine-tuned smaller models using InstructGPT-generatle reasoning samples. Wang et al. (2022) input LLM generated rationales into
SLM for question-answering Li et al. 2022) applied LLM--erived explanations for multi-task learning. Zhang et al. (2023) tunes models for following medical instructions on data generated by
GPT-4 and ChatGPT better align with user intents across diverse medical applications. However, these approaches involve prompting LLMs without considering privacy preservation, which is vital in real-world medical scenarios (Bardhan et al. 2022 Shivade et al., 2019; Miura et al.) 2020). We introduce keyword- prompting strategy to generate medical knowledge from LLMs, while balancing LLM privacy preservation and SLM performance enhancement.
Augmenting NLP Tasks with External Knowledge. Improving knowledge-intensive NLP tasks can be achieved by retrieving information from large evidence corpora like Wikipedia (Yu et al. 2023 Izacard and Grave 22200] Yasunaga et al. 2202b cj Guu et al. [020] Lee et al., [20 2019a] Lewis ct al. 2020). The retrieve-then then-read model (Chen et al. 2017) utilizes retrievers, such as BM25 (Robertson et al., 1994) and DPR (Karpukhin et al.] 2020), to identify relevant documents within a corpus. Then, a reader, like FiD (Izacard and Grave 2020)) analyzes these documents to enhance NLP tasks (Lewis al. 2020: Qu et al. | |achaan et al. 2021 Lee et al. 2019b). Some studies retrieve subgraphs from knowledge graphs to boost question-answering tasks (Yasunaga et al., 2022b Zhang et al.)
2022p. Recent research shows that pre-trained language models can "retrieve" information via direct
text generation (Petroni et al.] 2019; Roberts et al.) 2020). Liu et al. (202T): Yu et al. (003)) utilized
L.MSs to generate relevant contexts background documents for question-answering tasks. Our
work leverages LLMS as a knowledge base for medical knowledge retrieval.
Data Privacy in BioNLP. Biomedical data inherently contains sensitive information (Sousa and
Kern 2023). De-identification methods eliminate private details and replace them with synthetic data
(Act
1996 Chevrier al., 2019: Leevy al. 2020). For example. [Liu al. (2017); Dernoncourt
et al. (2017) treat de-identiiiiation as a NER problem, modeling by neural networks. Despite being de-identified, sharing restrictions still apply (Johnson al., 2016 2020). Differential privacy (DP) provides theoretical bounds on individual data privacy while allowing aggregate statistical disclosure for the entire database (Dwork 2008, Fernandes et al. 2019: Abadi et al., (016). Melamudd and Shivade (2019) train model using a DP-based approach to generate synthetic clinical notes for secure sharing. In this work, we migrate privacy concerns during LLM inference by prompting the
LLM with medical keywords extracted from raw data.
METHODOLOGY
3.1 PRIVACY-RESTRITTED CONTEXT PROMPTING IN LLMS
We consider dataset D (q: Ar, vr) where N denotes the total number of instances, qi a problem, A, a set of candidate answers, and y; the correct answer. A set of keywords k, is extracted from each problem di using a medical NER model (Neumann et al. 2019). To mitigate privacy leakage, we feed k; and A, into the LLM to generate medical context (Ho et al. 2022 Li et al. 2022).
We have a small set of denoting medical contexts, and d.. representing preliminary decisions, are generated based on partial data information ky and A², with M << S The medical context CP consists of an overall
instances E A?,
AA,CC' CP, {)) where both C..
'e set M 5 for experiments.
3

---

Published as a conference paper at ICLR 2024
Figure 3: LLM generates privacy-restricted medical contexts enhance SLM decision-making, (a) The LLM generates medical knowledge-intensine context for each instance using clinicians' few-shot demonstrations, extracted keywords from raw data (k), and candidate answers (A). The generation output comprises: overall context (co), specific context of each candidate answer (Ca, and preliminary decision of LLM (7), (b) The overall and specific contexts are then concatenated (+)
with the question as additional input to fine-tune SLM, enhancing its medical decision-making
Input
Input
examples
qiA previously healthy 32 year-old woman wwich the following neurotransmitters?
Question Keywords:
healthy, Pharmacotherapy, rouratrancnittoos
dj Dopamine
Candidate Answers:
Dopamine (b) Glutamate Norepinephrine Serotonin
The exhibiting symptoms of major...
Context:
Dopamine
LLM
The woman's is exhibiting symptoms of major depressive.. neurotranumitter it not a treatment for the patient's symptoms.
context
(G)
positive...
Specific
Giutamate
Linear-Soltmax
context
[c]: Norepinephrriee is...
Kay1
Serotonin is...
Preliminary
Therefore, the answer (d).
decision
Output
Output
(a)
(b)
context and specific contexts for each candidate answer. The overall context encapsulates high-level medical knowledge based keywords and candidate answers, while specific contexts provide detailed information for each candidate answer and its relationship to the overall context. The preliminary decision represents clinicians' predictions informed by these contexts. This prompt strategy simulates clinicians' reasoning steps by producing high-level guidance based partial data, examining individual candidate answers in-depth, and ultimately making an initial determination given prior analysis, as shown in Figure (a)
We utilize the LLM with E as demonstrations for in-context learning to generate medical contexts and preliminary decisions for all instances in D. For 1 S i S N, concatenate all instances in E, ki, and A; and then feed this concatenated string into the LLM for decoding. After this, we parse the decoded sentence into two parts: the context part C, which includes overall and specific contexts, and the preliminary decision part d which serves as a performance metric for the LLM. All instances of C; are used for SI.M training, regardless of the LLM'' preliminary decision accuracy. We find that these contexts contain valuable medical knowledge even with incorrect decisions. We defer more details of our experiments into Section 5.1
3.2
MEDICAL CAPABILITY IN SLMS
Given augmented dataset D' (9g A;, C,, v.))., our goal is leverage the medical context C₁ generated by the LLM to enhance the SLM's medical proficiency and predict yi for each instance. We omit for simplicity. Inspired by previous works (Wang et al. 2022), we treat C as additional input for the SLM to aid the decision-making, For each candidate answer aj E A, 0 < j < [A], we provide both the overall context, Co, and the specific context the answer, Cay concatenating these contexts with the question q and answer aj. The SLM generates a contextual representation
vector h for each choice, which then fed into linear layer to produce 8j. prediction score for the correctness of the answer choice:
hj SLM((g @ aj @ c. @ ca,l), 8, Linear(h;) (h;).
For each dj. the score sj is computed and normalized using the softmax function across all candidate answers, shown in as shown in Figure (b). During training, models optimized to maximize correct answer scores employing standard cross-entropy loss between predictions and ground truths.
'See Appendix A.7 for prompt details.
We only use to evaluate L.MM performance and add this into SLM training.
4

---

Published as a conference paper at ICLR 2024
In the inference phase, sy is calculated for each aj, and the answer with the highest score is the predicted answer.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets. We evaluate methods on the first three datasets for in-domain performance and on all four datasets for out-of-domain performance: 1. MedQA (Jin et al. 2020) contains 4-way multiple- choice questions from the US Medical Licensing Exam. It has 10, 78/1,
instances in the teining/develoomentttss sets. Results on the development and test sets are reported. 2. HEADQA (Vilares and (66mez-Rodriguez) 2019) features multiple-choice questions from specialized Spanish instances in
healthcare exams conducted between 2013 and 2017. The dataset has (667/1,366/2,77 the training/developmenn/esss sets. We report results on the development and test sets. 3. MedMCQA (Pal et al. 2022) is 4-way multiple-choice dataset from Indian medical school entrance exams. It instances in the waining/developmenn/ess sets. We use randomly selected subset of 10,000 training instances and report results on the development set, following previous work (Nori et al. 2023). MMLU-prooessional medicine (Hendrycks et al., 202T) is 4-way biomedical multiple-choice dataset, with 5/31/272 instances in the training/validation
has 252.8k/4.2ko
sets. We evaluate the Out-of-Domain (OOD) performance of our method on the test set without adaptation.
Context Generation with LLM. We use the gpt-3.5-turbo via OpenAl A and employ greedy decoding for in-context learning. Each dataset has five-shot medical examples, shown in Figure
Training SLMs with Contextual Information. After acquiring data from LLMs, we utilize BioLinkBert-Bass (Yasunaga et al.] 2022a), BioLinkBert-Large (Yasunaga et al. (022a), and BioMedLM (Bolton et al.) 2022) SLM backbones for Fine-Tuning with Context (FTC). We compare FTC with privacy-restrice baselines that leverage additional knowledge to aid medical decision making: QA-GNN (Yasunaga et al.) (2022b). GreaseLM (Zhang et al., 2022). DRAGON (Yasunaga et al. 2022c), MurKe (Miura et al. 2020), MOEBQA (Dai et al. (022). HDRN (Mao et al. 2022) and VOD (Livvin et al. 2022). Also, we perform SLM standard fine-tuning (SFT) without any external knowledge and LLM prompting with keywords and candidate answers (LLM) to validate our method's efficacy. To ensure fair comparison, keep the backbones and hyperparameters consistent for both FTC and SFT approaches. For BioLinkBERT--BR
We conduct three separate runs for each setting and report the average results along with the standard deviation. We report only a single run for BioLinkBERTT-arg and BioMedLM due to high computational cost 3) The performance are measured by accuracy (%).
4.2 SUPERIOR MEDICAL DECISION PERFORMANCE OF FTC
Table [ compare results between Fine-Tuning with Context (FTC) and baselines. FTC significantly outperforms standard fine-tuning (SFT), with improvements of up to 7.96%, 21.23%, and 17.05% in absolute accuracy the test sets of MedQA and HEADQA, and the development set of MedMCQA, respectively. Furthermore, FTC exceeds LLM by 14.20%, 15.75%, and 16.90% on these datasets in privacy-restricted scenarios. These results indicate that SL.Ms effectively leverage the medical knowledge provided by the LLM to aid decision-making highlighting the benefits of incorporating context from the LLM SLMs training process.
FTC with BioMedLM achieves SOTA performance on both MedQA and HEADQA datasets. Par- ticularly, for HEADQA, FTC with BioMedLM backbone outperforms the best baseline MOEBQA DDai et al. 2022) by 18.80% and 16.55% in absolute accuracy on the development and test sets, respectively. This demonstrates the considerable impact of the FTC on enhancing the performance of SLMSs in medical tasks. Compared to the complex VOD Liévin et al. (2022) baseline, retriever-and- reader framework with multiple training strategies, our approach more straightforward. We simply fine-tune SLMs and include only one context per candidate answer generated by the LLM. Despite this, our method achieves superior performance in MedQA and secures second place MedMCQA, utilizing less than 6% of the training data required by the VOD 10
https: http::::/ppaaafrrrs
http:/ccoooocccccceeeecc s//pt 3-5 We provide implementation and training details Appendix
A.2
in theory, FTC could be integrated with VOD Liévin et al. 2022 ready.
We intend to do this once their code is
5

---

Published as a conference paper at ICLR 2024
Table Performance comparison (%) on MedQA, HEADQA and MedMCQA. Best results are bold and second best are underline. f: results from their original papers. T: results from Yasunaga et al. (2022c). results from Bolton et 2022). 3: VOD uses 180k training instances for MedMCQA, in contrast to our approach which utilizes only 10k instances.
MedQA
HEADQA
MedMCQA
dev
dev
test
QA-GNN Yasunaga et al.
2022b
45.0
GREASELM M (Zhang al.
2022
45.11
DRAGON Yasunaga et al.
2022c
47.5
HDRN(Mao ct al
2022
47.61
MurKe Liu al. 2020)
46.7
MOEBOA Dai et al. 2022
39.91
41.61
44.3
46.71
VOD (Liévin el al. 2022]
BiooinkBERT &M25
41.0t
40.41
51.6
+ BioLinkBERT& BiOLiTKBERT
53.61
55.01
58,3
LLM
38.30
41.70
47.60
47.50
35.20
SFT (w/o addtional knowledge)
BioLinkBERT Basc
41.22..4
42.21o.91
39.141.a8
41.00o.34
32.152.29
+ BioLinkBERTT-arge
45.1*
39.53
41.61
35.86
+ BioMcdLM
50.31
48.68
50,33
43.63
FTC (Ours)
+
50.73g 35 51.02 53.85
50.17a.42
61.35o, 16
60.21
49. 19.20o_. 45
+ Biol inkBaRT-ee
53.10 55.90
62.30
62.18
50.38
+ BBBoMed.M
63.10
63.17
52.09
Table 2: Results (%) of LLM, SFT and FTC under different training sizes.
MedQA
HEADQA
MedMCQA
100
200
500
full
100
200
500
full
100
200
500
full
LLM
41.70
47.50
35,20
SFT 31.40um 33. 33.79 34 35.27 sa.
42.21p:m 36 44.54 39 19.00
34.88200
41.00
29.07 16 28.31n.80 31 31.43, 32.15
FTC 45.52 46.29).32 47.87 50.17,.2 55.031.10 10 16.18,.7 57 60.21 1.47 38.82;
40. n0.12 43.06 49.20
4.3 FEW-SHOT LEARNING ENHANCEMENT WITH FTC
Real medical environments often face scarce training data (Zhang et al. 2021 Ge et al. 2022 In this section, we explore additional contexts boost SLM's medical proficiency in few-shot setting. We experiment on LikBERT-BBaS with training sample sizes of {100, 200, 500) for all datasets. For every size, we randomly generate three data splits from the entire training set, performing a single for each split. Results shown in Table 2
FTC consistently surpasses SFT by a considerable margin, achieving absolute accuracy enhancements of up to 14.12% 22.57%, and 11.81% 81% for the test sets MedQA and HEADQA. and the development set of MedMCQA, respectively. These consistent gains demonstrate that our method not only enhances performance in full-training but also proves highly beneficial when training data is limited. Interestingly, SFT with full training data does not surpass LLM in HEADQA and MedMCQA, and achieves only comparable performance to LLM in MedQA.
In contrast, our FTC method consistently surpasses LLM and SFT with full training data in all tasks, even with few as 100 training data points. This underscores the efficacy of prompting medical knowledge from the L.M to boost the SLM's medical capacities.
4.4
(OOD) GENERALIZABILITY BOOST WITH FTC
To evaluate the generalizability of our approach, we investigate the OOD performance of FTC using BioLinkBERT-BB -Bse as the backbone, without additional training. The best model from the source domain in Section A.ction4.21ia is directly applied to the target domain. Table presents the OOD performance.
We present results for the test sets of all datasets, excluding MedMCQA, For those datasets with available development results, the results provided in the Appendix

---

Published as a conference paper at ICLR 2024
Table Accuracy comparison (%) between LLM on the target domain (upper), and FTC and SFT trained on a source domain (lower) and applied directly to the target domain.
MedQA
HEADQA
MedMCQA
MMLU HEADQA MedMCQA
HEADQA
MedMCQA MedQA MedMCQA
MedQA HEADQA MedQA
LLM
41.70
47.50
35.20
$2.94
SFT
35.57
31.322,38
35.623.19
34
30.141,07
31.770.37
10,90
38.842.1 13
32.976.07
FTC
47.26
49.25
55.27
61.90
41.144.26
45.98 58,95
54.532.11
54.66u.17
The OOD performancce of SFT is inferior compared LLM prompting in the target domain, indicating its limited generalization capabilities. Conversely, FTC consistently outperforms both SFT in OOD settings and LLM prompting baselines. This underscores the enhanced generalizability achieved by incorporating the medical context generated by the LLM into the SLM.
4.5 STRONG GENERAL APPLICABILITY OF FTC
Privacy concerns not only appear in the medical domain. In this section, we investigate whether our method is generally applicable beyond the medical domain. We use two general domain datasets, CommonsenseQA (CSQA) (Tamoor et al. 2018) and OpenbookQA (OBQA) (Mihaylov et al. 2018). un- der privacy-restricted settings in the full training scenarios. We adopt T5-base (Raffel et al., (020) as the SL.M backbone fol- lowing previous works (Li et al. [022 Wang et al. 2022) and utilize ieooonne-Decddrr (Izacard and Grave (020) to incor-
Table Accuracy comparison (% in general domain.
CSQA
OBQA
LLM SFT FTC
41.25
51.60
62.63p.17
56.93p.25
65.87p.23
68.60,.43
porate contexts. We conduct three separate runs for each setting
Table
shows the results.
FTC consistently outperforms LLM and SFT baselines on two datasets. Specifically, FTC performs 3.24% and 11 11.67% better than its standard finetuning counterpart, SFT, in CSQA and OBQA, respectively. This demonstrates the effectiveness of our method, utilizing the LLM as a strong knowledge base and prompting knowledge within the LLM in privacy-restricte scenarios, which in turn enhances the SLM's ' knowledge capacities and improves decision-making.
5 ANALYSIS
5.1 CONTEXT ANALYSIS
To further understand the effectiveness of context generated by the LLM on SLM performance, we perform ablation studies with BioLinkBert-Bait Base as the SLM backbone.
Role of Context Parts. We investigate the effectiveness of each part of the context by separately providing (1) overall context (Only Overall) and (2) specific context for each candidate answer (Only Specific) as additional information the SLM in both few-shot and full full-training settings. Results are shown in the upper part (a) of Figure 4 Despite decreased performance when providing only overall or specific context compared to FTC, SI.Ms with added medical context still outperform SFT baselines across various training settings, demonstrating the importance of both context aspects in informed decision-makinn settings could be attributed to it offering general medical knowledge, while specific context provides tailored knowledge for each candidate answer.
The more pronounced performance drop for overall context in most
Content Learned by SLM. Specific context for each candidate answer includes its relationship to overall context, as shown Figure 3 For example, the relation of answer (a) is "It is not a treatment for the patient's symptoms." We explore whether the SLM learns from medical knowledge merely cherry-picks relationships by removing all relationship information in specific contexts (No Relation), retaining only the knowledge content. Then, we train the SLM using these modified contexts in both few-shot and full-training scenarios. Results are shown in the lower part (b) of Figure 4 When relationships are removed, performance declines compared to the FTC. However, even without any relationship information, the SLM with medical contexts consistently and significantly outperforms the SFT. This suggests that, although relationships assist in meaking the SLM prioritizes medical knowledge from context over simply replicating relationships directly.
Context Case Study. To examine what medical knowledge generated by the LLM using keywords and candidate answers, and how FTC reasons with these generated contexts, analyzed instances
We defer detailed experiment settings to the Appendix A.6

---

Published as a conference paper at ICLR 2024
Figure 4: Results of ablation studies
The upper part examines the effect of context components on SLM training, while the lower part investigates the impact of relationships within the context.
B
FTC
Only Overall
Only Specific
SFT
50
45-
45-
40
40
35
30
100
200
500
full
30
100
200
500
full
25
100
200 MedMCQAA
500
ful
MedOA
HEADQA
(a)
FTC
No Relation
SFT
50
60
45
50
40
35
35
30
100
200
500
full 30
100
200
full 25 160
200
MedQA
HEADQA (b)
MedMCQA
Figure 5: Case Study: MedQA test set contexts and predictions. yellow highlights important local information; underlined indicates L.LM-selecter keywords for context generation; green and red signify correct and incorrect contexts that could aid or confuse the SI.M, respectively. FTC succeeded 113 instances where SFT and LLM failed: 45 Targeting (left) and 68 Denoising (right).
Question: After a cel progenitor leaves the bone marrow, most of its dovelopment takes place in the thymus. it passes through several stages during maturation. Fostivve selection important
Question: A healthy 23 year old male undergoing an esercise stress test as part of his physiology class. If blood were be sampled impled at different locations before and after the stress test, which area of the body would contain lowest orygen contentia
during maturation preservation self major histocompat complex MHC]-restticedd cells, During which the following devniopmentl stages does cell underge this process?
lymphocytes,
it ensures
time points?
Question Kaywordic healthy, male, soerise stress test, physiology, blood, sampled, locations, stress body, paspen, time
develooment, thymus, stages, maturation, Positive selection, process, maturation, T hyppoocytes, preservation, self- major histocompat bility complex, T cells, develoomental stages, I cell, process
Question Keywords: cell progentorr bone
Candidate Answers: (6) inferior vena caus [b] Cornnary sinus [c) Pulmonary Pulmonary vein
Candidate Answers: PreiT cell Double positive Single positive CD4 T Single poshive CDS+ cell
Context: During an exercise strees test, biood is sampled from various
locations the body to assess changes in phys ology and oaygen levels at
Context: the cell progenitors develop the bane marrow and undergo maturation
different time
cell maturation the precervation of salf-major histocampatiillly com complex (MHC) recognition.
thymus. Positive selection
[e]: infertar
decaygenated bllond from the lower body to heart, but not location where blood
large that
lal: PreiT calls are T call progenitors that have not yet undergone maturation in the
sampled during an exercise stress test.
thymus. They are not yet at the stage positive selection.
[b]: The coronary sinus is vein that crains blood from the heart muscle. it
Double positive cells at stage postive selection, (e) Single-positive CD4+ cells are cells that express only the CD4 co-receptor. They are at the stage of positive selection and are able to recognize self-MHC class il molecules.
cells that express both CD4 and CDS co-recoptors. They
not location where blood sampled led during
they are not yet singleposttive posttive cells.
[c]: pulmonary artery corries deoxggerated
heart
lungs, and blood may be sampled from this location during an exercise stress test.
(c) Single positive CDB+ T colls are I colls that express only the CDB co co-receptor. They are
[d]: The pulmonary win carries crygonated blood from the lungs the heart, but
stage pesitive selection
able recognize self MHC
molecules.
locarion where bloed sampled during everche stress test. LLM preliminary decision: (xIX SFT prediction: (dX FTC prediction: /bly Denoising
LM preliminary decision: (dIX SFT prediction: (s) Targeting
FTC prediction: Y
from the MedQA test set where the FTC correctly predicted answers while both the SFT and LLM failed. Clinicians identified two distinct categories among the cases: (I) Targeting, where the LLM successfully refines the target scope and generates high-quality medical contexts, albeit arriving at an incorrect answer; the SLM integrates these contexts with the raw question and correctly predicts the answer. (2) Denoising, where the LLM fails to figure out the correct relationship of the correct answer and generates noisy medical knowledge: the SLM effectively obtains useful information, combines it with localized data, and ultimately makes the correct prediction. Figure 5 provides examples of each case. This case study demonstrates that I.LMs can generate valuable medical information even when making incorrect decisions based partial data, and that the FTC can extract useful medical knowledge from noisy contexts, thereby enhancing SLM medical reasoning capabilities.
Context Quality. To further quantitatively demonstrate that LLM- context retains medical knowledge even with incorrect preliminary decisions. We compared FTC and Fine-Tuning with Context and Rejection (FTCR) in full training settings. FTCR uses context for SLM training only if the LLM's 's preliminary decision is correct; otherwise, no additional context is provided. The results are in Table 5 FTC consistently outperforms FTCR across three medical tasks, implying valuable
8

---

Published as a conference paper at ICLR 2024
Table 6: Privacy budget statistics in MedQA. Avg. K and Avg. Q are the average word count for keywords and raw questions, respectively, across the dataset. Budget is privacy budget.
Table 5: Performance comparison of FTC and FTCR in the full-training setting [H
MedQA
HEADQA
MedMCQA
FTCR
48.122..0
57.79p.at
46.550.28
Avg. K
Avg. Q Budget
FTC 50.17u.42
60.21
49.20
Train Dev Test
49.1
116.2
42.3%
50.7
119.6
42.4%
Table 7: Results of different information repre- sentation methods, maintaining the same privacy budget on MedQA.
Figure 6: LLM and FTC results under different keyword usage ratios MedQA. Standard divi- sions of FTC and SFT are omitted for simplicity.
Dev
Test
50-
LLM prompting
Random Span
27.59
28.52
Random Words
30.03
30.48
Keywords
38.30
41.70
30-
SLM fine-tuning
SFT
35.670.47
33.99p.s7
20
L
FTC
SFT
Random Span
Random Words
44.181.11
42.950.33
45.061.35
44. 10.100.8
25%
50%
75%
100%
1025%
50%
75%
100%
Keywords
46.42b.2s
47.91o.51
Dev
Test
medical knowledge remains in contexts even with LLM's incorrect decisions. SLM can harness these insights from impfect contexts to enhance medical capabilities. 5.2 PRIVACY ANALYSIS
We conduct privacy analysis MedQA using BoLLinnBERR-Bass and introduce the privacy budget, a metric estimating information usage, presented in Table The privacy budget is calculated as the ratio of the number of words provided the LLM to the total words in the original question. Lower privacy budgets signify better privacy preservation 15
Why Keywords? We evaluate the effectiveness of using keywords (Keywords) to represent raw data when querying LLMs and compare it to two other methods of raw data representation: (I) random consecutive word spans (Random Span), and (2) random word bags from the original data (Random Words). Given the same privacy budget, we randomly select shared set of 1000 training instances for each method, use these query the LLM for medical contexts, and then use these contexts additional input for SLM training. Table displays the accuracy of LLM prompting and SLM fine-tuning. All FTC methods consistently outperform the SFT baselines, demonstrating their effectiveness. Keywords perform better than Random Span and Random Words, providing more efficient representation of medical knowledge within the same privacy budget. LLM prompting performance parallels SLM training, underscoring the importance of raw data representation in context generation and effective SL.M training.
Privacy Budget-Model Performance Trade-off. We analyze the trade-off between privacy budget and model performance by generating context from the LLM using randomly selected (25%, 50%, 75%, 100%) of keywords and training the SLM with full training data and corresponding context. FigurefJdispplas the accuracy for LLM prompting and SLM fine-tuning at various privacy budgets. As privacy budget increases, performance improves. Impressively, FTC outperforms SFT using context from just 25% of keywords, resulting in LLM prompting performance below 15%- siggiiinificanlly lower than the 25% random guess rate. This suggests that L.LM -enerrtdd context maintains essential medical knowledge despite limited raw data information, and the SLM effectively learns from it.
CONCLUSION
We introduce simple yet effective pipeline that enhances the SLM performance in medical tasks by using medical keywords to prompt LLMs within privacy-restricted scenarios. Our experimental results across three medical tasks in various training settings underscore the effectiveness of our proposed approach. Through comprehensive analysis, gain a deeper understanding of our method capabilities and the impact LLMS on SLM performance in privacy-restricted scenarios.
We present results for test sets of all datasets, excluding McdMCQA, For those datasets with available
development set results, the results are provided in the Appendix A.3
We defer BPC evaluation result on privacy appendix A.4

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
We would like to express our gratitude to Zhiyu Chen from Meta Reality Labs, Ming Yi, and Hong Wang from the Computer Science Department UCSB, as well as the anonymous reviewers, for their invaluable feedback. Additionally, we extend our thanks to Rachael A Callcut and Anamaria J Roble for their insightful discussions and guidance on medical prompt designs. Furthermore, we gratefully acknowledge the generous financial support provided by the National Institutes for Health (NIH) grant NIH 7R01HL.149670.
REFERENCES
Tom Brown, Benjamin Mann, Nick Ryder, Mclanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Ben- jamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1901. Curran Associates, Inc., 2020. URL htttps::/proceeeeeeeet neurips neurips 96 7 741 211bfbaac142f64aa-aaaaa
CC hc.paper/20200 2tt.:////..111117700002202222110
pdf
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek B Rao, Parker Barnes, Yi Tay, Noam M. Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Benton C. Hutchinson, Reiner Pope. James Bradbury. Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier García, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hyeontack Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowyez, Erica Oliveira Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathleen S. Meier- Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language modeling with pathways. ArXiv, abs/2204.023111 2022.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens. Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, 2022.
OpenAl. Introducing chatgpt, 2022. URL https::/ wwt..ccmm/llo...... chhp:/ccattpppppp 2023-05-11.
Accessed:
OpenAI. Gpt-4 technical report, 2023.
Karan Singhal, Shekoofch Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis. Stephen Pfohl, et al. Large language models encode clinical knowledge. arXiv preprint 132 2022.
Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz. Capabilities of gpt-4 on medical challenge problems, 2023.
Valentin Liévin, Christoffer Egeberg Hother, and Ole Winther. Can large language models reason about medical questions?, 2023.
Shiyang Li, Jianshu Chen, Yelong Shen, Zhiyu Chen, Xinlu Zhang, Zekun Li, Hong Wang. Jing Qian, Baolin Peng, Yi Mao, Wenhu Chen, and Xifeng Yan. Explanations from large language models make small reasoners better, 2022.
Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu. Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-specific language model pretraining for biomedical
10

---

Published as a conference paper at ICLR 2024
natural language processing, ACM Transactions Computing for Healthcare (HEALTH), 3(1): 1-23, 2021.
Michihiro Yasunaga, Jure Leskovec, and Perey Liang. Linkbert: Pretraining language models with document links. arXiv preprint arXhv:2203.158227 2022a.
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, (0(4)):1334-222 2020.
Emily Alsentzer, John R Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew McDermott. Publicly available clinical bert embeddings. arXiv preprint arXiv:1904.0333 2019.
Elliot Bolton, David Hall, Michihiro Yasunaga. Tony Lee, Chris Manning, and Percy Liang. Pubmed gpt.
htt////ff.aafffff..e///////////a Accessed: 2023-03-29.
.ttll 2022.
Eric Lehman, Evan Hernandez, Diwakar Mahajan, Jonas Wulff, Micah J. Smith, Zachary Ziegler, Daniel Nadler, Peter Szolovits, Alistair Johnson, and Emily Alsentzer, Do we still need clinical language models?, 2023.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. Language models are unsupervised multitask learners. 2019.
Colin Raffel, Noam Shazeer, Adam Roberts. Katherine Lee, Sharan Narang. Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020. URL
html
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoe Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023.
Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. Reframing human-ai collaboration for generating free-text explanations. arXiv preprint arXiv: 12.08644, 2021.
Yao Fu, Hao-Chun Peng, Litu Ou, Ashish Sabharwal, and Tushar Khot. Specializing smaller language models towards multi-step reasoning. ArXiv, abs/2301.12726, 2023.
Namgyu Ho, Laura Schmid, and Se- Young Yun. Large language models reasoning teachers. ArXiv, abs/2212.10071, 2022.
Kumar Shridhar, Alessandro Stolfo, and Mrinmaya Sachan. Distilling multi-step reasoning ca- pabilities of large language models into smaller models via semantic decompositions. ArXiv, abs/2212.001933, 2022.
Mark Neumann, Daniel King, Beltagy, and Waleed Ammar. ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing. In Proceedings the 18th BioNLP Workshop and Shared Task, pages 319-327, Florence, Italy, August 2019. Association for Computational Linguistics. doi: 1.118663/11/11--5000 URL https: /ttp://wwwww... www.ww.wwbb......... org/ anthology/ W1 W19-5034]
Tiffany H Kung. Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, et al. Per- formance of chatgpt on usmle: Potential for ai-assisted medical education using large language models. PLoS digital health, 2(2):e0000198, 2023.

---

Published as a conference paper at ICLR 2024
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery. Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roherts, Denny Zhou, Quoe V. Le, and Jason Wei. Scaling Iiatruction-finettun language models, 2022.
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. Don't t stop pretraining: Adapt language models to domains and tasks, 2020.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network, 2015.
Peifeng Wang, Aaron Chan, Filip Ilievski, Muhao Chen, and Xiang Ren. Pinto: Faithful language reasoning using
rationales, 2022.
Xinlu Zhang, Chenxin Tian, Xianjun Yang, Lichang Chen, Zekun Li, and Linda Ruth Petzold. Alpacareeinstruction- large language models for medical application, 2023.
Jayetri Bardhan, Anthony Colas, Kirk Roberts, and Daisy Zhe Wang. Drugehrqa: A question answering dataset structured and unstructured electronic health records for medicine related queries. arXiv preprint arXiv:2205.01290, 2022.
Chaitanya Shivade et al. Mednli-a natural language inference dataset for the clinical domain. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium. Association for Computational Linguistics, pages 1586-1596, 2019.
Yasuhide Miura, Yuhao Zhang, Emily Bao Tsai, Curtis Langlotz, and Dan Jurafsky. Improving factual completeness and consistency of image-to-text radiology report generation. arXiv preprint arXiv:2010. 10044 2020.
Wenhao Yu, Dan Iter. Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. Generate rather than retrieve: Large language models strong context generators, 2023.
Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering, 2020. / wwtp://wwwwwwg/a//////abaaa/
Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. Qa-gnn: Reasoning with language models and knowledge graphs for question answering. 2022b.
Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D. Manning, Percy Liang, and Jure Leskovec. Deep bidirectional wegguugg-knwwwedde graph pretraining, In Neural Information Processing Systems (NeurIPS), 2022c.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming Wei Chang, REALM: retrieval- augmented language model pre-training. CoRR, abs/2002.08909, 2020. URL ltttps://axxiii
atg/abs// aataabs/00000 2002.000.. 08909
Kenton Lee, Ming- Wei Chang, and Kristina Toutanova. Latent retrieval for weakly supervised open domain question answering. CoRR, 00000000000000, 2019a. URL. htt.:ht:///axww..rg////////////e
1906. 00300
Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. eetrieval-augmentt generation for knowledge-immensieee NLP tasks. CoRR, abs/2005.11401. 2020. /ttp:://p://xx....mmmm wttp://wwww.aaaa....../.oo
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading wikipedia answer open-
domain questions. CoRR, abs/1704.00051. 2017. URL http:://rxiv. wttp abs////bb/11
00051
Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. Okapi trec-3. In Text Retrieval Conference, 1994.
12

---

Published as a conference paper at ICLR 2024
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. CoRR, abs/2004.04906, 2020.
URL httth::::///aa org. wwt.//as////// /2004 04906
Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang. RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering. In Proceedings of the 202| Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techmologies, pages 5835-5847, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/ 2021 Reel.nacc--on moadd.maii66666
URL https::////m ptttps://aclantholoosso org/ www.nnnaaaaaa.aa mww.. 466
Devendra Singh Sachan, Siva Reddy, William L. Hamilton, Chris Dyer, and Dani Yogatama. End-to- end training of multi-document reader and retriever for open-domain question answering. CoRR, abs/2106.05346, 2021. URL https: hwwp/aaaivvvv.. wwtpw/wwwwwwwaaaa///ot/a.ataaaaa 2ttp:///2266..33333
Kenton Lee, Ming-Wai Wei Chang, and Kristina Toutanova. Latent retrieval for weakly supervised open domain question answering. CoRR, hts//900.003000 2019b. URL http: http:::/aaxxvv.rrg/.oo/ 1906. 00300
Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D. Manning, and Jure Leskovec. Greaselm: Graph reasoning enhanced language models for question answering, 2022.
Fabio Petroni, Tim Rocktäschel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. Language models as knowledge bases? CoRR, abs/1909.01 01066, 2019.
/1099../119....0 C1066
Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of language model? CoRR, ths/2002.08910, 2020. URL https: httpttts///axxii.orrgggg
abs/ 2bss2002.cm 08910
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, and Hannaneh Hajishirzi. Generated knowledge prompting for commonsense reasoning. CoRR, abs/21 10.08387 2021. URL
http:///ww.v..rrr///////0/..0./0 08387
Samuel Sousa and Roman Kern. How to keep text private? a systematic review of deep learning methods for crivacy-preserving natural language processing. Artificial Intelligence Review, 56(2): 1427-1492, 2023.
Accountability Act. Health insurance portability and accountability act of 1996. Public law, 104:191, 1996.
Raphaël Chevrier, Vasiliki Foufi, Christophe Blavinnn, Arnaud Robert, and Christian Lovis. Use and understanding of anonymization and de-identification in the biomedical literature: scoping review. Journal of medical Internet research, 2 21(5)xe 2019.
Joffrey Leevy, Taghi Khoshgoftaar, and Flavio Villanustre. Survey on rnn and crf models for de-identification of medical free text. Journal of Big Data, 7:1-22, 2020.
Zengjian Liu, Buzhou Tang, Xiaolong Wang, and Qingcai Chen. De-identification of clinical notes via recurrent neural network and conditional random field. Journal of biomedical informatics, 75: S34 $42, 2017.
Franck Dernoncourt Ji Young Lee, Ozlem Uzuner, and Peter Szolovits. De-identification of patient notes with recurrent neural networks. Journal of the American Medical Informatics Association, 24(3):596-606, 2017.
Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Lco Anthony Celi, and Roger Mark. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1-9, 2016.
Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. Mimic-iv. PhysioNet. Available online at: https://physionet. www.onte//miiiccc/llooaaacssssss August 23, 202 I), 2020.
13

---

Published as a conference paper at ICLR 2024
Cynthia Dwork. Differential privacy: A survey of results. In Theory and Applications of Models of Computation: 5th International Conference, TAMC 2008, Xi an, China, April 25-29, 2008. Proceedings 5, pages 1-19. Springer, 2008.
Natasha Fernandes, Mark Dras, and Annabelle MeIver. Generalised differential privacy for text document processing. In Principles of Security and Trust: 8th International Conference, POST 2019, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2019, Prague, Czech Republic, April 6-11, 2019, Proceedings 8, pages 123-148. Springer International Publishing, 2019.
Martin Abadi, Andy Chu, lan Goodfellow, Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308-318, 2016.
Oren Melamud and Chaitanya Shivade. Towards automatic generation of shareable synthetic clinical notes using neural language models. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pages 35-45, Minneapolis, Minnesota, USA, June 2019, Association for Computational Linguistics. doi: 10. 10.1863//1//1911193 19-5. URL httph:t/p//. //acl htt///aaacthhhhoo antho logy
oxq/R19-1905
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams, 2020.
David Vilares and Carlos GódezzRodríguez. Head-qa: A healthcare dataset for complex reasoning. 2019.
Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmeqa A large-scale multi-subject multi-choice dataset for medical domain question answering, 2022.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021.
Damai Dai, Wen-Jie Jiang, Jiyuan Zhang, Weihua Peng, Yajuan Lyu, Zhifang Sui, Baobao Chang, and Yong Zhu. Mixture of experts for biomedical question answering. ArXiv, abs/2204.07469, 2022.
Jianuuo Mao, Jiyuan Zhang, Zengfeng Zeng, Weihua Peng, Wen-Jie Jiang, Xiangdong Wang, Hong Liu, and Yajuan Lyu. Hierarchical representation-basee dynamic reasoning network for biomedical question answering. In International Conference on Computational Linguisties, 2022.
Valentin Liévin, Andreas Geert Motzfeldt, Ida Riis Jensen, and Ole Winther. Variational open- open-domain question answering, 2022. URL
06345
Ye Liu, Shaika Chowdhury, Chenwei Zhang, Cornelia Caragea, and Philip S. Yu. Interpretable multi-step reasoning with knowledge extraction on complex healthcare question answering, 2020.
Xinlu Zhang, Shiyang Li, Zhuowei Cheng, Rachael Calleut, and Linda Petzold. Domain adaptation for trauma mortality prediction in ehrs with feature disparity. In 202/ IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 145-1152, 2021, doi: 15.1109// 2021.9660 0021.9669798.
Yao Ge, Yuting Guo, Yuan-Chi Yang, Mohammed Ali Al-Garadi, and Abeed Sarker. Few-shot learning for medical text: A systematic review, 2022.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsensega: A question answering challenge targeting commonsense knowledge. CoRR, abs/1811.00937. 2018. URL
http:://arxlv.or///b//////
.09937
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? new dataset for open book question answering. In EMNLP, 2018.
14

---

Published as a conference paper at ICLR 2024
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the- art natural language processing. In Proceedings of the 2020 Conference on Empirical Meth- ods in Natural Language Processing: System Demonstrations, pages 38-45, Online, October 2020. Association for Computational Linguistics. doi: 10.18653/v http:////222.m.mmm wwtp:0000emmmmdmmmme..o.m URL www...rrgggae 2020. emnlp-demos.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019.
15

---

Published as a conference paper at ICLR 2024
LIMITATIONS
While our work enhances SLM performance by using keyword representations of raw data, it only mitigates but does not eliminate privacy concerns. Given that FTC is based on GPT3.5, the medical knowledge it generates may be inaccurate or biased, which can impact SLM performance. Moreover, the inference time for LLMs may slower than that of SLMs, leading to longer overall inference times compared to models solely reliant on local SLMs. Furthermore, due to the training cut-off time, the medical knowledge LLM could be outdated, potentially hindering medical decision makiin.. We aim to integrate L.M with the internet and knowledge graph in future work to generate more reliable medical knowledge for enhancing SI.M decision-making These issues underscore the need for further research on the use of LLMs in privacy-restricted medical scenarios.
A APPENDIX
A.I DATA AND CODE
Our codes and generated data are public at httpss///gtthbb.com//Zaaaaas PrivacyBoost-S5
A.2 SLM IMPLEMENTATION AND TRAINING DETAILS
We implement both SFT and FTC based on huggingface transformers Wolf et al. (020)), and train on NVIDÍA A40-48GB GPUs For all datasets, we utilize A (Loshchilov and Hutter 2019) as optimizer. For MedQA and HEADQA, we set learning rates of 5 x 10-b, 5 x 10 and 2 x 10 for BioLinkBERT-Base. BioLinkBERT-Large and BioMedLM in both FTC and SFT settings. For MedMCQA, we set learning rates of 2 10-5 2 10-5, 5, and x 10 for BioL BioLinkBERT-Bass BioLinkBERT-Large, and BioMedLM both FTC and SFT settings. For BioLinkBERT Base and BioLinkBERT-Large limit training 100 epochs with 200-step warm-up and apply early stopping after epochs without validation improvement. Batch sizes are for few-shot and full- training scenarios across all datasets. For BioMedL.MM set the training epochs to 10 for all datasets. We run experiments with three random seeds {0, 1, 2 and report mean results and standard deviations.
A.3 ADDITIONAL EXPERIMENTAL RESULTS
Development sets results of MedQA and HeadQA. We report development sets results of MedQA and HeadQA in Table S and
9 and Figure 7
Table Results (%) on development sets MedQA and HEADQA between LLM, SFT, and FTC under different training sizes.
MedQA
HEADQA
100
200
500
full
100
200
500
full
LLM
38.30
47.60
SFT
33.28p.s5
34.01 1.00
34.20a.96
42.21os
36.546.0
39. 39.00p.61
34.882.00
41.48o.48
FTC
43,66
45.70m.10
45.62
50.730.5
55.03
56.18
57.45
60.21
Table 9: Results comparison of FTC and FTCR in the full-training setting on development sets of MedQA and HEADQA.
MedQA
HEADQA
FTCR
48.56..e
58.170.78
FTC
50.73
61.35
16

---

Published as a conference paper at ICLR 2024
Figure Accuracy comparison (%) of ablation studies on development sets of Med and HEADQA. The upper part of the table examines the effect of different context components on SLM training, while the lower part investigates the impact of relationships within the context.
FTC
Only Overall
Only Specific
SFT
60
45-
50-
40
40
35
100
200
500
fu
100
200
500
Tull
MedoA
HEADDA
(e)
FTC
No Relation
SFT
50
45
50
40
40
35
30
100
200
500
full
100
200
500
fu
MedQA
HEADOA
(b)
A.4 ALTERNATIVE MEASUREMENT FOR PRIVACY BUDGET
We further conducted BPC measurement experiments on the training, validation, and test sets of the MedQA dataset, respectively, to demonstrate the effectiveness of using keywords to represent raw medical data while keeping privacy. To evaluate the BPC of raw data and keywords, we separately input keywords with various proportions and raw data into BioMedLM across different sets. Specifically, for each list of keywords, we form a sentence of keywords by concatenating the list of keywords and separating each pair of keywords with an empty space. Subsequently, we calculate the corresponding BPC values assess the outcomes. The results are shown in Table I0
Table 10: Comparison of BPC values for raw data and concatenated keywords across different data
splits of the MedQA dataset
Raw Data
Keywords
25%
50%
75%
100%
Training
12.41
10.37
10.04
9.91
9.87
Validation
12.44
10.31
10.01
9.92
9.89
Test
12.44
10.32
10.06
9.94
9.91
The BPC values of keywords consistently exhibit lower values than those of the raw data.
A higher BPC is consistently observed across various subsets of the MedQA dataset when compared to the approach of inputting keywords into BioMedLM. This comparison implies that, on average, raw data holds greater level of uncertainty in contrast to the utilization of keywords. This disparity could be attributed to the fact that raw data encompasses a larger amount of -unrelalddde information, which includes privacy-related data, despite its comprehensive information coverage.
decrease in the proportion of keywords results in an increase in BPC.
Decreasing the number of keywords fed into the BioMedLM leads increase in BPC. This indicates that a higher volume of keywords contributes to a more meaningful representation of medical information in raw data, subsequently enhancing the performance of both LLM and FTC.
17

---

Published as a conference paper at ICLR 2024
Keywords representation obtains the lowest BPC compared to random words and random span. We further evaluate the BPC different raw data representation methods (random span, random words and keywords) while maintaining the privacy budget the same. Specifically, data representation content into BioMedI.N .M on the training, validation and test sets of the MedQA dataset, and calculate the corresponding BPC values respectively. The results are presented in Table
feed different
Table 11: BPC values different raw data representation methods.
Representation Methods
Keywords
Random words
Random span
Training
9.87
10.04
12.25
Validation
9.89
10.05
12.22
Test
9.91
10.04
12.22
Keyword representation consistently obtains the lowest BPC compared the other two data rep- resentation methods, providing more effective representation of medical knowledge within the same privacy budget. Interestingly, the order of BPC performance aligns with the performance of LLM prompting and SLM fine-tuning in Table 7 of our paper. The method with a lower BPC value achieves better performance in both LLM prompting and SLM fine-tuning. This further highlights the importance of raw data representation in generating high-quality context and effective SLM training.
A.5 EXPERIMENTS WHAT SLM LEARNS FOR DECISION MAKING.
We feed preliminary decisions (PD) context into SLM with backbone BioLinkBert-Bass three datasets. Three separate runs for each setting are conducted and the average results along with the standard deviation are reported. The results are shown in the Table 12
Table 12: Results comparison of SLM with preliminary decisions and FTC.
MEDQA
HeadQA
MEDMCQA
SLM w PD FTC
47 47.21g.31
53.641.09
45.42,.17
50.17
61.35p.16
49.200.45
FTC, which integrates extensive medical knowledge into the decision-making shows consistent and significant improvement over the SLM that only uses PD for context. These findings underscore the valuable contribution of leveraging comprehensive medical knowledge, provided by LLM, in enhancing the medical decision decision-making capabilities.
A.6 GENERAL DOMAIN EXPERIMENTAL SETUP
We perform experiments on two commonsense datasets to demonstrate the broad applicability of our approach.
Datasets. 1. Commonsense(AA Talmor et al. 2018) is multi-choice question-answering dataset featuring options per question, requiring commonsense reasoning. The dataset is split into 9741/1221/1144 140 instances for training, development, and test sets, respectively. As the test set not publicly accessible, we follow previous work (Li et al.) (202)) and report results on the develop- ment set. 2. OpenbookQA is 4-option multi-choice question-answering dataset that demands open book facts, broad common knowledge, and multi-hop reasoning (Mihaylov et al. 2018). The dataset is split into 4957/500/500 instances for training. development, and test sets, respectively. We report results on the test set.
Context generation from LLM. We utilize the grounded entities from Zhang et al. (022) as
keywords to query the GPT-3.5 gpt-3.5-tuee 3-rbbo engine, generating context through greedy decoding process (by setting the temperature 0). We employ the same demonstration format in the medical domain and create rritace-eettricted context in accordance with the in-context learning paradigm. For each dataset, supply seven-shot hand-crafted examples.
18

---

Published as a conference paper at ICLR 2024
SLM training. We employ T5-base (Raffel al., 2020 the SLM backbone for both SFT and FTC. In FTC, we use decision-making Specifically, each context related to candidate answer is concatenated with the question and all candidate answers, then processed independently by the encoder. The concatenated representations of all contexts subsequently fed into the decoder generate predictions. In both datasets, we utilize AdamW (Loshchilov and Hutter 2019) with learning rate of 5 x 10-5 for both SFT and FTC. We limit training 100 epochs with 200-step warm-up and apply early stopping after 5 epochs without validation improvement. The batch size is set to 8 for both datasets.
(Izacard and Grave 2020) to incorporate context information for
A.7 PROMPT DETAILS
In this section, we present examples of prompts for both medical and general domains. The context of each example consists of three parts: (1) An overall context, which provides high-level information derived from the extracted keywords and candidate answers (red); (2) A specific context, which focuses the knowledge associated with a candidate answer (blue) and its relation to the overall context (green); and (3) A preliminary decision, which draws a conclusion based on the contexts provided earlier (orange).
Prompts for medical datasets. Three clinicians were involved in the prompt design and writing. We held 5 meetings with clinicians to discuss the prompt design and iterate 4 versions. The final version of context needs around 10 minutes write per context. Our medical prompts on MedQA and MedMCQA are based on SSinghal et al. (2022), HEADQA is based on Wikipedia and written and verified by clinicians. Here we provide the prompts that we used our experiments.
Table 13: Prompts for MedQA
Question Keywords: male, marathon runner, office, complaint, right-sided rib pain, Physical examination, normal heart, lung findings, exhalation, dysfunction, ribs 4-5, right, muscles, muscle groups, dysfunction, direct method
Candidate Answers:
(a) anterior scalene (b) latissimus dorsi (c) pectoralis minor (d) quadratus
lumborum
Context: Normal heart and lung findings on a physical exam coupled with evidence of exhalation dysfunction ribs 4-5 on the right suggest a musculoskeletal cause exertional chest pain. (a): The anterior scalene muscle attaches to the first rib. It is not associated with exhalation dysfunction in ribs 4-5.
(b): The latissimus dorsi muscle attaches to ribs 9 and 10. It is not associated with exhalation dysfunction in ribs 4-5.
(c): The pectoralis minor muscle is attached to ribs 3, 4, and 5. Dysfunction in the fourth and fifth ribs can be caused by issues with the pectoralis minor muscle due to its attachment to these ribs. It is associated with exhalation dysfunction in ribs 4-5. (d): Quadratus lumborum muscle attaches to ribs 11 and 12. It is not associated with exhalation dysfunction in ribs 4-5.
Therefore, the answer is (c).
Question Keywords: male, office, low back pain, denies, any, trauma, says, truck, day.job, Exami- nation, patient, prone position, deep sacral, left, posterior inferior lateral angle, right, lumbosacral junction, springs, freely, compression, diagnosis
Candidate Answers: (a) left-on-left sacral torsion (b) left-on-right sacral torsion (c) right unilateral sacral flexion (d) right-on-right sacral torsion
The physical exam shows the deep sacral sulcus on the left, a posterior inferior lateral angle on the right and normal spring test.
Context:
(a): This condition is characterized by the deep sacral sulcus on the right, a posterior inferior lateral angle the left and normal spring test. It is not consistent with the findings from the physical exam. (b): The left-on-right sacral torsion would be indicated by a deep sacral sulcus on the right, a posterior inferior lateral angle on the left, and positive spring test. It is not consistent with the findings from the physical exam.
Continued next page
19

---

Published as a conference paper at ICLR 2024
Table 13 Continued from previous page
(c): This condition is characterized by posterior inferior lateral angle on the right, a deep sacral sulcus on the right, and an absence of normal spring test. It is not consistent with the findings from the physical exam.
(d): This condition is characterized by a deep sacral sulcus on the left, a posterior inferior lateral angle on the right and normal spring test. It is consistent with the findings from the physical exam. Therefore, the answer is (d).
Question Keywords: man, comes, office, nonproductive cough. runny nose, frontal headache, headache, morning, nd. ibuprofen, relief, not, shortness of breath, Medical history. no medications, ibuprofen, pain, Vital signs, temperature, 37.4, 99.4, : pulse, 88/min, 18/min, blood pressure, 120/84, Examination, nares, erythematous, mucous membranes, Examination, throat, erythema, follic- ular lymphoid hyperplasia, posterior oropharynx, no, cervical adenopathy, Lungs, clear, auscultation, patient's, symptoms
Candidate Answers: (a) Allergic rhinitis (b) Epstein-Barr virus (c) Mycoplasma pneumonia (d) Rhinovirus
Context:
Sore throats are common symptoms in multiple upper respiratory viruses. (a): A non-productive cough is a common symptom in upper respiratory viruses but is not present in allergic rhinitis. It is not the cause of the symptoms. (b): The absence of shortness of breath indicates mycoplasma is less probable. It is not the cause of the symptoms.
(c): Cervical adenopathy is commonly seen in cases of Epstein Barr virus. The absence of cervical adenopathy indicates Epstein Barr virus is less likely. is not the cause of the symptoms. (d): Rhinovirus can cause this patient's 's symptoms, including sore throat, runny nose and a frontal headache. It is the cause of the symptoms. Therefore, the answer is (d).
Question Keywords: healthy, woman, comes, physician, 8, months, husband, killed, car crash, decreased, difficulty falling asleep, states, sad, cries, frequently, door lock, five, house, five, pieces, toilet paper, perfectionist, urges, rituals, Pharmacotherapyy neurotransmitters
Candidate Answers: (a) Dopamine (b) Glutamate (c) Norepinephrine (d) Serotonin
Context:
The woman is exhibiting symptoms of major depressive episodes, such as difficulty falling asleep, frequent crying, and persistent feeling of sadness.
(a): Dopamine is neurotransmitter that increases positive emotions. It is implicated in many disease processes, including Parkinson's 's and ADHD, and is targeted by antipsychotic medications but not used as sleep aid. It is not treatment for the patient's symptoms.
(b): Glutamate is a neurotransmitter that is associated with multiple neurological disorders including epilepsy, stroke, and autism. It is not treatment for the patient's 's symptoms.
(c): Norepinephrine is a catecholamine with adrenergic properties. It is not a treatment for the patient's symptoms.
(d): Serotonin is a neurotransmitter which is the target for multiple antidepressants, anxiolytics, and antipsychotics. could be a treatment to address the patient's symptoms of depression and anxicty. Therefore, the answer is (d).
Question Keywords: man, comes, office, preoperative, evaluation, adrenalectomy, scheduled, 2, weeks, One, month, care, emergency department, pain, right flank, motor vehicle collision, blood pressure, 160/100, mm Hg and CT scan, abdomen, incidental, left adrenal mass, laboratory studies, complete blood count, serum electrolyte concentrations, liver function tests, reference ranges, patient, healthy, elevated blood pressure, no medications, follow-up visit, office 2, weeks, disclosed, elevated, urinary normetanephrine, metanephrine, plasma, concentrations, patient, surgeon, recommended, adrenalectomy, vital signs, temperature, 36.6, 97.9, pulse, 100/min, 14/min, blood pressure, 170/95, Physical examination, no significant, findings, preoperative, preparation, treatment Candidate Answers: (a) Labetalol (b) A loading dose of potassium chloride (c) Nifedipine (d) Phenoxybenzamine
Continued next page
20

---

Published as a conference paper at ICLR 2024
Table 13 Continued from previous page
The patient is being evaluated for adrenalectomy due to large left adrenal mass, which is likely causing elevated blood pressure as a symptom of pheochromocytoma. Elevated urinary normetanephrines confirm the diagnosis.
Context:
(a): This beta-blocker works by blocking the effects of adrenaline and other stress hormones on the heart and blood vessels. It is not treatment for pheochromocytome (b): The use of a potassium chloride loading dose is a treatment specifically for hypokalemia, which a condition where there are abnormally low levels of potassium in the blood. It is not treatment for pheochromocytome.
(c): This drug commonly prescribed to treat high blood pressure and angina. It can also help relieve symptoms of Raynaud's phenomenon. It is not a treatment for pheochromocytoma. (d): This medication is used as a preoperative preparation treatment to block hipha-adrenergic receptors in the body and it effectively treats hypertension caused by pheocchromoytoma. It is treatment for pheochromocytoma.
Therefore, the answer is (d).
Table 14: Prompts for HEADQA
Question Keywords: autosomal dominant trait
Candidate Answers: (a) The trait appears more frequently in males. (b) The unaffected people do not transmit the trait. (c) The trait tends to skip generations. (d) The affected people have both affected parents. (e) The trait tends to appear in the progeny of related parents.
Context: Autosomal dominant inheritance is mode of genetic transmission in which trait or condition can be passed down from parent to child. One copy of mutated gene from one parent can cause the genetic condition. For example, let ''' represent the affected allele and "a" represent the unaffected allele. An affected person may have the genotype AA or Aa, while an unaffected person has the genotype aa. Consequently, an individual with genotype AA has 100% chance of passing on the affected allele, and someone with genotype Aa has a 50% chance of doing so.
(a): Autosomal dominant inheritance is not influenced by an individual's 's sex, as it not -ex-dependent. The expression of the trait occurs regardless of gender. It is not characteristic of autosomal dominant inheritance.
(b): Unaffected individuals do not have the mutated gene and therefore cannot transmit the trait. It is a characteristic of autosomal dominant inheritance.
(c): Autosomal dominant traits can be passed down through multiple generations. Since the affected allele is dominant, an individual will express the trait as long as they inherit the affected gene. It is not a characteristic of autosomal dominant inheritance.
(d): Only one affected parent is needed to transmit on the autosomal dominant trait to their child, It is not a characteristic of autosomal dominant inheritance.
(c):A dominant gene can appear in any progeny, regardless of the parent. It is not a characteristic of autosomal dominant inheritance. Therefore, the answer is (b).
Question Keywords:
caring, patient, supraglottic laryngectomy Candidate Answers: (a) He has lost the ability to speak by extirpation of the true vocal cords. (b)
The tracheostomy they have performed will be permanent. (c) You have risk of bronchoaspiration due to difficulty swallowing. (d) You may have constipation due to cervical dissection. (e) A portion of the larynx has been removed along with vocal cord.
Context: Supraglottic laryngectomy or horizontal partial laryngectomy an operation to remove the epiglottis, false vocal cords, and superior half of the thyroid cartilage.
(a): Supraglottic laryngectomy removes the false vocal cords, but the true vocal cords are not affected, and the patient's 'ss ability to speak should not be significantly impacted. It is not typical to lose the ability to speak by extirpation of the true vocal cords.
(b): If tracheostomy tube is in place after the procedure, it is typically removed within 24-48 hours of surgery. It is not typical to involve A permanent tracheostomy as a part of the supraglottic laryngectomy process.
Continued on next page
21

---

Published as a conference paper at ICLR 2024
Table 14 Continued from previous page
(c): Supraglottic laryngectomy results in severe disturbance to the swallowing mechanism by removal of protective layers and sensation. There is an increased risk of bronchoaspiration. It is related to care of patients with supraglottic laryngectomy.
(d): Constipation is not a side effect of supraglottic laryngectomy. It is not related to care of patients with supraglottic laryngectomy.
(e): Supraglottic laryngectomy is an operation to remove the epiglottis, false vocal cords, and superior half of the thyroid cartilage. In this procedure, the true vocal cords are not typically affected, preserving the patient's 's ability to speak as much as possible. It is not common to remove portion of the larynx along with true vocal cord during this procedure. Therefore, the answer is (c).
Question Keywords: estrogenic treatment, adverse effects, NOT, adverse effect, pharmacological action
Candidate Answers: (a) Edema (b) Breast pain (c) Ovarian cancer (d) Sickness (e) Headaches
Estrogen therapy involves supplementing a patient with estrogen, the primary female sex hormone. Potential side effects include breast tenderness or swelling, edema, nausea, leg cramps, endometrial cancer, and more.
Context:
(a): Edema is potential adverse effect of estrogen therapy. Estrogen and aldosterone both originate from cholesterol, and an excessive amount of estrogen in the body can stimulate aldosterone receptors, leading to water retention in nephrons. This water retention can result in edema. It is non-adverse effect.
(b): Estrogen promotes ductal growth and fat deposition in the breasts. Excessive estrogen levels can lead to mammary duct hyperplasia, which may result in breast pain. It is not a non-adverse effect.
(c): Ovarian cancer is not known be an adverse effect of estrogenic treatment. It is a possible choice for a non-adverse effect.
(d): Edema a possible adverse effect of estrogenic treatment, and swelling in body parts may cause the feeling of sickness. It is not non-adverse effect.
(e): Headache a possible adverse effect of estrogenic treatment. It is not a non-adverse effect. Therefore, the answer is (c).
Question Keywords: cardiac valvular prosthesis, biological, mechanical, implanted, patient, aspects,
characteristics, patient, prosthesis, INCORRECT, statement
Candidate Answers: (a) Permanent anticoagulation is necessary in mechanical prostheses. (b) In general, biological prostheses are indicated in young patients, with long life expectancy. (c) Biological prostheses would be indicated in cases that present formal contraindication for anticoagulation. (d) The rate of structural deterioration of a biological prosthesis is inversely proportional to the age of the subject. (e) Biological prostheses do not require permanent anticoagulation.
Cardiac valvular prostheses (biological or mechanical) are artificial cardiac valves implanted into a patient's heart. Mechanical valves may last a lifetime, but they come with an increased risk of blood clots, necessitating the use of blood thinners such as warfarin. In contrast, biological valves, which are made from pig or cow tissue, do not increase the risk of bleeding or clotting but tend to wear out sooner.
Context:
(a): Mechanical valves increase the risk of blood clotting. is not an incorrect statement. (b): The latest revisions of the ESC/EACTS guidelines suggest that bioprostheses are acceptable in patients aged between 60 and 65 years at the time of surgery. The reoperation rate for structural valve degeneration (SVD) of bioprostheses occurred exclusively among patients younger than 56 years. Young patients are not typically recommended for a biological prosthesis.
(c): Biological prostheses, which are made from pig cow tissue, do not increase the risk of either bleeding or clotting but will wear out sooner. It is not an incorrect statement.
(d): The disadvantages of biological heart valves are smaller valve orifice area and the risk of structural valve degeneration, which may necessitate reoperation. Thus, the younger the patient, the higher risk of structural deterioration. It not an incorrect statement.
(c): Biological prosthesis do not increase the risk of clotting so do not require permanent anticoagulant. It is not an incorrect statement. Therefore, the answer is (b).
Continued on next page
22

---

Published as a conference paper at ICLR 2024
Table 14 Continued from previous page
Question Keywords:
connection, automatic, emotional responses, control, behaviors, guiding, behavior, manifestation, emotional responses
Candidate Answers: (a) The angular gyrus of the limbic system. (b) The convolution lobe of the insula. (c) The prefrontal orbitofrontal or ventromedial cortex. (d) The thalamus (e) The cortex of somatosensory association.
The prefrontal orbitofrontal cortex has multiple functions including mediating context specific responding, encoding contingencies in flexible manner, encoding value, encoding inferred value, inhibiting responses, learning changes in contingency, emotional appraisal, altering behavior through somatic markers, driving social behavior, and representing state spaces. The orbitofrontal cortex thus plays key role in emotion, by representing the reward value of the goals for action.
Context:
(a): The angular gyrus (AG) a hub of several networks that are involved in various functions, includ- ing attention, self- procesrocsssng semantic information processing, emotion regulation, and mentalizing. It is not the area responsible for connecting automatic emotional responses and controlling complex behaviors.
(b): The insula important for gustatory and sensorimotor processing, risk-reward behavior, auto- nomics, pain pathways, and auditory and vestibular functioning. It is not the area responsible for connecting automatic emotional responses and controlling complex behaviors.
(c): The prefrontal cortex guides behavior by controlling the manifestation of emotional responses through understanding rewards, encoding values, and driving behaviors. It is the potential correct answer.
(d): The thalamus acts as the body's information relay station. All sensory information (except for olfaction) must be processed through the thalamus before being sent to the cerebral cortex for interpretation. It is not the area responsible for connecting automatic emotional responses and controlling complex behaviors.
(c): The somatosensory cortex is responsible for processing all bodily sensations. These sensations originate from receptors located throughout the body that detect temperature, pain, touch, pressure, and proprioception. It is not the area responsible for connecting automatic emotional responses and controlling complex behaviors. Therefore, the answer is (c).
Table 15: Prompts for MedMCQA
Question Keywords: Maximum, increase, prolactin level
Candidate Answers: (a) Risperidone (b) Clozapine (c) Olanzapine (d) Aripiprazole
The four drugs in answer choices are all atypical antipsychotics, which are used to treat psychotic conditions like schizophrenia through blockage of dopamine and serotonin receptors. These drugs block dopamine D2 receptors and serotonin 5-HT2 receptors. Maximum increase in prolactin, or hyperprolactinemia, is one of the side effects of atypical antipsychotics, because dopamine tends to inhibit prolactin release from the anterior pituitary. (a): Risperidone is a type of atypical antipsychotics that block dopamine D2 receptor and serotonin 5-HT2 receptor. It is generally used to treat schizophrenia or disorders with concomitant psychosis. Hyperprolactinemia is one of the most common side effects of risperidone. is the drug to increase prolactin levels.
Context:
(b): Clozapine is used to treat schizophrenia or disorders with concomitant psychosis. Clozapine is associated with side effects such as agranulocytosis, seizures, and myocarditis, but it does not appear to elevate prolactin levels. It is not the drug to increase prolactin levels.
(c): Olanzapine is used to treat schizophrenia or disorders with concomitant psychosis. The side
effect of olanzapine does not include hyperprolactinemia. It is not the drug to increase prolactin
levels.
(d): Aripiprazole is generally used treat schizophrenia disorders with concomitant psychosis. The side effect of olanzapine does not include hyperprolactinemia. It is not the drug to increase
prolactin levels.
Therefore, the answer is (a).
Continued on next page
23

---

Published as a conference paper at ICLR 2024
Table 15 Continued from previous page
Question Keywords: male, complains, severe back pain, inability, left lower limb, Radiographic studies, compression, nerve elements, intervertebral, foramen, vertebrae L5, SI, structure, space- occupying lesion
Candidate Answers: (a) Anulus fibrosus (b) Nucleus pulposus (c) Posterior longitudinal ligament
(d) Anterior longitudinal ligament
Context: The male is complained of a severe back pain and inability to move, and radiographic
evidence shows the compression of nerve component. This may suggest herniated intervertebral disk through a tear in the surrounding annulus fibrosus. The soft, gelatinous nucleus pulposus is forced out through weakened part of the disk, compressing nerve components of the spinal cord and resulting in back pain and nerve root irritation. This impingement is resulting in paralysis, and should be considered a medical emergency.
(a): Annulus fibrosus is tough, circular exterior of the intervertebral disc, made up of fibrous connective tissue. It surrounds the soft inner core, the nucleus pulposus. It is not the component that is forced out by the tear.
(b): Nucleus pulposus is the inner core of the vertebral dise, The tear the annulus fibrosus causes to be forced out. It could result in compression of the nerve components of the vertebrae.
(c): Posterior longitudinal ligament connects and stabilizes the bones of the spinal column. It runs almost the entire length of the spine, from the 2nd vertebra in the cervical spine (neck) all the way down to the sacrum (end of the spine). This ligament is located adjacent to the spinal cord. is not easily teared or curved.
(d): Anterior longitudinal ligament is a ligament that runs down the anterior surface of the spine. It traverses all of the vertebral bodies and intervertebral discs on their ventral side. It has a high tensile strength and is resistant to tearing or deformation. It is not easily teared or curved. Therefore, the answer is (b).
Question Keywords:
Neuroendoerine cells, lungs
Candidate Answers: (a) Dendritic cells (b) Type pneumocytes (c) Type II pneumocytes (d) APUD cells
Context:
Neuroendocrine cells are part of the neuroendocrine system. The neuroendocrine cells of the lung make hormones that control the flow of air and blood in the lungs. This may suggest a herniated intervertebral disk through tear in the surrounding annulus fibrosus. The soft, gelatinous nucleus pulposus is forced out through a weakened part of the disk, compressing nerve components of the spinal cord and resulting in back pain and nerve root irritation. This impingement is resulting in paralysis, and should be considered a medical emergency.
(a): Dendritic cells are type of antigen-presenting cell in the immune system that act as messengers between the innate and adaptive immune systems. It is not a type of neuroendocrine cell.
(b): Type pneumocytes are alveolar cells that line the alveolar surface of the lungs and are responsible for gas exchange. It is not type of neuroendocrine cell.
(c): Type II pneumocytes are alveolar cells that secrete surfactant to reduce alveolar surface tension and prevent alveolar collapse. It is not a type of neuroendocrine cell.
(d): APUD cells are a type of neuroendocrine cell that function through amine precursor uptake and decarboxylation. It is accurate say that they a type of neuroendocrine cell.
Therefore, the answer is (d).
Question Keywords: Presence, remote, contamination, water
Candidate Answers: (d) Streptococei (b) Staphalococci (c) Clastridium pertringes (d) Vibrio
Infections that can be spread through water contamination are generally transmitted orally or via fecal matter. (a): Streptococci are spread through direct contact with the nose and throat discharges of an infected individual or with infected skin lesions. Water is not a medium for the spread of streptococei. is not related water contamination.
Context:
(b): Staphylococci is spread by skin contact, like bite or cut, It is not related water contamination. (c): Clostridium perfringens are one of the most common causes of food poisoning. They are environmentally stable and specific to contamination by sewage. Their spread is a indicator of water contamination.
(d): Vibrio species are gram nea-teegatie bacteria that spread through foodborne infection, but they are highly salt tolerant and unable to survive in fresh water. It is not related water contamination.
Continued next page
24

---

Published as a conference paper at ICLR 2024
Table 15 Continued from previous page
Therefore, the answer is (c).
Question Keywords:
True, Mooren's ulcer, 2007, 2013
Candidate Answers: (a) Painless condition (b) Affects cornea (c) Sudden loss of vision (d) Bilateral in majority of cases
Context: Mooren's ulcer is characterized by painful peripheral corneal ulceration of unknown etiology. The disease generally begins with intense limbal inflammation and swelling in the episclera and conjunctiva. Patients often experience severe pain, photophobia, and tearing along with red inflamed eye.
(a): Mooren's 's ulcer is a painful ulceration of the eye. It is not the truth of Mooren's 's uleer.
(b): Mooren's ulcer is characterized by painful peripheral corneal ulceration of unknown etiology. It is the truth of Mooren's 's ulcer.
(c): The symptoms of Mooren's 's ulcer do not include sudden loss of vision. It is not the truth of
Mooren's 's ulcer.
(d): About one third of Mooren's ulcer cases present bilaterally, The proportion is less than half. It is not the majority of cases.
Therefore, the answer (b).
Prompts for general domain datasets. Our prompts on CommonsenseeQ and OpenbookQA are based on al. 2022).
Table 16: Prompts for Commonsense QA
Question Keywords:
fountain pen, people, ink, absorb, pen, hand done, extra, use, fountain
Candidate Answers: (a)shirt pocket (b) calligrapher's 's hand (c) inkwell (d) desk drawer (c) blotter
Context: Fountain pens need to be filled with ink for writing. Extra ink should be absorbed using special tools.
(a): A fountain pen can be conveniently carried in a shirt pocket. It is not associated with the tool to absorb extra ink from fountain pens.
(b): Calligraphers use fountain pens to create stunning handwriting. It not associated with the tool to absorb extra ink from fountain pens.
(c): An inkwell serves as a container for the ink used fountain pen. is not associated with the tool to absorb extra ink from fountain pens.
(d): A fountain pen can be kept safely in a desk drawer. It is not associated with the tool to absorb
extra ink from fountain pens.
(e): Blotters are designed to absorb excess ink from pens. It is the tool for absorbing extra ink. Therefore, the answer is (e).
Question Keywords: fox, forest, walk, look, city
Candidate Answers: (a) pretty flowers (b) hen house (c) natural habitat (d) storybook (e) dense
forest
Context: Foxes are animals that typically live in forests. They walk from the city to the forest to look for their living place.
(a): Pretty flowers are in forests. It is not reason for a fox walking into the forest.
(b): Foxes sometimes prey on chickens in hen houses. It not reason for a fox to walk into the forest.
(c): Forests are the natural habitat of foxes. Foxes walk from city to forest to look for their natural
habitat. (d): Forests and foxes common subjects in storybooks. It is not reason for fox walking to the forest.
(e): Dense forest is type or category of forests characterized by having high density of trees and vegetation. is a type of forest.
Therefore, the answer is (c) or (e).
Question Keywords: grape, put, check
Continued on next page
25

---

Published as a conference paper at ICLR 2024
Table 16 Continued from previous page
Candidate Answers: (a) mouth (b) grocery cart (c) super market (d) fruit basket (c) fruit market
Context:
Grapes need to be put into place for checking out.
(a): Grapes can be eaten by mouth. It is not place to put grapes for checking out.
(b): Grapes can be brought during grocery shopping and people put groceries into grocery carts
hefore checking out. It could be a potential place put grape.
(c): Super markets sell grapes. It is not a place to put grapes for checking out.
(d): Fruit markets sell grapes. It is not place to put grapes for checking out.
(e): Fruit baskets are often used as gifts to hold and present a variety of fresh grapes. It is not a place put grapes for checking out. Therefore, the answer is (b).
Question Keywords:
drawstring bag, head, woman, bag, drawstring, check, baggage Candidate Answers: (a) garbage can (b) military (c) jewelry store (d) safe (e) airport
Context: A woman can check baggage such as drawstring bag at the check-in counter. (a): garbage can container that specifically designed to hold and contain trash or waste materials. It is not related to the context.
(b): Military refers to the armed forces of a country, which is responsible for defending the nation and its interests against external threats. It is not a place where woman can check bags.
(c): Jewelry stores sell jewelry. It is not a typical place to check baggage.
(d): Check baggage could keep the bag safe. A woman can check her drawstring bag keep the bag safe.
(e): Airport place where the woman can check her drawstring bag baggage the check-in- counter. It is common to check baggage in airport. Therefore, the answer is (e).
Question Keywords:
cable, entertainment, home, require, equipment
Candidate Answers: (a) radio shack (b) substation (c) television (d) cabinet (e) desk Context: A cable transmits electricity or information and data to home entertainment equipment that requires electricity.
(a): Radio Shack is a retailer that sells cable. It is not a home entertainment equipment used cable. (b): Cables used to transmit electrical energy between substations and other parts of the electrical power system. It is not a home entertainment equipment used cable. (c): Television is a type of home electric entertainment equipment that requires cable. It home entertainment equipment used cable.
(d): Cabinet is place to store cable. is a home entertainment equipment used cable.
(e): Desk with built-in cable management features can help keep cables tidy. It is not a home entertainment equipment used cable. Therefore, the answer is (c).
Question Keywords: people, populate, might, may, sammy, go
Candidate Answers: (a) populated areas (b) race track (c) desert (d) apartment (e) roadblock
Context:
People may like go to places where people populate together. (a): Populated areas are locations where people gather and live in close proximity to each other. could be a place where
people populate together. (b): Deserts are inhospitable environments for people. It is not place where people populate together. (c): People go to race competitions on the race track. It could place where people populate together. (d): Apartments serve living spaces for people. It is not place where people populate together. (e): Roadblocks are structures set up to restrict or regulate the movement of people and vehicles. It is not place where people populate together. Therefore, the answer is (a) or (c).
Question Keywords: highway, maps, replace, street, google, map, highway, gps, service
Candidate Answers: (a) united states (b) mexico (c) countryside (d) atlas (e) oceans
Context: Google Maps and GPS services have replaced traditional physical maps for navigating highways and streets.
(a): People in the United States use Google Maps and GPS services to navigate highways and streets. It is not the tool that GPS replaced with.
Continued on next page
26

---

Published as a conference paper at ICLR 2024
Table 16 Continued from previous page
(b): People in Mexico use Google Maps and GPS services to navigate highways and streets. It is not the tool that GPS replaced with.
(c): Google Maps and GPS services cover the countryside. It is not the tool that GPS replaced with.
(d): Google Maps and GPS services have replaced traditional physical maps for navigating highways and streets. Atlases are examples of traditional physical maps. It is the tool that GPS replaced with. (e): Google Maps and GPS services cover the oceans and are commonly used in marine navigation. is not the tool that GPS replaced with. Therefore, the answer is (d).
Table 17: Prompts for Openbook QA
Question Keywords: acid, enviromment, aquatic, rain, effect, acid rain Candidate Answers: (a) decrease plant life (b) increase in fish population (c) increase in plant growth (d) cleaner and clearer water
Context:
The acid rain is type of rain that has an acidic effect due to the presence of acid in the atmosphere. Acid rain is harmful to the environment, especially aquatic life.Thee acid in the rain can have a negative effect on the water quality of aquatic environments.
(a): Acid rain can have a negative effect plant life. The acid in the rain can damage plant cells and cause a decrease in plant growth, leading to a decrease in plant life. is likely to have a decrease in plant life by acid rain.
(b): Acid rain can have a harmful effect on aquatic life, including fish. The acid the water can make it difficult for fish to breathe and can harm their reproductive systems. is not likely have an increase in fish population by acid rain.
(c): As previously mentioned, the acid in the rain can damage plant cells and cause a decrease in plant growth. It is not possible to have an increase in plant growth.
(d): Acid rain can have harmful effect on water quality, making it more acidic and harmful to aquatic life. It is not possible to have cleaner and clearer water. Therefore, the answer (a).
Question Keywords:
moon, surface
Candidate Answers: (a) is smooth on the entire surface (b) contains large cavities cause by explosions (c) contains an internal core of cheese (d) is filled with lakes
Context: The moon is natural satellite that orbits around the Earth. Its surface is covered with dead volcanoes, impact craters, and lava flows, some visible to the unaided stargazer.
(a): The moon has mountains, craters, and other features caused by impacts from meteoroids and asteroids. It is not entirely smooth on the surface.
(b): Impact craters are formed when an asteroid craters, each of which was formed when an asteroid or comet collided with the Moon's 's surface. The moon's 's surface contains large cavities caused by explosions from impacts.
(c): The core is largely composed of iron and some nickel. The inner core is solid mass about 480 km in diameter. It does not contain an internal core of cheese. (d): The moon has lunar maria composed of basalt formed from surface lava flows that later congealed. It is not filled with lakes. Therefore, the answer (b).
Question Keywords: car, approach, night
Candidate Answers: (a) the headlights become more intense (b) the headlights recede into the dark (c) the headlights remain at a constant (d) the headlights turn off
Context: Headlights of a
source of light. As car approaches, the source of light becomes closer, and that source will appear brighter.
(a): As the car becomes closer, the distance to the source of light decreases. The headlights become brighter and more intense. This is a possible phenomenon.
(b): If the source does not change and the headlights are closer, the headlights cannot become dimmer. This is not a commonsense relation.
Continued next page
27

---

Published as a conference paper at ICLR 2024
Table 17 Continued from previous page
(c): If the distance to the source of light changes, the brightness of headlights will change. It is not able to remain constant.
(d): Turning off the headlights would cause the driver to be driving in complete darkness, which is dangerous and can lead to accidents. is not a reasonable condition. Therefore, the answer is (a).
Question Keywords:
change, easter, weather change, weather, christmas Candidate Answers: (a) the air may chill (b) the ground may freeze (c) the plants may die (d) the ground may warm
Context: In the US, Christmas falls in the winter season, while Easter arrives at the beginning of spring.
(a): The air becomes chill as temperature drops. The temperature commonly increases from winter to spring. It is not likely scenario.
(b): During winter, the ground usually freezes, whereas in spring, it does not. It is not probable scenario.
(c): Extreme cold hot weather can cause plants to die. The beginning of spring provides suitable weather conditions for plants to grow. It is not common to have plants die. (d): As winter transitions into spring, the weather becomes warmer. The temperature of the ground is influenced by the weather. Therefore, the answer is (d).
Question Keywords: heat, recipe, moisture, good, ocean
Candidate Answers: (a) a violent storm (b) violent sea animals (c) condensation (d) inland storms
Context: The ocean, vast body of water that covers large portion of the Earth's surface, serves as a source of heat and moisture.
(a): The heat and moisture present in the ocean can create ideal conditions for a hurricane or typhoon. Hurricane and typhoon are violent storms.
(b): Violent sea animals are not related to heat and moisture the ocean. is not a likely choice.
(c): Condensation is the process by which water vapor becomes liquid, which is the reverse of evaporation. This can happen one of two ways: either the air is cooled to its dew point or becomes saturated with water vapor that it cannot hold any more water. It is not likely to occur in hot conditions.
(d): Although heat and moisture can cause inland storms, they are not directly related the ocean. It not a likely choice.
Therefore, the answer (a).
Question Keywords: hummingbird, take
Candidate Answers: (a) bees (b) energy (c) pollen (d) honey
Context: Hummingbirds dip their long bills into flowers to drink nectar to get energy.
(a): Hummingbirds and bees are both attracted to the sweet nectar produced by flowers, but bees extract the nectar from the base of the flowers, while hummingbirds dip their long bills into the flowers to drink the nectar and obtain energy. No relationship can be found. (b): Hummingbirds obtain energy by getting nectar from flowers through dipping their long bills into the flowers. No relationship can be found.
(c): When hummingbirds drink nectar, they also inadvertently take grains of pollen which stick to their feathers and bills, and get carried to the next flower they visit. No relationship can be found. (d): Hummingbirds do not produce or consume honey. This fact is unrelated to their method of obtaining energy by drinking nectar from flowers. Therefore, the answer is None.
Question Keywords:
responsible, sun
Candidate Answers: (a) puppies learning new tricks (b) children growing up and getting old (c) flowers wilting in vase (d) plants sprouting, blooming and wilting
Context: The sun is the source of energy for physical cycles on Earth.
Continued on next page
28

---

Published as a conference paper at ICLR 2024
Table 17 Continued from previous page
(a): Puppies learning new tricks involves the acquisition and processing of information, which is essential for the puppics to learn and adapt to their environment. It not directly related the effect
of the sun.
(b): Children grow up and age over time. The sun is not directly responsible for the passage time itself. It is directly related to the effect of the sun.
(c): Flowers in vase become wilting because they are cut from their original source of nutrients and water and are no longer able receive the essential nourishment they need to stay healthy and vibrant. It is not directly related to the effect of the sun.
(d): Plants need sunlight photosynthesize and grow, and the sun't heat and light play crucial role
in the process of plant growth and decay, It is the thing that the sun is responsible for. Therefore, the answer (d).
29


Reviews:
[{"review_id": "9sLDVBo2SI", "reviewer": "Reviewer_EBQC", "summary": "In situations where text data is subject to privacy protection constraints, this paper designs a small-scale language model to perform diagnoses of diseases. Utilizing the rich prior medical knowledge in LLM, the approach involves generating a medical knowledge-intensive context using privacy-protected text. This generated context, along with key terms extracted from the text and questions, is then input into the SLM, which is fine-tuned during training. Experiments across multiple datasets demonstrate that this fine-tuning process effectively enhances the accuracy of the diagnostic model.", "questions": "1. The experimental datasets in this paper are all question-and-answer test datasets, and whether the methods of this paper are applicable to medical report datasets requires additional experimentation. This is because in medical reports, how to generate high-quality questions using other LLM interfaces is a question worth studying.\n\n2. Large language models provide additional domain knowledge, but in the context of specific medical tasks, will the direct transfer of knowledge from LLMs to SLMs lead to incorrect information leakage into SLMs? How can we ensure that LLMs only enhance information relevant to the current medical issue without introducing additional errors or irrelevant information? This is a very important issue in the medical field, as it directly relates to patient diagnosis.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper focuses on a very important research topic in the field of medicine: how to effectively extract more useful information from incomplete text under the conditions of privacy protection. The author has made full use of the domain knowledge in LLM to effectively fine-tune the SLM, which ensures that the lightweight models can achieve high accuracy.\n\n2. This paper presents rich and comprehensive experiments. Beyond basic decision-making tasks, it also explores solutions for few-shot experiments and out-of-distribution (OOD) model generalization using the methods discussed in this paper.\n\n3. This paper fully utilizes the rich domain knowledge in LLMs to expand the knowledge base of medical reports, achieving excellent diagnostic accuracy even while ensuring privacy protection.", "weaknesses": "1. The contribution of this paper to the algorithm and the significance of the clinical problems it addresses seem not to be very high.\n\n2. The main work of this paper appears more as an engineering problem, transferring domain knowledge from LLMs to SLMs. From the perspective of algorithmic contribution, there seems to be some room for improvement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "78lQNasrZS", "reviewer": "Reviewer_TtE2", "summary": "The paper studied medical QA problems by incorporating large language models (LLMs) to assist small-language models (SLMs). To protect the private information in the data, the authors propose to first extract keywords and then use the keywords to query LLMs for intermediate content which can be used for SLMs to enhance prediction accuracy.", "questions": "* There is no clear motivation to see SLM for prediction. Although the authors mention that ChatGPT requires access to data, it is essentially ignored that open-source LLMs, for example, Llama, can be used. Is there any evidence for the large gap between open-source LLMs and ChatGPT on the concerned medical tasks?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. (originality) The proposed method is novel by extracting keywords and privately incorporating LLM for SLM-based predictions.\n2. (clarity) Overall, the paper is fair in presentation. The demonstrations of synthetic medical data with private information and extracted keywords are helpful for understanding the concepts.\n3. (significance) Versus the compared baselines, the proposed methods significantly improve the prediction accuracy on three medical QA tasks.\n4. (quality) The authors thoroughly evaluate the performance of the proposed method.", "weaknesses": "1. (Clarity) There is no specific definition of the private information. From Figure 1, it seems that privacy definition is restricted to private identifiable information (PII). The authors should clarify the scope of privacy risks. Importantly, the proposed method cannot address general private information leakage that is considered by strict formulations like differential privacy.\n2. (Quality) The evaluation of privacy is not strict. \n  - Risks: It is possible that the keyword extraction includes private identifiable information (PII), for instance, names and dates as shown in Figure 1. There is no theoretical guarantee for privacy protection or empirical evaluation of the leakage rates of such PII.\n  - Metric: The authors used the privacy budget for quantifying privacy risks:  the ratio of the number of words provided to the LLM to the total words in the original question. However, I doubt if the metric can imply some privacy risks. There essentially lacks an intuitive explanation of the relationship between the privacy budget and privacy risks.\n3. (Motivation) As the authors said, SLM presents a large gap compared to LLMs and thus there is no clear motivation to use SLM for prediction. Although the authors mention that ChatGPT requires access to data, it is essentially ignored that open-source LLMs, for example, Llama, can be used. In the paper, there is no referred evidence for the large gap between open-source LLMs and ChatGPT on the concerned medical tasks. Thus, I strongly doubt if the motivation of the paper can hold.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x343VevwM7", "reviewer": "Reviewer_gXvF", "summary": "This paper tried to improve the performance of small medical language models by introducing knowledge from large language models, which keeps the privacy of clinical text when using large language models.  The proposed method uses keywords instead of full raw text to generate initial evidence from LLM and feed the evidence to small language model.", "questions": "By adding the LLM-generated content, will the new concatenated input be too long and out of the word window in SLM? How do you deal with the long content problem?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Privacy-preserving is an essential and common need when using LLM in clinical text. This paper tried to solve this problem by using keywords instead of raw text, the idea is novel and experiments demonstrated the effectiveness of this approach.", "weaknesses": "1. As this research utilized a named entity recognition model to extract keywords, it is possible that the NER model can extract privacy information such as patient names. Is there any filtering or postprocessing step to avoid that? In addition, it is not guaranteed that NER system will never extract sensitive patient information; for example, if the NER system incorrectly extracts a patient's address as a symptom, then the address may be leaked to LLM. Although it is very rare, it is still necessary to comment on this. \n2. As the LLM already provides a preliminary decision, I am curious about the performance if we only feed the preliminary decision from LLM to SLM. It is worth knowing which part of the LLM-generated information improves the SLM most. \n3. The related work section need to discuss more LLM application in the clinical area, especially the knowledge-enhanced LLM in clinical settings. For example, paper \"Qualifying Chinese Medical Licensing Examination with Knowledge Enhanced Generative Pre-training Model.\" also utilized external knowledge for clinical questions. \n4. By adding the LLM-generated content, will the new concatenated input be too long and out of the word window in SLM? How do you deal with the long content problem?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DT9YN4xa8A", "reviewer": "Reviewer_JSi7", "summary": "This article discusses a method to improve the application of SLM in the medical field, utilizing LLM's medical proficiency to boost SLM performance in medical tasks under privacy-restricted scenarios which has important social significance. The method was tested on MedQA, HEADQA, MedMCQA, and MMLU-professional medicine datasets, showing some improvements over existing methods. Additionally, the authors compared results across different sizes of training sets.", "questions": "please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "see summary", "weaknesses": "1). Imprecise example of Privacy Protection.\nThe example in Figure 1 indicates that personal privacy issues are only present in the first sentence, and the key words \"man\" and \"admitted\" in that sentence have almost no impact on the subsequent content. Could it then be possible to simply delete the first sentence to achieve privacy protection, as extracting key words here does not seem to play a significant role.\n\n2). Privacy Protection as an Innovation Point\nRegarding the extraction of key words for privacy protection, the paper uses a medical NER model proposed by Neumann et al in 2019. We suggest further improvement of this model, for example, considering age as a crucial keyword for certain diseases and extracting it as necessary to better enrich the innovative aspects of the paper.\n\n3). Ambiguity of Symbols in Annotations\nAnnotation 13 on page 8 only appears in the content of the article but is not explained.\n\n4) The overall innovation of the methodology needs improvement, as the majority of the content relies on existing methods, such as the medical NER (Named Entity Recognition) model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)