Open (Clinical) LLM s are Sensitive to Instruction Phrasings (2024)

Abstract: Instruction-tuned Large Language Models (LLMs) can perform a wide range of tasks given natural language instructions to do so, but they are sensitive to how such instructions are phrased. This issue is especially concerning in healthcare, as clinicians are unlikely to be experienced prompt engineers and the potential consequences of inaccurate outputs are heightened in this domain. This raises a practical question: How robust are instruction-tuned LLMs to natural variations in the instructions provided for clinical NLP tasks? We collect prompts from medical doctors across a range of tasks and quantify the sensitivity of seven LLMs—some general, others specialized—to natural (i.e., non-adversarial) instruction phrasings. We find that performance varies substantially across all models, and that—perhaps surprisingly—domain-specific models explicitly trained on clinical data are especially brittle, compared to their general domain counterparts. Further, arbitrary phrasing differences can affect fairness, e.g., valid but distinct instructions for mortality prediction yield a range both in overall performance, and in terms of differences between demographic groups.

## introduction modern llms-e.g. gpt-3.5+ @xcite , the flan series @xcite , alpaca @xcite , mistral @xcite -can execute arbitrary tasks zero-shot, i.e., provided with only instructions rather than explicit training examples. llms have also shown promising improvements in performance on classification and information extraction (ie) tasks, such as named entity recognition @xcite and relation extraction @xcite @xcite in both general and specialized domains like biomedical and * equal contribution scientific literature @xcite @xcite . however, prior work has shown that llms do not "understand" prompts @xcite and are sensitive to the particular phrasings of instructions @xcite . domain experts in specialized domains such as medicine are especially likely to interact with models by providing instructions (i.e., in zero-shot settings), and are unlikely to be talented prompt engineers. for instance, a clinician might task a model to "extract and summarize the findings of the patient's last x-ray", or ask "when did the patient last receive a painkiller?". it is unrealistic to finetune models for every possible such task; hence the appeal of models responsive to arbitrary prompts. a downside, however, is that a clinician's particular phrasing may dramatically affect model performance (figure 1 ). such unpredictability is especially troublesome in healthcare, where poor performance might ultimately impact patient health. in this work we ask: how sensitive are llmsgeneral and domain-specific-to plausible instruction phrasing variations for clinical tasks? our analysis deepens prior work on robustness by focusing on the clinical domain; this is important both due to the higher stakes and because clinical notes differ qualitatively from general domain text. for example, notes in ehr often contain grammatical errors ("pt complains of headache, and feel dizzy."); abbreviations not defined in context ("pt" could be "patient" or "prothrombin time"), and; domain-specific jargon ("edema", "diuretic"). therefore, one of the key aspects we consider is the domain-specificity of models. are clinical llms more (or less) robust to different valid instruction phrasings written by doctors, compared to their general domain counterparts? to assess this, we evaluate recently released llm variants trained on synthetic datasets comprising automatically generated clinical notes @xcite , and medical dialogue from case reports found in biomedical literature @xcite . we find that performance varies substantially given alternative instruction phrasings for both general and clinical llms. figure 2 shows the distribution of deltas between the best and worst performing prompts across a set of clinical classification and information extraction tasks. finally, we investigate how instruction phrasings impact the fairness of predictions, by which here we mean observed differences in performance between demographic subgroups. the degree to which llms might perpetuate and exaggerate such disparities in clinical use is a topic of active research @xcite @xcite . here we contribute to this by investigating the interaction between prompt phrasings and fairness. we find significant performance differences (up to 0.35 absolute difference in auroc) in a mortality prediction task from mimic-iii between white and non-white subgroups and also a significant disparity between male and female patients (up to 0.19 absolute difference in auroc). to facilitate future research in this direction, we release our code and prompts foot_0 . ## experimental framework our experimental setup is intended to quantify the robustness of llms to natural variations in instructional phrasings for clinical tasks. we considered a set of ten clinical classification tasks and six information extraction tasks drawn from mimic-iii @xcite and prior i2b2 and n2c2 challenges, foot_1 summarized in ## results we present our main results for mortality prediction and drug extraction in figure 3 - results for the other classification and information extraction tasks can be found in appendix a.2, figures 12 and 13, respectively. most models show significant variability in performance for alternative but semantically equivalent instructions in both classification and extraction tasks. to further examine these observed disparities, we plotted the distri-0 0.4 0.5 0.6 0.7 0.8 0.9 1.0 auroc mistral (7b) llama-2 (13b) llama-2 (7b) alpaca (7b) clin-camel (13b) asclepius (7b) medalpaca (7b) mimic mortality prediction 0 0.2 0.4 0.6 0.8 f1 clinical models general models drug extraction bution of deltas between the best and worst performing prompts for each task in figure 2 . we see that performance deltas can go up to 0.6 absolute auroc points for classification tasks and up to 0.4 absolute f1 points for extraction tasks. in the mortality prediction task, we find that llama 2 (13b) outperforms all other models, including the domain-specific ones (figure 3 ). however, for the other classification tasks, mistral yields the best results often outperforming the larger models whilst exhibiting less variance (figure 12 ). regarding the clinical models, we observe that asclepius consistently attains the best performance in classification tasks albeit with comparable variance. in the drug extraction task, llama 2 (7b) attains the best results on average but with comparable variance to other general llms. however, the results for clinical models are mixed: while clinical camel can achieve the highest performance given the best prompt, it also has the highest variance and lowest median performance. medal-paca comes close to clinical camel in the best case scenario but with less variance and better median performance. asclepius has a median performance similar to that of medalpaca but with a much lower variance. we observe similar trends for the other information extraction tasks: llama 2 (7b) consistently outperforms other general llms with similar variance, whereas none of the clinical models is clearly superior across tasks -however, asclepius seems to have the least variance overall. to better understand the differences between the general domain and clinical llms, we compared their average performance given the best, median and worst prompts. figures 4 and 5 show the results per model averaged across all classification and extraction tasks, respectively. surprisingly, we find that general domain models outperform their domain-specific counterparts -with the exception of alpaca which performs poorly across all tasks. again we observe that even though clin-ical camel can outperform its general domain analog in extraction tasks given the best prompt, it also shows more variance and much lower performance in the worst case. finally, we investigated whether the observed performance variability can be explained by individual differences between experts in prior experience with llms or aptitude in writing effective instructions. to assess this, we measured the performance deltas between each prompt and the median prompt for each classification and extraction task. figure 6 shows the results for llama 2 (7b) and results for other models can be found in appendix a.2, figures 14 and 15. we find that there are indeed significant differences at the individual level, both in terms of variance and overall performance, particularly for classification tasks. only roughly half the users can (somewhat) consistently beat the median performance across tasks. we also note these differences can not be solely explained by prior experience with llms -some novice users are able to consistently write more effective instructions as compared to other experienced users. however, one caveat is that this prior experience is most likely with larger commercial models which may be more robust to instruction variations. ## conclusions this paper presents a large-scale evaluation of instruction-tuned open-source llms for clinical classification and information extraction tasks on clinical notes (from ehr). we specifically focus on model robustness to natural differences in prompts written by medical professionals. we recruited 12 practitioners with different professional and demographic backgrounds, medical specialties, and years of experience to write prompts for 16 clinical tasks spanning binary classification, outcome prediction, and information extraction. there are a few main generalizable takeaways relevant to machine learning in healthcare in this work. first, the performance llms realize on the same clinical task varies substantially across prompts written by different domain experts, and this holds across all models. second, the domainspecific (clinical) models we evaluated perform, in general, worse than their general domain counterparts. third, prompt variations have concerning implications for fairness -we find that alternative prompts yield different levels of fairness. based on these findings, we recommend that practitioners exercise caution when using instruction-tuned llms for high stakes clinical tasks which may ultimately impact patient health. crucially, clinicians using llms should be made aware that subtle, plausible variations in phrasings may yield quite different outputs. beyond healthcare, this work enriches our understanding of (the lack of) llm robustness and-we hope-will motivate research into new methods to improve models in this respect.