Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks (2024)

Abstract: Medical knowledge is context-dependent and requires consistent reasoning across various natural language expressions of semantically equivalent phrases. This is particularly crucial for drug names, where patients often use brand names like Advil or Tylenol instead of their generic equivalents. To study this, we create a new robustness dataset, RABBITS , to evaluate performance differences on medical benchmarks after swapping brand and generic drug names using physician expert annotations.We assess both open-source and API-based LLMs on MedQA and MedMCQA, revealing a consistent performance drop ranging from 1-10%. Furthermore, we identify a potential source of this fragility as the contamination of test data in widely used pre-training datasets.

## introduction large language models (llms) are poised to transform medicine by providing data processing and decision support capabilities @xcite . however, the medical deployment of llms demands high accuracy and reliability, as errors can result in severe health consequences @xcite @xcite . a key challenge is the synonymy and context-specific nature of medical language; for instance, patients might use brand names like advil or tylenol instead of pharmaceutically equivalent generic terms such as ibuprofen or acetaminophen. llms must, therefore, be able to provide consistent and accurate advice in the face of this variability. fluctuations could lead to risks like medical misinformation, medication errors due to incorrect medication advice, and biases toward or against proprietary products. our study investigates the effects of substituting drug names-from brand to generic and vice versa-on llm performance. building on the need for robustness in medical llm applications, numerous efforts have developed knowledge benchmarks @xcite @xcite @xcite . yet, these initiatives primarily tackle general language tasks and often neglect the unique challenges of medical terminology in real-world settings. there is an unmet need to overcome this research gap, as the variability in medical language implies that conventional robustness evaluations might not sufficiently cater to specialized healthcare demands. a key reason for this gap is the lack of publicly available, expert-annotated datasets specific to the healthcare domain. to address this issue, our work leverages existing medical benchmarks and employs physician expert annotators to substitute brand names with their generic counterparts and vice versa. our findings reveal a surprising drop in the performance of llms on common medical benchmarks when the drug names are swapped from generic to brand names: 4% drop in accuracy on average. this is concerning given that patients commonly use brand names and are less likely to spot errors, especially given existing misconceptions that brand drugs are superior to equivalent generics @xcite . furthermore, we identify a potential source for this fragility: open pretraining datasets contain substantial amounts of benchmark test data. our research introduces a novel category of robustness evaluation centered on drug name interchangeability. we present rabbits (robust assessment of biomedical benchmarks involving drug term substitutions for language models) a specialized dataset and leaderboard to aid in evaluating llm performance in healthcare. specifically, our study combines and modifies select questions from the medmcqa @xcite and medqa @xcite benchmarks to: ## conclusion we find decreased performance on common medical benchmarks when using different names for the same drug, despite llms' ability to match these names, and that these trends scale with llm size. this suggests that llm performance may be driven by memorization and not reasoning ability. rabbits underscores the importance of dataset contamination and model robustness evaluations, particularly in the medical domain. future research should refine strategies and explore new methods for robustness and fairness evaluation.