Language Models are Surprisingly Fragile to Drug Names in Biomedical
  Benchmarks
Authors: name: Jack Gallifant; name: Shan Chen; name: Pedro Moreira; name: Nikolaj Munch; name: Mingye Gao; name: Jackson Pond; name: Leo Anthony Celi; name: Hugo Aerts; name: Thomas Hartvigsen; name: Danielle Bitterman
Published: 2024-06-17 20:09:24

Abstract: Medical knowledge is context-dependent and requires consistent reasoning
across various natural language expressions of semantically equivalent phrases.
This is particularly crucial for drug names, where patients often use brand
names like Advil or Tylenol instead of their generic equivalents. To study
this, we create a new robustness dataset, RABBITS, to evaluate performance
differences on medical benchmarks after swapping brand and generic drug names
using physician expert annotations.
  We assess both open-source and API-based LLMs on MedQA and MedMCQA, revealing
a consistent performance drop ranging from 1-10\%. Furthermore, we identify a
potential source of this fragility as the contamination of test data in widely
used pre-training datasets. All code is accessible at
https://github.com/BittermanLab/RABBITS, and a HuggingFace leaderboard is
available at https://huggingface.co/spaces/AIM-Harvard/rabbits-leaderboard.