D iversity M ed QA : A Benchmark for Assessing Demographic Biases in Medical Diagnosis using Large Language Models (2024)

Abstract: As large language models (LLMs) gain traction in healthcare, concerns about their susceptibility to demographic biases are growing. We introduce DiversityMedQA, a novel benchmark designed to assess LLM responses to medical queries across diverse patient demographics, such as gender and ethnicity. By perturbing questions from the MedQA dataset, which comprises of medical board exam questions, we created a benchmark that captures the nuanced differences in medical diagnosis across varying patient profiles. To ensure that our perturbations did not alter the clinical outcomes, we implemented a filtering strategy to validate each perturbation, so that any performance discrepancies would be indicative of bias. Our findings reveal notable discrepancies in model performance when tested against these demographic variations. By releasing DiversityMedQA, we provide a resource for evaluating and mitigating demographic bias in LLM medical diagnoses.

## introduction large language models (llms) are increasingly used in various domains, including medicine, due to their ability to process and generate humanlike text @xcite . in healthcare, llms can enhance efficiency and consistency in diagnosing diseases, providing treatments, and disseminating medical information, which is particularly important given current staffing shortages and rising demand @xcite @xcite . however, integrating llms into healthcare requires addressing potential biases to avoid exacerbating existing disparities. most research in the field has focused on the general capabilities and performance of llms in medical applications @xcite . some studies have specifically examined biases related to race and gender in medical education and practice, emphasizing the risk of existing disparities in healthcare @xcite . biases in medical research, including methodological flaws and conflicts of interest, further bar quality studies and effective policy-making @xcite @xcite . other research has explored methodologies to benchmark diagnostic accuracy and bias in medical llms, such as using medqa derived from medical board exams @xcite . however, there currently isn't a widely accepted dataset that effectively tests for bias in medical question and answering for gender and ethnicity. this lack of a standardized benchmark limits the ability to systematically evaluate and compare the performance of different llms in mitigating biases, particularly in the context of medical diagnostics. our research introduces a benchmark for measuring bias in llms used for medical diagnoses. using medqa with perturbed data points reflecting different demographics, we assess gpt-3.5, gpt-4.0, gpt-4o, llama3-8b, and gemini models' accuracy. this augmentation provides insights into llm performance across demographics, highlighting areas for improvement to ensure equitable healthcare outcomes. bias in llms language models (lms) are prone to bias due to their training data, leading to less accurate outputs. fairpair evaluates bias by generating alternate sentences from common sents, using sentiment and token dissimilarity scoring to quantify gender bias @xcite . gender bias in lms has also been assessed using benchmarks like winobias, revealing tendencies to conform to stereotypical gender roles rather than reflecting actual gender distributions @xcite . ## conclusion our study identified significant biases in llama3-8b, emphasizing the need for further model development before medical deployment. the new gpt-4 turbo and gpt-4o models outperformed others, suggesting improvements in addressing surface-level biases with each technological advancement. we introduced the diversitymedqa dataset to measure bias in medical diagnoses. by adding perturbed demographic data to medqa, we evaluated gpt-3.5 turbo, gpt-4 turbo, gpt-4o, gemini 1.5 flash, and llama3-8b. we also implemented a filtering method using llm prompting to assess gender and ethnicity biases in medical questions. this method, applied with gpt-4 turbo, can extend to other models and datasets. although the newer models performed well, the significant gender and ethnicity bias in llama3-8b was the most critical finding, as other models generally showed no bias in z tests. outputs from gpt-3.5 turbo and llama3-8b also displayed bias, emphasizing the need for further alignment improvements to ensure equitable healthcare outcomes.