One Size Fits None: Rethinking Fairness in Medical AI (2025)

Abstract: Machine learning (ML) models are increasingly used to support clinical decision-making. However, real-world medical datasets are often noisy, incomplete, and imbalanced, leading to performance disparities across patient subgroups. These differences raise fairness concerns, particularly when they reinforce existing disadvantages for marginalized groups. In this work, we analyze several medical prediction tasks and demonstrate how model performance varies with patient characteristics. While ML models may demonstrate good overall performance, we argue that subgroup-level evaluation is essential before integrating them into clinical workflows. By conducting a performance analysis at the subgroup level, differences can be clearly identified—allowing, on the one hand, for performance disparities to be considered in clinical practice, and on the other hand, for these insights to inform the responsible development of more effective models. Thereby, our work contributes to a practical discussion around the subgroup-sensitive development and deployment of medical ML models and the interconnectedness of fairness and transparency.

## introduction medical machine learning (ml) models are trained on datasets containing diverse patient characteristics. however, when certain subgroups are overor underrepresented, models may show unequal performance, raising fairness concerns. addressing such disparities requires evaluation across subgroups-ideally with an intersectional perspective that considers overlapping dimensions of disadvantage @xcite . this leads to the central question: how should we address subgroup performance disparities in the context of fairness in medical ml? fairness is a multifaceted concept that frequently arises in the context of machine learning systems. a common definition describes fairness in decisionmaking as the 'absence of any prejudice or favoritism toward an individual or group based on their inherent or acquired characteristics' @xcite . therefore, an ml system can be considered unfair if, despite the goal of achieving equally good performance across different subgroups, it exhibits substantial performance disparities. those disparities often result from bias, for example through biased training data (data bias) or a biased algorithm itself (algorithmic bias). both terms encompass various subtypes of bias, such as minority bias, missing data bias or cohort bias that can lead to a poorer performance for certain subgroups @xcite . in machine learning, representation and performance disparities have been documented across modalities. for instance, large language models used in clinical settings may perpetuate stereotypes or marginalize certain identities when sociodemographic diversity is absent in training data @xcite . similar issues arise in structured ehr modeling, where label noise and skewed sampling exacerbate subgroup-specific errors @xcite . to address these challenges, prior work has taken different approaches. some studies aim to improve dataset diversity or subgroup visibility in clinical training data @xcite . others propose fairnessaware optimization objectives or subgroup-specific tuning to reduce performance gaps @xcite . the importance of documentation and benchmarking has also been emphasized-especially in clinical imaging and foundation models-through standardized evaluation protocols across sensitive attributes @xcite . our work contributes to this growing field by offering a structured analysis of subgroup variation across three real-world multimodal medical predic-tion tasks: mortality, triage, and graft failure, and advocating for routine reporting and subgroup validation as an integral part of the ethical assessment of medical ml model evaluation. ## experiment we conduct our experiments on three multimodal clinical datasets, each containing textual data (e.g., clinical notes), structured static data (e.g., demographics), and, in two cases, time-series data (e.g., vital signs). all tasks involve patient-level predictions in distinct clinical settings. mortality based on the mimic-iii @xcite dataset from a us intensive care unit, this task involves predicting in-hospital mortality after the first 48 hours of admission @xcite . data includes demographics, time-series vitals, and admission notes. it is framed as a binary classification and evaluated using auc-roc (roc) and auprc (prc). graft failure this dataset comes from a german transplant center and includes structured data (e.g., demographics, comorbidities), time-series labs and vitals, and clinical texts. the task is to predict graft failure within 360 days of each visit, using binary classification with roc and auprc as metrics. triage this dataset contains semi-structured ambulance records from a german emergency department, including structured features (e.g., vitals, pain score, glasgow coma scale) and short text notes, describing the accident and situation of patient. the task is to classify patient urgency according to the manchester triage system (mts), a multi-class classification problem evaluated using precision, recall, and f1 score. ## discussion our results highlight the variability of ml model performance across patient subgroups on different multimodal datasets in multiple tasks. while overall metrics may suggest good performance, a closer look reveals that models can underperform for specific subgroups, such as older patients, individuals from certain ethnic groups, but also patients with lower data quality or a particular transplant. this poses a potential risk, particularly in clinical decision-making, where complex and difficult decisions must be made for vulnerable patient populations. 1 corresponding confidence intervals as well as further details about the significance test, are reported in the appendix. as we have shown, fairness can be understood as the requirement that different subgroups should exhibit similar performance and that the model should not 'favor' any particular subgroup. however, in order to be fair and to pursue the goal of achieving equal performance across all subgroups, transparency is essential. first, it must be recognized that the model performs differently across different subgroups. with this knowledge of the subgroupspecific performance disparities a particular model can still be used-especially since, in many realworld scenarios, achieving fairness in the sense of identical performance for all subgroups may not be feasible. but for that to be responsible, it is important that these models are accompanied by documentation similar to an 'information leaflet' or a 'package insert' @xcite that includes subgroup-level performance metrics, an overview of the training data distribution, and disclaimers when certain subgroups are likely underrepresented. the eu ai act even demands a respective documentation for high-risk ai systems (european union, 2024). to this end, best practices and standards for reporting subgroup performance need to be developed. such information can then guide clinicians in interpreting predictions, managing uncertainty, and identifying when to override or ignore model outputs. at the same time, this transparency must not become a substitute for fairness, allowing largely unfair and biased models to be used uncritically and thereby reinforcing existing inequalities. rather, transparency and fairness must be closely intertwined, with the recognition of poorer performance for certain subgroups prompting targeted efforts to improve outcomes specifically for those groups. ultimately, the goal should not be to prevent the use of models that do not perform equally for all possible subgroups, but to ensure they are used with awareness, and that this insight is used to improve the model specifically for those disadvantaged groups. a biased model with clear warnings and transparent evaluation may still bring benefit in clinical practice, especially in settings where no decision support exists otherwise. however, it is precisely this transparency enabled by subgroup analysis that can help further improve the model or even develop a new model specifically for those subgroups that are otherwise underrepresented. finally, the knowledge about surprising performance discrepancies across patient subgroups can also trigger further research, as the underlying causes could also be medical rather than solely data-driven. ## conclusion in this paper, we presented a pragmatic perspective on fairness challenges in medical machine learning. through empirical subgroup analyses on three diverse clinical tasks, we showed that performance disparities across patient populations are not only common but often hidden by aggregate metrics. since 'one size fits all' solutions, where ml models aim but fail to perform equally across all subgroups, are rarely adequate in real-world scenarios, we have demonstrated the importance of linking fairness and transparency: making biases visible, reporting subgroup-specific performance, and acknowledging data limitations. also, we need further efforts to help overcome access barriers to clinical research and optimal care, as this would also help to improve medical datasets used to develop and train fair models. likewise, best practices and standards for evaluating and reporting subgroup performance need to be developed. this transparency serves two purposes: it allows physicians to weigh in on the model's performance across subgroups for clinical decision-making, and at the same time, it enables targeted optimization of the model for those groups that are currently disadvantaged. in doing so, we can foster more responsible use of ml models in healthcare.