Connecting algorithmic fairness and fair outcomes in a sociotechnical simulation case study of AI-assisted healthcare

Domain: neuroscience
Published: 2025-12-19
Editorial outcome: minor_revision

Abstract
Artificial intelligence (AI) has vast potential for improving healthcare delivery, but concerns regarding biases in these systems have raised important questions regarding fairness when deployed clinically. Most prior studies on fairness in clinical AI focus solely on performance disparities between subpopulations, which often fall short of connecting the technical outputs of AI systems with sociotechnical outcomes. In this work, we present a simulation-based approach to explore how statistical definitions of algorithmic fairness translate to fairness in long-term outcomes, using AI-assisted breast cancer screening as a case example. We evaluate four fairness criteria and their impact on mortality rates and socioeconomic disparities, while also considering how clinical decision makers’ reliance on AI and patients’ access to healthcare affect outcomes. Our results highlight how algorithmic fairness does not directly translate into fair and equitable outcomes, underscoring the importance of integrating sociotechnical perspectives to gain a holistic understanding of fairness in healthcare AI.

Introduction
Widespread research and development of artificial intelligence (AI) systems in clinical settings has raised important questions regarding their implications on real-world fairness and health equity. Numerous prior studies to date have reported performance disparities between sociodemographic subgroups in a variety of use cases of AI systems developed for clinical decision support and medical image analysis1–6. Such systematic performance disparities between subpopulations on the basis of sex, gender, age, race, or socioeconomic status in clinical AI systems may pose a critical risk of introducing or further entrenching inequities in healthcare. Consequently, extensive research has focused on developing and evaluating bias mitigation methods7–15. These technical interventions into AI systems primarily aim to achieve mathematical formalizations of fairness (hereafter: “algorithmic fairness”), such as equal error rates across subgroups16. In many such studies, it is implied that after resolving performance disparities, the resulting AI systems will be fair. However, this framing focuses solely on the technical aspect of AI development, largely neglecting the reality of AI as a component of a sociotechnical system.

Sociotechnical systems are those that intrinsically contain relationships between technology, humans, and societal institutions17. AI systems designed for use in healthcare exemplify this concept particularly well—these technological systems are intended to be used by healthcare practitioners to assist in clinical decision-making, and the results of those decisions will have direct and tangible impacts on the lives of patients. Furthermore, the use of clinical AI can shape future interactions within the healthcare system, for example, by impacting human behaviors and diagnostic outcomes (which may also be captured in ensuing datasets used in the development of other AI systems). Selbst et al.17 describe that when failing to consider this broader sociotechnical scope, AI researchers pursuing algorithmic fairness lack the information necessary to understand how technical interventions translate to fair outcomes within the context of a deployed AI system.

Moreover, methods for defining and achieving algorithmic fairness in the medical domain have largely been borrowed from AI research applications such as finance or college admissions18. Thus, applying these methods to clinical scenarios often fails to consider important nuances. For example, algorithmic fairness does not account for the complex relationships between biological factors, environmental variables, and social determinants of health in the manifestation of medical conditions, and overlooks the intrinsic heterogeneity within patient populations defined by broad subgroups such as race and gender19. The “leveling down” effect20 on subgroup performance that is often a result of certain bias mitigation strategies also may pose a salient ethical dilemma when considering the principle of non-maleficence in medical care21. In addition, the emphasis on neutrality that is inherent to algorithmic fairness (e.g., equal performance between subgroups) fails to consider existing inequities faced by underprivileged groups—an algorithmically fair AI system will not be effective in mitigating health inequity if a subpopulation experiences barriers to accessing adequate healthcare in the first place22.

McCradden et al.23 emphasized that while algorithmic fairness is an important aspect of responsible and ethical use of AI in healthcare, only satisfying technical definitions of fairness is generally not enough to ensure that patients will experience fair treatment and outcomes. However, investigating outcomes-based measures of fairness is a complex task that requires more than just quantifying subgroup performance. Studies that have attempted to explore and quantify outcome-based fairness have typically done so in simulation-based settings using dynamic models, since a temporal aspect is required to gain an understanding of long-term outcomes and possible feedback loops within a sociotechnical system. For instance, D’Amour et al.24 simulated long-term fairness outcomes in various non-healthcare scenarios that utilize algorithmic systems. They noted that a dynamic analysis is crucial for gaining insight into how algorithmic outcomes may impact future interactions within a sociotechnical system (for instance, algorithmic decisions in bank lending can affect credit scores, which can in turn impact future loan approvals). This study, along with several others25–27, framed their analyses around a single stage in which algorithmic decisions are made. In contrast, Scher et al.28 simulated various stages within a sociotechnical system using a compartmental modeling approach, which enabled them to investigate the impact that each component had on long-term fairness outcomes.

However, none of these prior simulation-based works have focused on medical applications of AI. Within this context, mammography screening for breast cancer is a relevant and interesting use case for studying the potential long-term fairness impacts of healthcare AI systems for a number of reasons. First, research and development of AI-based mammographic analysis systems have surged in recent years, with over 20 applications that have already been approved or cleared by the United States (U.S.) Food and Drug Administration (FDA)29. Second, however, is that these systems were rarely evaluated for subgroup-specific performance prior to FDA clearance30,31 (Supplementary Table 1). Notably, the FDA-cleared Profound AI system was retrospectively found to exhibit false positive rate disparities across different categories of age and self-reported race32. This emphasizes that, although considerable progress has gone toward the development of recommendations for standards that ensure safe and ethical deployment of medical AI systems33–35, the current regulatory landscape makes it challenging to know if and how algorithmic bias could manifest during clinical deployment of these systems36. Finally, it is well established that considerable sociodemographic disparities exist in breast cancer screening, treatment, and mortality in the U.S.37–41 Taken together, these factors make breast cancer screening a compelling case study for an initial investigation into how algorithmic fairness in AI-assisted clinical decision-making systems could impact long-term health equity dynamics.

In this work, we develop an innovative sociotechnical system-inspired simulation and use it to investigate the relationship between algorithmic fairness and the fairness of long-term outcomes in a healthcare setting. Specifically, we propose a compartmental model to simulate the case example of AI-assisted breast cancer screening and its surrounding sociotechnical context. In contrast to studies that aim to accurately model real-world sociodemographic populations to simulate clinical outcomes, this work aims to gain a high-level understanding of how algorithmic fairness may interact with healthcare system components and existing disparities. To this end, we model a simplified version of the real-world breast cancer screening and care process in the U.S., which is not intended to exhaustively represent all nuanced aspects of disease pathology, medical institutions, socioeconomic factors, or treatment. Rather, we use clinically informed design choices to model relevant abstractions of different stages and patient characteristics. Importantly, we do not aim to explicitly represent any real-world sociodemographic groups in this work. Instead, we model two hypothetical groups, with “privileged” and “unprivileged” as pseudo-demographic labels, for which we draw assumptions largely from information gathered on socioeconomic disparities in breast cancer treatment, screening, and mortality in the U.S. Modeling these abstract representations of subpopulations which faced differential barriers and degrees of access to healthcare facilitates our primary goal of exploring how technical definitions of fairness influence clinical populations in a society where challenges to health equity exist. In summary, we present a dynamic model to simulate and analyze long-term impacts of algorithmic fairness in a clinical AI system. We utilize our sociotechnical system-inspired simulation to investigate the extent to which mathematical formalizations of fairness in an AI-based computer-aided diagnosis system can translate to fair health outcomes.

Results
Overview
We developed a discrete-time compartmental model to simulate a population of women undergoing annual mammography screening for the detection of breast cancer as illustrated in Fig. 1. Within this population, we modeled two hypothetical subgroups, with the “privileged” group having a higher average socioeconomic index than the “unprivileged group”. The socioeconomic index of an individual in the population was a single value that represented an aggregate measure of factors related to financial accessibility, insurance status, healthcare policy impacts, income level, and access to healthcare-related resources. This value determined the probability of experiencing barriers to accessing care within the breast cancer screening and subsequent treatment process, including the opportunity to get screened, the likelihood of attending diagnostic follow-up appointments, the chance of experiencing treatment delays, and the ability to receive adjuvant treatment. Breast cancer screening was modeled as a double-reader process (radiologist and AI), with the radiologist rendering the final decision, influenced by how much importance was given to the AI decision compared to their own. Here, we assumed the radiologist to be “unbiased”, i.e., their diagnostic performance was equal between the privileged and unprivileged groups.

For the AI system, four different statistical definitions of algorithmic fairness were explored: (1) “Naive”, with disparities in both true positive rate (TPR) and false positive rate (FPR) between the privileged and unprivileged groups, (2) “equal opportunity” with TPR parity and disparity in FPR (EqOppTPR), (3) “equal opportunity” with FPR parity and disparity in TPR (EqOppFPR), and (4) “equalized odds” with both TPR and FPR parity (EqOdds). More specifically, in scenarios with TPR and/or FPR disparities, the privileged group had a higher TPR and/or FPR, respectively, compared to the unprivileged group. It was assumed that imposing error rate parity was achieved through “fair representation learning” methods, which resulted in increased performance for the initially lower-performing group (e.g., TPR increased, FPR decreased) and decreased performance for the initially higher-performing group (e.g., TPR decreased, FPR increased).

Individuals who received a breast cancer-negative imaging-based diagnosis returned to the population pool, while those who received an imaging-based positive diagnosis continued to the diagnostic confirmation stage, which represented follow-up imaging and biopsy analysis. If it was determined that a simulated individual received a false positive diagnosis from mammography screening, they returned to the population pool with a reduced probability of attending their screening appointment in the following year. All individuals with a true positive diagnosis went on to receive treatment, which consisted of a two-stage process, simulating surgical and adjuvant interventions. Undergoing treatment incurred a financial burden, which was modeled by a reduction in socioeconomic index; this financial burden was greater for the unprivileged group. It was assumed that all stages, from screening to treatment, were completed within one time step, assumed to be one year. Individuals were presumed deceased if the severity of their breast cancer exceeded a predefined threshold at the end of a given time step. Group-specific mortality rate and socioeconomic outcomes were computed at the end of each time step.

Using this simulation, we: (1) compared how each algorithmic fairness scenario impacted group-wise mortality and socioeconomic disparities over time, and (2) assessed how these outcomes were impacted by (a) various performance disparity ranges in the AI system, (b) the radiologist’s role in diagnostic decision-making, and (c) non-algorithmic interventions within the system. Modeling details for each stage illustrated in Fig. 1 are presented in the Methods section, along with justification for corresponding parameters.

Comparison of outcome disparities over time for different algorithmic fairness constraints
The simulation was run for 50 time steps (years) for each of the four algorithmic fairness scenarios (see Methods, Table 2), where it was assumed that the radiologist gave equal importance to their own diagnostic decision and the decision of the AI system, such that the final diagnosis had equal contribution from the two “readers”. Disparities in breast cancer mortality rates and average socioeconomic index between the privileged and unprivileged groups over time are visualized in Fig. 2. We observed stark differences in annual mortality rates between the privileged and unprivileged groups, regardless of the algorithmic fairness constraints imposed. The Naive and EqOppFPR scenarios led to very similar outcomes and likewise, the EqOppTPR and EqOdds scenarios also led to similar results (Fig. 2A). Specifically, the average mortality rate ratio for the EqOppFPR scenario was 3.82 (95% CI: 3.80, 3.83) compared to 3.77 (95% CI: 3.76, 3.70) in the Naive scenario (Fig. 2B). This was reduced to 3.05 (95% CI: 3.04, 3.06) and 3.07 (95% CI: 3.06, 3.08) under the EqOppTPR and EqOdds algorithmic fairness scenarios, respectively. Comparing the Naive and EqOdds scenarios, this amounted to an absolute reduction in mortality rate disparity of 12.22 (95% CI: 11.91, 12.53) per 100k individuals. Although the reduced disparity in mortality rates implies a positive health equity outcome, it is notable that in the EqOppTPR and EqOdds scenarios, the privileged group's mortality rate increased (Fig. 2A). This was a result of the assumed “leveling down” effect that meant reduced sensitivity for the privileged group to meet algorithmic fairness targets. With respect to the average socioeconomic index of the two groups, the Naive and EqOppFPR systems reduced the inter-group disparity relative to EqOppTPR and EqOdds (Fig. 2C).

Assessment of the effect of AI system group-wise performance differences on outcomes
We also analyzed average mortality rate ratio and socioeconomic index disparities over 50 years for a range of values corresponding to AI system performance disparities between the privileged and unprivileged group. Here, higher values of ΔTPR represent reduced TPR for the unprivileged group relative to a static TPR for the privileged group, and higher values of ΔFPR represent increased FPR for the privileged group relative to a static FPR for the unprivileged group (See Methods, “Experiments”). These results are presented in Fig. 3, where it is observed that TPR disparities had a larger impact on determining outcomes compared to FPR disparities (Fig. 3A). For instance, at a given ΔFPR, lowering the TPR for the unprivileged group by 30% increased the mortality rate ratio by over 15%. This can be attributed to higher rates of false negatives (missed diagnoses) in the unprivileged group. In contrast, at a given ΔTPR, a 30% higher FPR for the privileged group led to less than a 5% reduction in the mortality rate ratio. However, this finding can be attributed to a higher mortality rate for the privileged group due to the false positives negatively impacting the likelihood of attending screening in the subsequent year. Regarding socioeconomic index disparities, higher values of ΔTPR reduced the inter-group socioeconomic index disparity with ΔFPR having minimal impact (Fig. 3B).

Evaluation of the impact of radiologist reliance on AI decisions
The simulation was also run for a range of values corresponding to how much weight, or “importance” the radiologist gave to the AI system when making a breast cancer diagnosis. More specifically, an AI reliance parameter value of one in our simulation meant that the diagnosis was based entirely on the decision of the AI system, while a value of zero meant that the diagnosis was based entirely on the radiologist’s decision. The average mortality rate ratio over 50 years for this analysis is visualized in Fig. 4. Here, it can be seen that the mortality rate disparity surged with a higher degree of reliance on the AI system under both the Naive and EqOppFPR scenarios. The mortality rate disparity was unaffected by AI reliance under the EqOdds scenario, and decreased slightly with full reliance under EqOppTPR compared to zero reliance. However, as can be seen from the actual misdiagnosis rates (FPR and FNR), presented for AI reliance parameter settings of 0.0, 0.5, and 1.0 in Table 1, the EqOppTPR and EqOdds scenarios resulted in increased FPR disparities when more weight was given to the AI model decision. Thus, the mortality rate disparity decreased under EqOppTPR scenario only due to increasing FPR for privileged group, leading to fewer of this group returning for screening in following years, and therefore dying at a rate more similar to that of the unprivileged group (Supplementary Table 2).

Sensitivity analysis of non-algorithmic healthcare system interventions
The screening, treatment delay, and adjuvant treatment stages of the simulation were modeled to have the largest discrepancies between the privileged and unprivileged group, with the unprivileged group being 1.5 times less likely to get screened, twice as likely to experience delays, and half as likely to obtain adjuvant treatment (see Methods). Therefore, we compared outcomes for the different algorithmic fairness scenarios over a range of values corresponding to how likely an individual of a specific socioeconomic index was to undergo each step, which practically corresponded to a range of proportions of each group getting screened, experiencing treatment delays, or receiving adjuvant treatment. The results of this analysis are illustrated in Fig. 5. Here, it can be seen that treatment delays and access to adjuvant treatment both had minor impacts on mortality rate disparities relative to screening access. In particular, when approximately 93% of the unprivileged group was able to be screened, the mortality rate disparity in the Naive scenario was reduced to 56.88 (95% CI: 56.68, 57.08) per 100k, from 105.48 (95% CI: 105.26, 105.71) per 100k in the baseline setting when approximately 65% of the unprivileged group was screened (illustrated in Fig. 2A). This corresponded to an absolute reduction in mortality rate per 100k for the unprivileged group from 144.06 (95% CI: 143.86, 144.27) to 95.38 (95% CI: 95.21, 95.54). On the other hand, reducing the proportion of the unprivileged group experiencing treatment delays to 0.05 reduced their mortality rate per 100k to 143.67 (95% CI: 143.46, 143.88) and increasing the proportion of the unprivileged group receiving adjuvant treatment to 0.95 reduced their mortality rate per 100k to 143.89 (95% CI: 143.68, 144.11) (Supplementary Tables 3–5). However, higher rates of screening and adjuvant treatment in the unprivileged group also correlated with an increase in inter-group socioeconomic index disparity, as a result of treatment costs.

Methods
Population characteristics
The simulated population is composed of 500,000 women within the age range of recommended mammographic screening73. Therefore, the age, x_A, of each individual is drawn from a uniform distribution spanning 40 to 75 years:

x_A ~ U(40, 75) (1)

We model a scenario in which a binary attribute of group membership, x_G, corresponds to whether an individual in the population was “privileged” or “unprivileged”. The “privileged” group is defined by having a higher socioeconomic index, x_SEI, on average than the “unprivileged” group, as parameterized by Eq. 2 (Fig. 6).

x_SEI ~ N(1, 0.5), if x_G = privileged
        N(-1, 0.5), if x_G = unprivileged (2)

The x_SEI of each individual in the population is a single value representing an aggregate measure of socioeconomic status, which in reality could include factors such as household income, insurance coverage, healthcare policy impacts, education level, occupation, and access to resources74. In our simulation, higher values of x_SEI correspond to a higher socioeconomic status level.

Breast cancer incidence and mortality risk score
Each individual in the population is defined as either having breast cancer, i.e., x_BC = 1, or being “healthy”, i.e., x_BC = 0. The probability of the incidence of breast cancer is defined as a function of age, based on previously reported age-specific rates for females in the U.S. between 2007-2011, published by the Surveillance, Epidemiology, and End Results (SEER) Program of the National Cancer Institute75 (Eq. 3).

p(x_BC = 1 | x_A = a) = 
{ 2.44 × 10^-4, if a ∈ (40, 50)
  4.44 × 10^-4, if a ∈ [50, 60)
  6.20 × 10^-4, if a ∈ [60, 75] } (3)

Individuals with breast cancer also have an associated mortality risk score, x_R, which abstractly represents the relative severity of the disease, similar to how cancer stage is defined in the real world. For all individuals with breast cancer (x_BC = 1), their corresponding value of x_R is drawn from a truncated exponential distribution bounded between x_R = 1 and x_R = 4, where high values of x_R correspond to more severe breast cancer (Fig. 7). Intuitively, this represents a scenario in which most cases of breast cancer are less severe in the beginning, while a smaller number manifest more rapidly and aggressively76. For all healthy individuals in the population, the mortality risk score is defined to be zero.

Screening
Each individual in the population is assumed to have the opportunity to receive mammography-based breast cancer screening once each year. In the U.S., rates of breast cancer screening are strongly associated with social determinants of health, particularly with socioeconomic indicators such as income, education level, employment status, and social support37. For instance, women with lower household income, lack of health insurance, or who face food insecurity are less likely to receive mammography screening38. Therefore, we model the probability of an individual getting screened for breast cancer in a given year as a logistic function dependent on x_SEI:

p(screening | x_SEI) = 1 / (1 + e^(-a_s * (x_SEI - b_s))) (4)

where a_s and b_s are fixed parameters.

Future screening attendance behavior has also been associated with prior false positive results. Although previous studies have shown variable effects77, a recent large-scale cohort-based study in the U.S. found that women who received a biopsy recommendation after a false positive screening result were significantly less likely to return for screening in the next 9-30 months78. Based on these findings, individuals in our simulation who receive a false positive diagnosis have the probability of receiving a screen in the following time step reduced by 10%.

Diagnosis
The diagnosis provided from the AI-assisted mammography screening is modeled as a shared decision-making process between a radiologist and an AI system. Practically, this means that the AI system and the radiologist each have a separate, independent decision function, and the output of each respective function represents the probability that a particular individual had breast cancer. These separate decision functions are then combined to represent the final AI-assisted diagnosis.

For both the radiologist and the AI system, the likelihood of an accurate breast cancer diagnosis is a function of the mortality risk factor, x_R. Intuitively, more severe breast cancer (i.e., higher x_R) should be easier to identify on a mammogram, while less severe cases may be more difficult to diagnose. The AI decision function is also dependent on group membership, x_G, since we simulate a scenario in which the AI model is “algorithmically unfair”, i.e., there are performance disparities between the two groups. Therefore, the AI decision function is represented as f(x_R, x_G) (5) while the radiologist's decision function is represented as g(x_R) (6). The final diagnosis provided by the radiologist is modeled as a convex combination of the two decision functions, weighted by a parameter α, which represented how much the radiologist relied on the AI decision versus their own (Eqs. 7–8).

d = α * f + (1 - α) * g, where α ∈ [0, 1] (7)

x_D ~ Bernoulli(d) (8)

Here, d is the probability of a breast cancer-positive diagnosis, which considers the independent decisions of both the AI system and the radiologist, and x_D is the final AI-assisted diagnosis of either healthy (x_D = 0) or breast cancer-positive (x_D = 1).

The output of the AI decision function f(x_R, x_G) represents the probability of a breast cancer-positive diagnosis given an individual’s mortality risk score and group membership. More specifically, the probability of f resulting in a breast cancer-positive diagnosis is defined for each value of x_R. For individuals without breast cancer (x_BC = 0), x_R = 0 by definition, therefore, the likelihood of a breast cancer-positive diagnosis corresponds to the likelihood of a false positive diagnosis. For individuals with breast cancer (x_BC = 1), x_R ∈ [1, 4], therefore, the probability of a breast cancer-positive diagnosis corresponds to the probability of a true positive diagnosis. This probability of a breast cancer-positive diagnosis is defined for x_R = 0, x_R = 1, and x_R = 4, with the range of values between x_R = 1 and x_R = 4 assumed to be linear. The AI decision function f(x_R, x_G) is illustrated in Fig. 8.

The output of the radiologist's decision function g(x_R) also models the probability of a breast cancer-positive diagnosis. However, this is only dependent on the mortality risk score of an individual. f(x_R, x_G) is parameterized separately for each algorithmic fairness scenario, while g(x_R) is kept constant. Details can be found in the “Algorithmic Fairness Scenarios” and “Experiments” subsections.

Following the AI-assisted mammography screening, all individuals screened negative for breast cancer return to the population pool. Those who are screened positive are intended to move to a diagnostic confirmation stage, which would include follow-up imaging and biopsy analysis. However, it has been shown that vulnerable populations79, and those who face high out-of-pocket costs associated with low insurance coverage55 are less likely to proceed with additional diagnostic evaluation following mammography screening. Therefore, we assume that the probability of moving to the diagnostic confirmation stage is associated with x_SEI, given as:

p(follow-up | x_SEI) = 1 / (1 + e^(-a_f * (x_SEI - b_f))) (9)

where a_f and b_f are fixed parameters. Individuals who do not proceed to this stage return to the population pool. After diagnostic confirmation, any individuals who were incorrectly screened positive in the mammography screening step (i.e., false positives) return to the population pool, while those with an accurate breast cancer diagnosis move on to the treatment stage.

Treatment
Prior to receiving treatment, a portion of the individuals diagnosed with breast cancer are assumed to experience delays in receiving care. Numerous studies have demonstrated that racialized and lower socioeconomic status groups are more likely to experience delays in receiving breast cancer treatment41,80–82, and that delays in receiving surgical treatment are associated with increased breast cancer mortality39. Therefore, the probability of an individual experiencing treatment delays is modeled as a logistic function dependent on socioeconomic index:

p(delay | x_SEI) = 1 / (1 + e^(-a_d * (x_SEI - b_d))) (10)

where a_d and b_d are fixed parameters. Individuals who experience treatment delays also experience an increase in x_R by a value of 0.5 to represent the resulting increase in mortality risk as reported in the literature39.

Treatment for breast cancer is modeled as consisting of two stages, which represent surgical treatment and adjuvant treatment, respectively. Every individual diagnosed with breast cancer is assumed to undergo surgical treatment, while the probability of an individual undergoing the adjuvant treatment is dependent on their socioeconomic index:

p(adjuvant treatment | x_SEI) = 1 / (1 + e^(-a_t * (x_SEI - b_t))) (11)

where a_t and b_t are fixed parameters. This design choice arose from established literature, which showed that while the vast majority of women diagnosed with breast cancer undergo surgical treatment, there are disparities in receiving adjuvant treatments (such as chemotherapy, radiotherapy, or hormone therapy) between racial and socioeconomic groups40,81,83,84.

It is assumed that surgical treatment has the greatest effect in reducing mortality risk, while adjuvant treatment provides a smaller scale but nonetheless important benefit85,86. Therefore, the effect of the surgical treatment is modeled as a reduction in mortality risk score, x_R, by a value of 1, while the effect of the adjuvant treatment is modeled as an additional reduction in x_R by a value of 0.5.

Each treatment is also associated with a reduction in socioeconomic index, x_SEI, to simulate the financial burden of breast cancer treatment, such as costs and loss of income87. The reduction in x_SEI that results from treatment is modeled to be twice as much for the unprivileged group compared to the privileged group, based on the assumption that the unprivileged group would be more likely to have poorer insurance coverage, hourly wages versus yearly salary-based professions, and less of a financial safety net, resulting in a higher overall economic impact of undergoing treatment46,47.

After each individual completes their respective treatment(s), their mortality risk score x_R is assessed against a threshold value x_R,thresh defined here to be x_R,thresh = 3. If x_R exceeds x_R,thresh, the individual is assumed to have died, while if x_R was below x_R,thresh, the individual is assumed to have been cured, and they return to the population pool as healthy (x_BC = 0).

End of time step
In our simulation framework, one year corresponds to a single time step. At the end of each time step, which contains the screening, diagnosis, and treatment stages, mortality is also computed for individuals with breast cancer who remained in the population pool. These individuals either had false negative screenings or were never screened in the first place due to their socioeconomic status. It is assumed that the mortality risk scores for these individuals are increased by a value of x_R = 1, representing the amount of disease progression over a single year without treatment. Then, it is assessed whether the mortality risk scores for these individuals exceed x_R,thresh. If so, they are assumed to be deceased.

Subsequently, the age of each individual is increased by one year, and those whose age exceeds 75 years, as well as those who died, are removed from the population pool. An influx of new individuals between the ages of 40−45, assumed to be newly eligible for annual breast cancer screening, is then added, such that the total population size remains at 500,000. Each year, the incidence of new breast cancer cases is computed for those in the population pool according to Eq. 3. It is assumed that every individual in the population pool has the same likelihood of getting breast cancer, regardless of whether they already had it and were successfully treated.

Outcome metrics
Group-specific mortality rates (MR) were computed based on the number of individuals who died of breast cancer per group in a given year:

MR_group = N_died,group / N_group (12)

Mortality rate ratios between the unprivileged and privileged groups were also computed to evaluate the disparity in breast cancer mortality:

MRRatio = MR_unprivileged / MR_privileged (13)

Disparities in socioeconomic outcomes between groups were evaluated with the difference in mean socioeconomic index between groups:

ΔSEI = |μ_SEI,privileged - μ_SEI,unprivileged| (14)

Algorithmic fairness scenarios
In the context of the simulation, we assumed that the “Naive” model (i.e., the model without any algorithmic fairness interventions applied to it) initially had group-wise disparities in both the true positive rate (TPR) and false positive rate (FPR), such that it tended to predict the positive class more often for the privileged group, resulting in higher sensitivity but lower specificity for this group relative to the unprivileged group. Three algorithmic fairness methods were simulated, corresponding to equal opportunity with TPR parity (EqOppTPR), equal opportunity with FPR parity (EqOppFPR), and equalized odds with both TPR and FPR parity (EqOdds). These criteria are generally used as algorithmic fairness constraints when groups under consideration may have different prevalence rates of positive instances16, making them a commonly used measure of algorithmic fairness in disease prediction tasks21. For the scenarios with TPR and/or FPR parity, it was assumed that the performance of the group with the more favorable TPR or FPR in the Naive model was “leveled down” to achieve equal performance between groups. This can be the case with fair representation learning/“in-processing” bias mitigation methods20,22,60, which have been commonly proposed in the literature for algorithmic fairness in healthcare setting8,11,15. Details for each of the four scenarios are listed in Table 2.

Experiments
Each of the four algorithmic fairness scenarios was simulated for 50 time steps (years), with the first time step being discarded as a simulation “ramp-up” period. Parameters for the logistic functions were selected to reflect values reported in the literature of the relative differences in the rates that the privileged group would undergo screening, diagnostic follow-up, treatment delay, and adjuvant treatment compared to the unprivileged group. These corresponded to the unprivileged group being, on average, approximately 1.5 times less likely to get screened88 (a_s = 5, b_s = -6.25), 0.95 times as likely to attend diagnostic follow-up appointments (a_f = 0.77, b_f = -2)79, twice as likely to experience treatment delays39 (a_d = 0.95, b_d = -1.2), and half as likely to receive adjuvant treatment40 (a_t = 0.45, b_t = -1.8) in the Naive scenario. These parameters also reflected rates across the overall population that approximately corresponded to those reported in the literature for the various stages of the simulation (i.e., screening uptake: ≈80%89, lost to follow-up post-screening: ≈10%79, experiencing delays: ≈20%80, receiving adjuvant treatment: ≈80%40).

The baseline AI reliance parameter was set at α = 0.5. The radiologist decision function, which models the likelihood of a true or false positive diagnosis given an individual’s mortality risk score, was defined according to Eqs. 15–17.

p(TP | x_R = 1) = 0.65 (15)
p(TP | x_R = 4) = 0.9 (16)
p(FP) = 0.1 (17)

Sensitivity analyses were also conducted to assess the impact of several variables on average mortality and socioeconomic outcomes over the course of 50 in-simulation years. In each sensitivity analysis experiment, only the variable of interest was modified, with the rest of the parameters being equal to that in the baseline scenario (detailed in the previous paragraph).

In the first sensitivity analysis experiment, performance disparities in TPR and FPR between the privileged and unprivileged groups of the AI system were varied between values of 0.0 and 0.3 in increments of 0.05. Specifically, performance disparities in TPR amounted to lower TPR for the unprivileged group relative to a constant TPR for the privileged group (Eqs. 18–19).

p(TP | x_R = 1, x_G = privileged) = 0.75 (18)
p(TP | x_R = 4, x_G = privileged) = 0.95 (19)

Likewise, performance disparities for FPR amounted to increased FPR for the privileged group relative to a constant FPR for the unprivileged group (Eq. 20).

p(FP | x_G = unprivileged) = 0.05 (20)

In the second sensitivity analysis experiment, the AI reliance parameter varied between no reliance (α = 0) and full reliance on AI decision (α = 1) in increments of 0.1. In the final sensitivity analysis experiment, the parameters defining the logistic functions for screening, treatment delay, and adjuvant treatment were also varied such that approximately 5%, 25%, 50%, 75%, and 95% of the unprivileged group underwent screening, experienced treatment delays, and accessed adjuvant treatment. Additionally, group-specific mortality rates for a range of AI system TPR and FPR were computed, and reported in Supplementary Fig. 1 (Source data 8 file).

Statistics and reproducibility
Each experimental setup was run for 100 simulations with distinct random seeds, and 95% confidence intervals were computed for outcome metrics.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "This manuscript explores the important and timely intersectionality of algorithmic fairness and health equity. In the authors’ words, their aim was to leverage application of AI-based assistive software to breast cancer screening as a case study “to compare how various algorithmic fairness scenarios impacted patient outcomes”. While the report is conceptually valuable and potentially addresses a relevant use case, there are significant methodological, interpretive, contextual concerns specific to AI applications in cancer care (and healthcare in general) that limit impact. These are outlined in the major comments below. The manuscript could be significantly strengthened with revisions that clarify its contribution, address key assumptions, and better articulate its practical implications. Of note, this review combines input from both a health data science specialist and a senior oncologist.\n1. Introduction/Discussion: While these sections are well-referenced and present some interesting insights, they are long-winded, repetitive, and would benefit from editing for length and language. On a fundamental level, the authors should be cognizant of how the healthcare community is becoming acutely jaded to jargon such as “sociotechnical systems/context”. If there is a field emblematic of constant, timeless incorporation of radical and incremental technological change into human society, it would be healthcare. Data science is hardly special; one could argue that technological advances of tomographic imaging, robotic surgery, and immunotherapy were as or more significant than ML or LLM (at least so far) and have never stopped contending with equity issues. The authors should be aware (and could specifically cite) that any technological advance of any kind typically worsens health disparity – this is the well-described “Fundamental Cause Theory” (PMID: 34949900). As a case in point, although the county-funded hospital we are associated with serving “underprivileged” patients does has mammography CDS software, it does not have resources for formal AI implementation infrastructure akin to TRAIN.\n2. Clinicians are well aware of distinctions between technical device performance across patient populations and the availability/use of these devices (a great example is the FDA’s current struggle with regulating the accuracy/reliability of pulse oximetry monitors across different skin colors, and potential cost/impact of upgrading monitors in low resource settings); therefore, this modeling exercise is not addressing a novel problem. What would be more interesting is to leverage the exercise to model in a more sophisticated fashion across patient, systems, and societal levels. For example, the authors chose to simplify social risk across two arbitrary groups labeled “privileged” vs “unprivileged”. This approach does not work ideally for breast screening. “Privilege” is a soft term that most equate with income/wealth. The strongest predictor for breast screening uptake is not patient-level financial resources, it is insurance coverage status. Which in the U.S. (the context of these reviewers) is a policy issue operating at a societal level (the ACA marketplace mandates mammography coverage, which itself depends on state-level eligibility policies). The authors could consider adding fairly simple inputs into their model if they want to make it more relevant to the U.S. than Canada (or more closely confining their modeling to specific geographic locations) to incorporate this (ACA expansion/non-expansion state status, yes/no state income taxation). The authors could then better investigate what individual patient-level characteristics to add to their modeling which best predict for mammography utilization (i.e. patient educational level, self-identified race, and income above/below 200% FPL, etc.).\n3. Results (lines 267-268): This will again be a U.S.-centric comment. The BCSC data referenced by the authors demonstrated only a short-term, incremental attrition of mammography utilization after FP results in a U.S. population, so this is a flawed assumption in the model. It has been shown that patient cost-sharing in the U.S. is a far better predictor of supplementary post-screening imaging utilization (PMID: 36972047) and should have a much larger downstream impact on clinical outcomes. On a policy level, many U.S. states are legislating coverage of such supplemental imaging, which could be added simply as a dichotomous variable to this model. Again, an example of how a medical economics perspective could significantly improve the authors’ modeling without undue complexity. Or they could clearly specify that they are confining their modeling to Health Canada or another nationally funded healthcare system.\n4. The manuscript places significant emphasis on the finding that “TPR fairness (equal cancer detection rates) helped reduce mortality disparity, but FPR fairness (equal false alarms) didn't help and sometimes made things worse.” However, this outcome is largely expected given the clinical context of breast cancer, where early and accurate detection is critical to survival. The disproportionate impact of false negatives versus false positives is well understood in oncology and medical decision-making. The authors should clarify whether this finding is intended as a validation of the simulation framework or presented as a novel contribution. If the latter, the manuscript would benefit from a more detailed discussion of what new understanding this result adds to the existing literature.\n5. The authors should add a clear concluding statement that highlights the specific contribution of this study beyond known issues such as “leveling down” trade-offs or the role of clinician behavior—which are already discussed in the introduction and elaborated in the discussion. The final paragraph should also offer practical guidance for AI developers or healthcare teams, such as how to balance fairness metrics with clinical priorities, or how to design systems that account for human-AI interaction.\n6. The abstract is well written, but it doesn’t clearly highlight the main findings of the study (e.g., takeaways from Table 1).\n7. The provided GitHub link appeared to be empty at the time we checked so we couldn’t review the code. We are particularly interested in performance of the authors’ modeling of radiologist reliance on AI decisions.\n8. In the Methods section, provide a clearer rationale or citation for choosing the values of logistic function parameters (e.g., for screening probability or treatment delay).\n9. The authors should clarify how they selected the four specific fairness scenarios (Naive, Equal Opportunity via TPR, Equal Opportunity via FPR, and Equalized Odds). Are there other studies in non-medical settings that used these scenarios?\n10. The role of the radiologist is modeled with a fixed AI reliance parameter, assuming constant behavior over time and across groups. In practice, clinician trust and use of AI can evolve dynamically and be influenced by feedback, institutional protocols, or prior AI performance. The authors should discuss how static AI reliance might oversimplify human–AI interaction, and whether the outcomes would differ under more adaptive or heterogeneous reliance models.\n11. The finding that increasing screening access significantly reduces mortality disparities is compelling but may be sensitive to model assumptions. The authors should clarify how screening parameters (e.g., effectiveness, risk progression) were chosen and whether they reflect real-world data. Additionally, the observed trade-off with increased socioeconomic disparity deserves further discussion and ethical framing.", "reviewer_2": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.", "reviewer_3": "This study addresses an important question: how algorithmic fairness metrics translate into real-world health outcomes when AI is integrated into clinical care. It shows that equalizing true positive rates (TPR) between groups reduces mortality gaps more effectively than adjusting false positive rates (FPR) or applying equalized odds. It also finds that improving access to screening has a much stronger impact on health disparities than any adjustment to the AI model itself. Overall, the results highlight that improving fairness in clinical AI requires addressing broader healthcare system factors, not just model performance. The simulation approach presented is innovative and thoughtfully designed. I believe the framework is generally strong; however, it suffers from conceptual gaps, lack of validation against real-world clinical data, and some issues with assumptions and interpretation of the findings.\nThe simulation results generally support the overall direction of the conclusions; However, some of the claims are stated too strongly given the limitations of the study design. For example, the finding that increasing screening access has a greater impact than model fairness interventions is important but may not reflect practical feasibility, as the model does not account for real-world barriers such as accessibility, insurance coverage, and trust in the healthcare system.\nThe methods are generally sound and clearly described, and the availability of code allows full reproducibility. I did not find major errors in the data analysis itself. However, there are inconsistencies between the modeling assumptions and the interpretation of the findings. For example:\n• Simulated mortality rates presented in Results, and Figure 2 appear substantially higher than actual U.S. rates without any comparison to empirical data. I suggest validating the simulated mortality rates against SEER or CDC datasets, or at least discussing the discrepancy and its implications for interpreting the findings.\n• Page 5, Line 142: \"We modeled a simplified version of the real-world breast cancer screening and care process...\"\nIt appears that the model assumes screening, diagnosis, treatment, and outcomes all occur within a single annual time step, ignoring real-world delays that are known to affect outcomes. I recommend incorporating a sensitivity analysis introducing plausible delays, or explicitly discussing how this simplification might bias mortality disparity findings.\nThere are also several important assumptions that need to be fully discussed or justified, including:\n• Line 171: \"Here, we assumed the radiologist to be 'unbiased'.\"\nModeling the radiologist as unbiased oversimplifies human decision-making, especially given documented studies on cognitive and institutional biases in healthcare. This assumption may have a strong impact on the interpretability of the outcome and practicality of the model in the real-world clinical settings.\n• Lines 182–184: \"It was assumed that imposing error rate parity was achieved through fair representation learning...\"\nAssuming fairness by reducing performance for the initially higher-performing group raises ethical concerns, particularly in clinical settings where non-maleficence is a core principle. I encourage the authors to discuss alternative bias mitigation strategies and better justify the ethical trade-offs involved.\n• Line 429: \"Ensuring that 92% of the unprivileged group received annual screening reduced this value by approximately 52 individuals per 100k...\"\nThis is a strong claim, especially given that achieving such high screening uptake faces complex socioeconomic challenges (e.g., access, insurance, trust). I suggest tempering the claim and more explicitly discussing these challenges.\n• Line 444: \"We modeled our simulation after the healthcare system in the U.S.\"\nTo improve generalizability, it would be helpful to briefly discuss how the model structure could be adapted to different healthcare systems and settings.\n• Line 488: \"...Fair AI\" be seen as a magic bullet with the ability to solve complex...\"\nThe analogy of \"magic bullet\" feels out of place here"}, "author_response": "AR: We are very appreciative of the external expertise from both the health data science and oncology domains. We believe that the suggestions provided by these reviewers are very important to consider and add more context, which we have incorporated into the paper as detailed in the comments below.\nTo clarify the contributions as requested by the reviewers, we would like to emphasize that our paper is not intended to be a precise clinical modelling study. In other words, we are not aiming to study the outcomes of specific clinical populations. Instead, the main goal of our work is to gain a better understanding of how technical approaches to AI fairness have the potential to impact health disparities. We intentionally chose not to model specific patient-level characteristics to avoid misinterpretation, as our focus is solely on exploring the potential impacts of algorithmic fairness within the context of a societally-embedded healthcare system — something that is typically overlooked by the technical community working on AI-based healthcare applications. This is touched on in our responses to comments #2 and #3, but we have also attempted to clarify the aim and scope of the paper more precisely in the abstract, introduction, and title of the paper, as follows:\nTitle:\n“Connecting algorithmic fairness and fair outcomes: A sociotechnical simulation case study of AI-assisted breast cancer screening”\nAbstract:\n“We modeled two hypothetical populations who experienced different levels of access to healthcare, and analyzed how four different definitions of algorithmic fairness impacted mortality rates and socioeconomic disparities over time.”\n“This work aims to explore the degree to which algorithmic fairness may translate to fair and equitable outcomes in clinical populations, with relevance far beyond this use case of AI-assisted breast cancer screening.”\nIntroduction:\n“In contrast to studies which aim to accurately model real-world sociodemographic populations to simulate clinical outcomes, this work aims to gain a high-level understanding of how algorithmic fairness may interact with healthcare system components and existing disparities.”\nR1: 1. Introduction/Discussion: While these sections are well-referenced and present some interesting insights, they are long-winded, repetitive, and would benefit from editing for length and language. On a fundamental level, the authors should be cognizant of how the healthcare community is becoming acutely jaded to jargon such as “sociotechnical systems/context”. If there is a field emblematic of constant, timeless incorporation of radical and incremental technological change into human society, it would be healthcare. Data science is hardly special; one could argue that technological advances of tomographic imaging, robotic surgery, and immunotherapy were as or more significant than ML or LLM (at least so far) and have never stopped contending with equity issues. The authors should be aware (and could specifically cite) that any technological advance of any kind typically worsens health disparity – this is the well-described “Fundamental Cause Theory” (PMID: 34949900). As a case in point, although the county-funded hospital we are associated with serving “underprivileged” patients does has mammography CDS software, it does not have resources for formal AI implementation infrastructure akin to TRAIN.\nAR: We appreciate the feedback and have attempted to remove redundancies throughout the introduction and discussion.\nWe also recognize that terms can often be overused and lose meaning over time. However, we, and our clinical collaborators, believe that the term “sociotechnical systems/context” remains highly relevant and appropriate to use in our paper. It has been a relatively short amount of time since the technical side of the healthcare AI community has begun to consider the fairness of these systems, but (as we mention in our response to comment #2) even this type of work typically positions fairness as a statistical property, and not as a human-defined, social quality. Therefore, we think it is important to further disseminate the concept that technologies do not act in isolation when integrated into the social context they were designed for. We do appreciate that the reviewers have pointed us to the reference on Fundamental Cause Theory, which is also highly relevant in the context of our paper. We have integrated this into the revised conclusion paragraph as follows:\n“However, AI as a technology is not unique in its potential to impact health disparities — “privileged” populations in society have historically been able to gain disproportionate benefits in health and longevity as a result of new medical technologies.67 [...]”\nR1: 2. Clinicians are well aware of distinctions between technical device performance across patient populations and the availability/use of these devices (a great example is the FDA’s current struggle with regulating the accuracy/reliability of pulse oximetry monitors across different skin colors, and potential cost/impact of upgrading monitors in low resource settings); therefore, this modeling exercise is not addressing a novel problem. What would be more interesting is to leverage the exercise to model in a more sophisticated fashion across patient, systems, and societal levels. For example, the authors chose to simplify social risk across two arbitrary groups labeled “privileged” vs “unprivileged”. This approach does not work ideally for breast screening. “Privilege” is a soft term that most equate with income/wealth. The strongest predictor for breast screening uptake is not patient-level financial resources, it is insurance coverage status. Which in the U.S. (the context of these reviewers) is a policy issue operating at a societal level (the ACA marketplace mandates mammography coverage, which itself depends on state-level eligibility policies). The authors could consider adding fairly simple inputs into their model if they want to make it more relevant to the U.S. than Canada (or more closely confining their modeling to specific geographic locations) to incorporate this (ACA expansion/non-expansion state status, yes/no state income taxation). The authors could then better investigate what individual patient-level characteristics to add to their modeling which best predict for mammography utilization (i.e. patient educational level, self-identified race, and income above/below 200% FPL, etc.).\nAR: We would like to first address the reviewers’ initial point regarding the novelty of the underlying question in our modelling study. We believe that it is important to note that there are at least a couple of aspects in which bias in medical AI systems is unique from performance differences in traditional medical devices. For instance, the ability of AI systems to identify patterns that are not obvious to the human eye but that are predictive of self-reported race, for instance (PMID: 35568690), means that identifying if and how algorithmic decisions are biased could be a much more complex task compared to if a device performs poorly on visually evident characteristics like skin tone. Additionally, AI systems have the ability to pick up on patterns of historical, societal biases and propagate them through to clinical predictions in a manner that may not be immediately apparent without comprehensive analysis of the model and the data it was trained on (PMID: 31649194). However, the main novelty in our study is related to the observation that, within the technical sphere of the community exploring AI for healthcare applications, algorithmic fairness is too-commonly sought after without consideration for how it could impact outcomes in real populations. This can be seen by, for example, the high number of papers proposing bias mitigation strategies to alleviate “unfairness” in healthcare models in prominent medical AI conferences (e.g., paper references 8-10) as well as in high-impact multidisciplinary and health-related journals (e.g., paper references 11-15). Typically, such papers conflate their aims of achieving equal error rates between groups with “fairness” and “health equity”. Therefore, we do see this work as addressing a novel research direction — specifically, by bridging the gap between the purely statistical view of fairness that is commonly aimed for in the technical community, and how fairness and healthy equity are viewed and contended with by clinicians and ethicists. We have revised our conclusions to emphasize this as the primary contribution of our work (see also the response to comment #5 below):\n“Although AI has the potential to improve access to healthcare across resource divides,68 recent works have brought attention to the potential of AI to introduce, perpetuate, or even amplify disparities in healthcare and patient outcomes.3,69 Within technical communities, algorithmic fairness is often seen as a strategy to prevent or mitigate this problem. However, the results of our work exemplify how integrating algorithmic fairness into an AI system does not necessarily provide a simple solution to health equity challenges.”\nWith respect to the comments regarding the “privileged” and “unprivileged” groups modelled in our study, we acknowledge that this is a significant oversimplification of social determinants of health in real-world populations. However, we have intentionally used these abstract group labels as a proxy, which are meant to encompass a variety of real-world sociodemographic factors that are associated with barriers to healthcare, including insurance status, healthcare policy impacts, and other patient-level characteristics. We appreciate that the reviewers pointed to insurance as being a strong predictor of screening uptake, therefore, we have explicitly included this in describing the healthcare accessibility risk factors associated with our two groups.\nOverview:\n“The socioeconomic index of an individual in the population was a single value that represented an aggregate measure of factors related to financial accessibility, insurance status, healthcare policy impacts, income level, and access to healthcare-related resources”\nMethods:\n“The x of each individual in the population is a single value representing an aggregate SEI measure of socioeconomic status, which in reality could include factors such as household income, insurance coverage, healthcare policy impacts, education level, occupation, and access to resources.73”\nWe have also added more emphasis and discussion of insurance coverage as an important factor to consider in this context, which is integrated into our response to comment #3 below.\nAs mentioned above, we intentionally avoid including specific patient-level characteristics as inputs to our simulation, as to not conflate our work (where we aim to understand the impact of algorithmic fairness within a sociotechnical systems) with a clinical modelling/epidemiological study (which aims to understand effects of clinical interventions on specific populations). We have attempted to clarify this in the following revisions:\nIntroduction:\n“Importantly, we did not aim to explicitly represent any real-world sociodemographic groups in this work. Instead, we modeled two hypothetical groups, with “privileged” and “unprivileged” as pseudo-demographic labels, for which we drew assumptions largely from information gathered on socioeconomic disparities in breast cancer treatment, screening, and mortality in the U.S. Modelling these abstract representations of subpopulations which face differential barriers and degrees of access to healthcare facilitated our primary goal, which was to explore how technical definitions of fairness influence clinical populations within a society where challenges to health equity exist.”\nDiscussion:\n“Importantly, the two groups we modelled were not intended to represent any particular demographic group in the U.S. or elsewhere, but were a broad abstraction of populations with differential levels of access to appropriate healthcare. The parameters we used were selected based on various literature sources (see Methods) that investigated disparities relevant to different factors, such as income level, race, and insurance status, which are subject to the limitations from those respective studies.”\n“However, we emphasize that the goal of this work was to compare the outcomes of algorithmic fairness scenarios relative to each other by modeling the healthcare process just well enough such that differences in outcomes between the two groups can be quantified.”\nWe are open to suggestions from the reviewers if they believe that there are more appropriate group-wise labels that can be used in our study.\nR1: 3. Results (lines 267-268): This will again be a U.S.-centric comment. The BCSC data referenced by the authors demonstrated only a short-term, incremental attrition of mammography utilization after FP results in a U.S. population, so this is a flawed assumption in the model. It has been shown that patient cost-sharing in the U.S. is a far better predictor of supplementary post-screening imaging utilization (PMID: 36972047) and should have a much larger downstream impact on clinical outcomes. On a policy level, many U.S. states are legislating coverage of such supplemental imaging, which could be added simply as a dichotomous variable to this model. Again, an example of how a medical economics perspective could significantly improve the authors’ modeling without undue complexity. Or they could clearly specify that they are confining their modeling to Health Canada or another nationally funded healthcare system.\nAR: We would like to thank the reviewers for pointing us to patient cost-sharing as a predictor of post-screening imaging use.\nWe believe that the BCSC reference (Miglioretti et al. 2024) remains a valid assumption in our simulation since we only use this information to model the reduced likelihood of returning to screening in the following year. From Miglioretti et al.:“Our primary outcome was a binary indicator of return to routine screening within 9 to 30 months of each eligible screening examination [...]”\nThe reviewers’ reference to cost-sharing (Hughes et al.) more specifically refers to the likelihood of receiving follow-up evaluation after a screening mammogram, which we agree should be included as a relevant factor within the context of our study that could shed more light on the interplay of medical economics with screening efficacy.\nHowever, the study cited by the reviewers reports relatively low effects (e.g., in Table 5, averaged over “Any subsequent breast image”, the marginal effects are between 0.7% and 2.4% for the different plan types), which is likely because the authors control for other factors (age, race/ethnicity, comorbidities, location). We believe that, in our study, it would be more appropriate to model disparities in screening follow-up based on Goldman et al. (PMID: 23358386), which compared high-level “vulnerable” vs. “non-vulnerable” populations. This is more consistent with our modelling approach, since our socioeconomic index variable is also intended to broadly encompass various socioeconomic-related factors, rather than specific characteristics, such as insurance type.\nTherefore, in response to this comment, we have modified the simulation using Goldman et al. as a reference, such that approximately 10% of all individuals screened do not attend follow-up/diagnostic confirmation, with the relative risk for the unprivileged group being approximately 0.95. We believe that this remains consistent with the paper cited by the reviewers (Hughes et al.) on the effect of out-of-pocket costs, since the ability to pay for these additional costs would be associated with higher socioeconomic stability. This modification to the simulation is reflected in the revised results, as well as in the Overview and Methods sections as follows:\nOverview:\n“This value determined the probability of experiencing barriers to accessing care within the breast cancer screening and subsequent treatment process, including the opportunity to get screened, the likelihood of attending diagnostic follow-up appointments, the chance of experiencing treatment delays, and the ability to receive adjuvant treatment.”\n“[...] while those who received an imaging-based positive diagnosis continued to the diagnostic confirmation stage, which represented follow-up imaging and biopsy analysis.”\nMethods:\n“Following the AI-assisted mammography screening, all individuals screened negative for breast cancer return to the population pool. Those who are screened positive are intended to move to a diagnostic confirmation stage, which would include follow-up imaging and biopsy analysis. However, it has been shown that vulnerable populations,78 and those who face high out-of-pocket costs associated with low insurance coverage54 are less likely to proceed with additional diagnostic evaluation following mammography screening. Therefore, we assume that the probability of moving to the diagnostic confirmation stage is associated with x , SEI given as:\n𝑝 𝑓𝑜𝑙𝑙𝑜𝑤 𝑢𝑝 𝑥 𝑆𝐸𝐼 = 1 / (1+ 𝑒 𝑎𝑓𝑥𝑆𝐸𝐼 + 𝑏𝑓)\nwhere a and b are fixed parameters. Individuals who do not proceed to this stage return to the population pool. After diagnostic confirmation, any individuals who were incorrectly screened positive in the mammography screening step (i.e., false positives) return to the population pool, while those with an accurate breast cancer diagnosis move on to the treatment stage.”\n“Parameters for the logistic functions were selected to reflect values reported in the literature of the relative differences in the rates that the privileged group would undergo screening, diagnostic follow-up, treatment delay, and adjuvant treatment compared to the unprivileged group. These corresponded to the unprivileged group being, on average, approximately 1.5 times less likely to get screened87 (a =5, b =-6.25), 0.95 times as likely to attend diagnostic follow-up appointments (a=0.77, b=-2), twice as likely to experience treatment delays39 (a =0.95, b =-.2), and half as likely to received adjuvant treatment40 (a=0.45, b=-1.8) in the Naive scenario.”\nWe also agree with the comment that medical economics and health policy are important factors to consider in the context of this work. Therefore, we have integrated more detailed discussion on this, as detailed below:\nDiscussion:\n“In practice, however, increasing access to screening, especially to the high levels simulated in our sensitivity analysis, is an extremely complex task. Systemic challenges, including cultural barriers, geographical limitations, and financial accessibility, mean that there is no simple solution to achieving perfect screening uptake. As an example, in the largely privatized healthcare system of U.S., which our simulation was loosely modeled after, the Affordable Care Act introduced measures intended to reduce disparities in access to preventive screening, such as for breast cancer, by eliminating out-of-pocket costs for patients.53 However, this legislation does not guarantee universal access to workup following mammography screening, meaning that disparities remain since some patients still must pay higher out-of-pocket costs for follow-up imaging and diagnostic procedures.54 Thus, the interplay of health policy and medical economics plays a crucial role in ensuring that initial barriers to access — before interaction with an AI system even begins — are effective in reducing health disparities. This highlights that even if AI systems are “fair”, the role they play in impacting health equity can be largely influenced by external social, economic, and political factors.”\n“AI policy and the degree to which AI systems are regulated and integrated into health systems would also be an important consideration for expanding this simulation framework to include different medical institutions, perhaps serving different populations.”\nR1: 4. The manuscript places significant emphasis on the finding that “TPR fairness (equal cancer detection rates) helped reduce mortality disparity, but FPR fairness (equal false alarms) didn't help and sometimes made things worse.” However, this outcome is largely expected given the clinical context of breast cancer, where early and accurate detection is critical to survival. The disproportionate impact of false negatives versus false positives is well understood in oncology and medical decision-making. The authors should clarify whether this finding is intended as a validation of the simulation framework or presented as a novel contribution. If the latter, the manuscript would benefit from a more detailed discussion of what new understanding this result adds to the existing literature.\nAR: We agree that it is important to clarify interpretation of this result within the context of our paper, both from a healthcare and AI development perspective. We do not see this as a novel clinical finding per se, but as a way to reiterate how following algorithmic fairness constraints without considering the sociotechnical context of an AI system can lead to a mismatch in goals and intended outcomes, a primary aim that we wish to highlight in our paper. We have clarified this in the following revision to the Discussion:\n“ While the disproportionate impact of false negatives compared to false positives is intuitive from a clinical perspective, it points to a potential disconnect between current algorithmic fairness research and real-world health outcomes: Although FPR disparities and equalized odds are often targeted in bias mitigation studies as important measures to minimize,15,42,43 their real-world impact in a healthcare setting is likely to be less dire than that of TPR disparities (which imply differential levels of missed diagnoses).”\nR1: 5. The authors should add a clear concluding statement that highlights the specific contribution of this study beyond known issues such as “leveling down” trade-offs or the role of clinician behavior—which are already discussed in the introduction and elaborated in the discussion. The final paragraph should also offer practical guidance for AI developers or healthcare teams, such as how to balance fairness metrics with clinical priorities, or how to design systems that account for human-AI interaction.\nAR: We have modified the concluding section of the paper to better summarize the takeaways of the paper, reduce redundancies, and have also included a new discussion of practical guidance for developing fair healthcare AI systems as follows:\n“This work aimed to explore the ways in which algorithmic fairness could translate to fair health outcomes, by considering interactions between an AI system, clinical users, and patient populations who face differential barriers to accessing healthcare. The results of our simulation study demonstrated how statistical definitions of fairness commonly applied to AI systems do not necessarily lead to equitable health outcomes when considering the sociotechnical context they exist in. However, AI as a technology is not unique in its potential to impact health disparities — “privileged” populations in society have historically been able to gain disproportionate benefits in health and longevity as a result of new medical technologies.67 Although AI has the potential to improve access to healthcare across resource divides,68 recent works have brought more attention to the potential of AI to introduce, perpetuate, or even amplify disparities in healthcare and patient outcomes.3,69 Within technical communities, algorithmic fairness is often seen as a strategy to prevent or mitigate this problem. However, the results of our work exemplify how integrating algorithmic fairness into an AI system does not necessarily provide a simple solution to health equity challenges. This highlights the importance of avoiding “technological solutionism”,17 which assumes that any and all problems can be solved with technological interventions like AI. Instead, AI, and algorithmic fairness specifically, should be viewed as one tool within an interdisciplinary toolbox that can complement other approaches for addressing large, complex societal challenges such as health disparities. Practical approaches for ensuring that AI improves patient outcomes and equity in healthcare rather than hindering it include translational trials70 and rigorous validation of these systems across diverse populations. Additionally, interdisciplinary research, development, and assessment throughout the entire AI life cycle can help to facilitate the implementation of holistically responsible healthcare AI,71 as designing such systems with only technical goals in mind will inevitably limit applicability and responsible integration into clinical care. While algorithmic fairness has the potential to prevent AI-induced health disparities, technical approaches to fairness should be seen as only one tool among others in the pursuit of ethically and equitably integrating AI technologies into medical care.”\nR1: 6. The abstract is well written, but it doesn’t clearly highlight the main findings of the study (e.g., takeaways from Table 1).\nAR: We have revised the abstract to include more takeaways as follows:\n“We also assessed how radiologists’ reliance on AI and access to appropriate healthcare affected outcomes in each scenario. The results demonstrated that only the methods which equalized true positives rates were effective in reducing mortality rate disparities. However, these methods also increased socioeconomic disparities. Furthermore, our findings showed that the degree to which the radiologist relied on biased AI decisions had a profound impact on mortality rate disparities. Moreover, we noted that enforcing algorithmic fairness had minimal effects on mortality compared to increasing access to screening.”\nR1: 7. The provided GitHub link appeared to be empty at the time we checked so we couldn’t review the code. We are particularly interested in performance of the authors’ modeling of radiologist reliance on AI decisions.\nAR: Our code was attached as a separate zip file at initial submission, which should be available in the reviewer portal. We have also provided an updated code file reflecting the revised experiments. The GitHub repository will be updated with the final version of the code upon acceptance.\nR1: 8. In the Methods section, provide a clearer rationale or citation for choosing the values of logistic function parameters (e.g., for screening probability or treatment delay).\nAR: We have clarified the rationale for selecting these parameters as follows:\n“Each of the four algorithmic fairness scenarios was simulated for 50 time steps (years), with the first time step being discarded as a simulation “ramp-up” period. Parameters for the logistic functions were selected to reflect values reported in the literature of the relative differences in the rates that the privileged group would undergo screening, diagnostic follow-up, treatment delay, and adjuvant treatment compared to the unprivileged group. These corresponded to the unprivileged group being, on average, approximately 1.5 times less likely to get screened87 (a=5, b=-6.25), 0.95 times as likely to attend diagnostic follow-up appointments (a=0.77, b=-2), twice as likely to experience treatment delays39 (a =0.95, b =-.2), and half as likely to received adjuvant treatment40 (a=0.45, b=-1.8) in the Naive scenario. These parameters also reflected rates across the overall population that approximately corresponded to those reported in the literature for the various stages of the simulation (i.e., screening uptake: 80%,88 lost to follow-up post-screening: 10%,78experiencing delays: 20%,79 receiving adjuvant treatment: 80%40).”\nR1: 9. The authors should clarify how they selected the four specific fairness scenarios (Naive, Equal Opportunity via TPR, Equal Opportunity via FPR, and Equalized Odds). Are there other studies in non-medical settings that used these scenarios?\nAR: The methods section has been updated to address this comment as follows:\n“In the context of the simulation, we assumed that the “Naive” model (i.e., the model without any algorithmic fairness interventions applied to it) initially had group-wise disparities in both the true positive rate (TPR) and false positive rate (FPR), such that it tended to predict the positive class more often for the privileged group, resulting in higher sensitivity but lower specificity for this group relative to the unprivileged group. Three algorithmic fairness methods were simulated, corresponding to equal opportunity with TPR parity (EqOppTPR), equal opportunity with FPR parity (EqOppFPR), and equalized odds with both TPR and FPR parity (EqOdds). These criteria are generally used as algorithmic fairness constraints when groups under consideration may have different prevalence rates of positive instances,16 making them a commonly used measure of algorithmic fairness in disease prediction tasks.21 For the scenarios with TPR and/or FPR parity, it was assumed that the performance of the group with the more favorable TPR or FPR in the Naive model was “leveled down” to achieve equal performance between groups. This can be the case with fair representation learning/“in-processing” bias mitigation methods,20,22,60 which have been commonly proposed in the literature for algorithmic fairness in healthcare setting.8,11,15 Details for each of the four scenarios are listed in Table 2.”\nR1: 10. The role of the radiologist is modeled with a fixed AI reliance parameter, assuming constant behavior over time and across groups. In practice, clinician trust and use of AI can evolve dynamically and be influenced by feedback, institutional protocols, or prior AI performance. The authors should discuss how static AI reliance might oversimplify human–AI interaction, and whether the outcomes would differ under more adaptive or heterogeneous reliance models.\nAR: We agree that this is a very important point that deserves more discussion. We have incorporated the following additional discussion regarding this point into the revised version of the paper, which also integrates a similar comment made by R3 regarding how humans working with AI may also introduce bias:\n“In this work, we included this radiologist reliance parameter to explore how factors external to the direct technical system can strongly influence outcomes associated with that system. However, we modeled a static level of AI reliance, which has limitations. Namely, this approach oversimplifies the relationship between human decision-makers and AI systems, where, in reality, it is likely to be more dynamic and evolving, leading to different outcomes. For instance, a radiologist may realize over time that an AI system has lower performance for specific subgroups, and consequently internally calibrate their diagnoses with this knowledge in mind.3,23 On the other hand, humans could also introduce their own biases into clinical decision-making processes. Here, we model an “unbiased” radiologist, but the influence of cognitive racial and gender bias on patient care has also been widely documented.49,50 While previous studies have shown that humans can be influenced by biased AI systems51,52, the degree to which AI systems could conversely temper human implicit biases has not been widely explored so far. Altogether, this demonstrates that further research into the interaction of biases between AI and human users in shared decision-making settings is warranted. ”\nR1: 11. The finding that increasing screening access significantly reduces mortality disparities is compelling but may be sensitive to model assumptions. The authors should clarify how screening parameters (e.g., effectiveness, risk progression) were chosen and whether they reflect real-world data. Additionally, the observed trade-off with increased socioeconomic disparity deserves further discussion and ethical framing.\nAR: In addition to clarifying the parameters relevant to the logistic regression models as mentioned in our response to comment #8 above, we have clarified and emphasized the limitations of the assumptions we make with respect to treatment efficacy and risk progression.\n“A primary limitation of this work is that various assumptions and simplifications were made, such that our simulation does not comprehensively model the real-world breast cancer screening and treatment processes. For instance, we modelled a simplified approximation of clinical decision-making based on a mortality risk factor that abstractly represented cancer severity, which did not consider specific patient-level physiological characteristics, such as tissue density, family history, or cancer subtype. We also assumed that all stages within the simulation occur within a single time step (one year), and computed population outcomes per time step. In reality, while all stages from screening to treatment could occur within one year, delays incurred can prolong this process and lead to worse outcomes.58 In our simulation, we modeled the impact of treatment delays by increasing mortality risk by a factor corresponding to half of a year of disease progression, but assumed that all stages were completed in a year regardless of delays. We also assumed that individuals who were successfully treated became cancer free (with the same likelihood of getting cancer again as the rest of the population), only women received screening, and socioeconomic index, which was intended to encompass various complex socioeconomic-associated health risk factors, was only impacted by undergoing treatment. Importantly, the two groups we modelled were not intended to represent any particular demographic group in the U.S. or elsewhere, but were a broad abstraction of populations with differential levels of access to appropriate healthcare. The parameters we used were selected based on various literature sources (see Methods) that investigated disparities relevant to different factors, such as income level, race, and insurance status, which are subject to the limitations from those respective studies. These various assumptions and simplifications in our simulation, which do not exhaustively reflect the real world, mean that the specific numerical outcomes are unlikely to be directly applicable to data from real clinical populations. For instance, in our simulation, the absolute numbers for mortality rates are much higher than real-world values for the U.S. reported by SEER (simulation, Naive: 91.04 [90.92, 91.16] per 100k, SEER: 19.2 per 100k59). However, we emphasize that the primary goal of this work was to compare the outcomes of algorithmic fairness scenarios relative to each other by modeling the healthcare process just well enough such that differences in outcomes between the two groups can be quantified. Therefore, we believe that this simulation did not need to capture every detail of the screening and treatment process, as we drew on clinical expertise and established literature to simplify the process into components that were pertinent to our analysis.”\nWe have also expanded the discussion of the trade-offs related to socioeconomic index disparities:\n“This trade-off was also reflected in the sensitivity analysis, where reduced mortality disparities resulting from higher screening rates were observed at the expense of a higher socioeconomic index disparity, due to a higher assumed financial burden of breast cancer treatment for those in the unprivileged group.45,46 In addition to existing economic barriers and hardships that may be faced by lower-socioeconomic status individuals (“should I delay or forgo treatment, or go into medical debt?”), there is a risk of AI-assisted mammography analysis being associated with higher out-of-pocket costs not covered by insurance.47 Thus, even if AI models are “algorithmically fair”, they may still pose a higher barrier to under-resourced populations, therefore directing benefits of these systems toward more socioeconomically privileged populations.”\nR1: (Remarks on code availability): The provided GitHub link appeared to be empty at the time we checked so we couldn’t review the code.\nAR: (Copied response to comment #7): Our code was attached as a separate zip file at initial submission, which should be available in the reviewer portal. We have also provided an updated code file reflecting the revised experiments. The GitHub repository will be updated with the final version of the code upon acceptance.\nReviewer #2 (R2): I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.\nAR: We would like to thank the reviewer again for their contribution to improving our manuscript.\nReviewer #3 (R3): This study addresses an important question: how algorithmic fairness metrics translate into real-world health outcomes when AI is integrated into clinical care. It shows that equalizing true positive rates (TPR) between groups reduces mortality gaps more effectively than adjusting false positive rates (FPR) or applying equalized odds. It also finds that improving access to screening has a much stronger impact on health disparities than any adjustment to the AI model itself. Overall, the results highlight that improving fairness in clinical AI requires addressing broader healthcare system factors, not just model performance. The simulation approach presented is innovative and thoughtfully designed. I believe the framework is generally strong; however, it suffers from conceptual gaps, lack of validation against real-world clinical data, and some issues with assumptions and interpretation of the findings.\nAR: We appreciate the positive comments from the reviewer. As mentioned in our responses to R1 above, we have attempted to clarify the primary aim of this paper throughout the manuscript. We would like to highlight that this work is not intended to be a clinical modelling study, but rather an exploratory simulation of the degree to which notions of algorithmic fairness can impact health disparities using the case example of AI-assisted breast cancer screening. Practically, we have revised sections of the abstract and introduction, as well as modified the title to address this:\nTitle:\n“Connecting algorithmic fairness and fair outcomes: A sociotechnical simulation case study of AI-assisted breast cancer screening”\nAbstract:\n“We modeled two hypothetical populations who experienced different levels of access to healthcare, and analyzed how four different definitions of algorithmic fairness impacted mortality rates and socioeconomic disparities over time.”\n“This work aims to explore the degree to which algorithmic fairness may translate to fair and equitable outcomes in clinical populations, with relevance far beyond this use case of AI-assisted breast cancer screening.”\nIntroduction:\n“In contrast to studies which aim to accurately model real-world sociodemographic populations to simulate clinical outcomes, this work aims to gain a high-level understanding of how algorithmic fairness may interact with healthcare system components and existing disparities.”\nWe hope that we have been able to adequately address the remainder of the reviewer’s concerns in the revisions detailed below.\nR3: The simulation results generally support the overall direction of the conclusions; However, some of the claims are stated too strongly given the limitations of the study design. For example, the finding that increasing screening access has a greater impact than model fairness interventions is important but may not reflect practical feasibility, as the model does not account for real-world barriers such as accessibility, insurance coverage, and trust in the healthcare system.\nAR: We have made a strong effort to use more precise language regarding our aims and to soften language throughout the updated discussion and conclusion sections, specifically to convey how the results of this work are not intended to be construed as directly applicable to real-world outcomes, but as an examination of the degree to which algorithmic fairness can effectively address health equity challenges.\nAs part of this, we have also revised the Results section subheadings to be descriptive of the analyses performed rather than the results of the simulation, to avoid misinterpretation of the findings:\nLarge mortality rate disparities were observed regardless of algorithmic fairness constraints Comparison of outcome disparities over time for different algorithmic fairness constraints\nAI system true positive rates were a primary driver of outcome disparities\nAssessment of the effect of AI system group-wise performance differences on outcomes\nDegree of radiologist reliance on AI decisions strongly impacted mortality rate disparities Evaluation of the impact of radiologist reliance on AI decisions\nScreening access had the largest impact on mortality rate disparity Sensitivity analysis of non-algorithmic healthcare system interventions\nWe would like to note that the numerous barriers are intended to be implicitly included within the construct of socioeconomic index, which functions as the primary determinant of access to appropriate healthcare within our breast cancer screening example. We have also clarified this aspect in the sections described below:\nOverview:\n“The socioeconomic index of an individual in the population was a single value that represented an aggregate measure of factors related to financial accessibility, insurance status, healthcare policy impacts, income level, and access to healthcare-related resources.”\nMethods:\n“The x of each individual in the population is a single value representing an aggregate measure of socioeconomic status, which in reality could include factors such as household income, insurance coverage, healthcare policy impacts, education level, occupation, and access to resources.73”\nR3: The methods are generally sound and clearly described, and the availability of code allows full reproducibility. I did not find major errors in the data analysis itself. However, there are inconsistencies between the modeling assumptions and the interpretation of the findings. For example:\n Simulated mortality rates presented in Results, and Figure 2 appear substantially higher than actual U.S. rates without any comparison to empirical data. I suggest validating the simulated mortality rates against SEER or CDC datasets, or at least discussing the discrepancy and its implications for interpreting the findings.\nR3: • Page 5, Line 142: \"We modeled a simplified version of the real-world breast cancer screening and care process...\"\nIt appears that the model assumes screening, diagnosis, treatment, and outcomes all occur within a single annual time step, ignoring real-world delays that are known to affect outcomes. I recommend incorporating a sensitivity analysis introducing plausible delays, or explicitly discussing how this simplification might bias mortality disparity findings.\nAR: We would like to address the above two comments together, as we have integrated both points raised by the reviewer in an expanded discussion of our limitations, which also includes points raised by R1. Here, we expand on the various assumptions made for the purpose of our simulation (including the single-year time step) and how they may impact interpretation of results. We agree that it is important to discuss discrepancies between mortality rates in our simulation compared to the real world. Therefore, we have compared our results to SEER-reported values and expanded on why these discrepancies may exist, as recommended by the reviewer. We hope that this modification, in conjunction with the various places throughout the paper where we have emphasized the simplifications of our simulation and intention for it to not be a clinical modelling study, clarifies interpretation of the results.\n“A primary limitation of this work is that various assumptions and simplifications were made, such that our simulation does not comprehensively model the real-world breast cancer screening and treatment processes. For instance, we modelled a simplified approximation of clinical decision-making based on a mortality risk factor that abstractly represented cancer severity, which did not consider specific patient-level physiological characteristics, such as tissue density, family history, or cancer subtype. We also assumed that all stages within the simulation occur within a single time step (one year), and computed population outcomes per time step. In reality, while all stages from screening to treatment could occur within one year, delays incurred can prolong this process and lead to worse outcomes.58 In our simulation, we modeled the impact of treatment delays by increasing mortality risk by a factor corresponding to half of a year of disease progression, but assumed that all stages were completed in a year regardless of delays. We also assumed that individuals who were successfully treated became cancer free (with the same likelihood of getting cancer again as the rest of the population), only women received screening, and socioeconomic index, which was intended to encompass various complex socioeconomic-associated health risk factors, was only impacted by undergoing treatment. Importantly, the two groups we modelled were not intended to represent any particular demographic group in the U.S. or elsewhere, but were a broad abstraction of populations with differential levels of access to appropriate healthcare. The parameters we used were selected based on various literature sources (see Methods) that investigated disparities relevant to different factors, such as income level, race, and insurance status, which are subject to the limitations from those respective studies. These various assumptions and simplifications in our simulation, which do not exhaustively reflect the real world, mean that the specific numerical outcomes are unlikely to be directly applicable to data from real clinical populations. For instance, in our simulation, the absolute numbers for mortality rates are much higher than real-world values for the U.S. reported by SEER (simulation, Naive: 91.04 [90.92, 91.16] per 100k, SEER: 19.2 per 100k59). However, we emphasize that the primary goal of this work was to compare the outcomes of algorithmic fairness scenarios relative to each other by modeling the healthcare process just well enough such that differences in outcomes between the two groups can be quantified. Therefore, we believe that this simulation did not need to capture every detail of the screening and treatment process, as we drew on clinical expertise and established literature to simplify the process into components that were pertinent to our analysis.”\nR3: There are also several important assumptions that need to be fully discussed or justified, including:\n Line 171: \"Here, we assumed the radiologist to be 'unbiased'.\"\nModeling the radiologist as unbiased oversimplifies human decision-making, especially given documented studies on cognitive and institutional biases in healthcare. This assumption may have a strong impact on the interpretability of the outcome and practicality of the model in the real-world clinical settings.\nAR: We agree that the static, unbiased nature of the human decision-maker is highly simplified compared to the real world. Thus, we have expanded the discussion on this to address this point as well as comments made by R1, as follows:\n“In this work, we included this radiologist reliance parameter to explore how factors external to the direct technical system can strongly influence outcomes associated with that system. However, we modeled a static level of AI reliance, which has limitations. Namely, this approach oversimplifies the relationship between human decision-makers and AI systems, where, in reality, it is likely to be more dynamic and evolving, and lead to different outcomes. For instance, a radiologist may realize over time that an AI system has lower performance for specific subgroups, and consequently internally calibrate their diagnoses with this knowledge in mind.3,23 On the other hand, humans could also introduce their own biases into clinical decision-making processes. Here, we model an “unbiased” radiologist, but the influence of cognitive racial and gender bias on patient care has also been widely documented.49,50 While previous studies have shown that humans can be influenced by biased AI systems51,52, the degree to which AI systems could conversely temper human implicit biases has not been widely explored so far. Altogether, this demonstrates that further research into the interaction of biases between AI and human users in shared decision-making settings is warranted.52”\nR3: • Lines 182–184: \"It was assumed that imposing error rate parity was achieved through fair representation learning...\"\nAssuming fairness by reducing performance for the initially higher-performing group raises ethical concerns, particularly in clinical settings where non-maleficence is a core principle. I encourage the authors to discuss alternative bias mitigation strategies and better justify the ethical trade-offs involved.\nAR: We thank the reviewer for raising this point as an important aspect that deserves more discussion. In response, we have justified why we chose to model this approach in the Methods, and expanded on ethical implications and alternatives in the Discussion:\nMethods:\n“This can be the case with fair representation learning/“in-processing” bias mitigation methods,20,22,60 which have been commonly proposed in the literature for algorithmic fairness in healthcare setting.8,11,15”\nDiscussion:\n“Additionally, the specific results of this simulation are contingent on our assumption of fair representation learning methods being applied to achieve error rate parity between groups, which lowered group-specific performance. While this “leveling down” can be a side effect of methods commonly developed and applied within medical contexts,60 reducing performance for a subpopulation represented in a dataset has been widely noted to be contradictory to the medical ethics principle of non-maleficence.22,61 The results of our simulation provide a bigger picture, specifically showing that leveling down TPR would practically translate to a higher mortality rate for that group. However, we assume here that the Naive model is able to faithfully capture features associated with malignant breast cancer in mammograms, when in reality, AI models trained on medical image data are susceptible to learning spurious correlations, which could result in artificially inflated performance.62 Thus, leveling down may not always be inherently negative from a clinical perspective, if it means that the model is encouraged to learn generalizable and reliable representations of disease.63 Regardless, algorithmic fairness methods that aim to improve worst-group performance while keeping performance for initially higher-performing groups constant60,64 may be more aligned with medical ethics. However, we note that in our simulation, the specific algorithmic fairness scenario had a relatively small impact on outcomes relative to the absolute value of group-wise disparities that existed due to the other barriers we modelled within the healthcare system.”\nR3: • Line 429: \"Ensuring that 92% of the unprivileged group received annual screening reduced this value by approximately 52 individuals per 100k...\"\nThis is a strong claim, especially given that achieving such high screening uptake faces complex socioeconomic challenges (e.g., access, insurance, trust). I suggest tempering the claim and more explicitly discussing these challenges.\nAR: We agree that this result comes with significant limitations and is not necessarily a practical solution. The intention of this aspect of the work was to perform a sensitivity analysis of possible other, non-technical interventions that could be more effective at reducing disparities. We have incorporated an additional discussion around this comment, which includes points brought up by R1. The revised section is as follows:\n“Specifically, enforcing equalized odds only reduced the mortality rate disparity by approximately 12 individuals per 100k, while on the other hand, ensuring that 93% of the unprivileged group received annual screening reduced this value by approximately 57 individuals per 100k, without increasing the mortality rate for the privileged group. In practice, however, increasing access to screening, especially to the high levels simulated in our sensitivity analysis, is an extremely complex task. Systemic challenges, including cultural barriers, geographical limitations, and financial accessibility, mean that there is no simple solution to achieving perfect screening uptake. As an example, in the largely privatized healthcare system of U.S., which our simulation was loosely modeled after, the Affordable Care Act introduced measures intended to reduce disparities in access to preventive screening, such as for breast cancer, by eliminating out-of-pocket costs for patients.53 However, this legislation does not guarantee universal access to workup following mammography screening, meaning that disparities remain since some patients still must pay higher out-of-pocket costs for follow-up imaging and diagnostic procedures.54 Thus, the interplay of health policy and medical economics plays a crucial role in ensuring that initial barriers to access — before interaction with an AI system even begins — are effective in reducing health disparities. This highlights that even if AI systems are “fair”, the role they play in impacting health equity can be largely influenced by external social, economic, and political factors.”\nR3: • Line 444: \"We modeled our simulation after the healthcare system in the U.S.\"\nTo improve generalizability, it would be helpful to briefly discuss how the model structure could be adapted to different healthcare systems and settings.\nAR: We have expanded our discussion of the adaptation of our simulation to other healthcare systems as follows:\n“We loosely modeled our simulation after the healthcare system in the U.S., where existing disparities in breast cancer screening access, treatment, and mortality are well documented.37–41 However, even in universal healthcare systems, disparities can be prevalent, particularly in screening uptake. For instance, in Canada, factors associated with lower mammography usage also include lower socioeconomic status, geographical remoteness, ethnic minorities, and immigration status.55 Studies from European countries also revealed lower rates of screening associated with area-level socioeconomic deprivation.56 In a study from Taiwan, another universal healthcare system, top reasons for not getting screened included logistics, lack of family history, and discomfort.57 Therefore, adapting our simulation approach to other contexts would require the consideration of region-specific patient-level characteristics (e.g., relating to health policy, socioeconomic variation, and cultural norms) and healthcare systems (e.g., fee-for-service physician models, referral waiting times, and public health campaigns). AI policy and the degree to which AI systems are regulated and integrated into health systems would also be an important consideration for expanding this simulation framework to include different medical institutions, perhaps serving different populations.”\nR3: • Line 488: \"...Fair AI\" be seen as a magic bullet with the ability to solve complex...\"\nThe analogy of \"magic bullet\" feels out of place here\nAR: This phrase has been removed in the revised conclusion.\nMinor modifications:\nIn revising the manuscript, we realized that Figure 5 could be improved in order to clarify that adjusting the logistic function parameter for each stage represented a change in proportions of both groups. Therefore, we modified the x-axis to correspond to the parameter and overlaid the proportion of each group who underwent screening, delays, or adjuvant treatment. The text was also clarified as follows:\n“Practically, this corresponded to a range of proportions of each group undergoing each of these steps, since the socioeconomic index of an individual dictated disparities with respect to healthcare access and delays.”\nThe revised and original figures are pasted below. While we believe the revised version is more informative, we are open to feedback from the reviewers if they would prefer the original version.\nFigure 5: Average outcomes over 50 in-simulation years for different parameters defining how likely an individual of a certain socioeconomic index was to access screening (left column), experience treatment delays (middle column), and receive adjuvant treatment (right column). P and P represent the priv unpriv proportion of the privileged or unprivileged group, respectively, who proceeded to each stage. Top row (A, B, C): Mortality rates for each group, bottom row (D, E, F): socioeconomic index disparities (difference in mean socioeconomic index score between privileged and unprivileged groups). Error bars representing 95% confidence intervals over 100 simulations are too small to visualize. Only naive and equalized odds scenarios are presented for ease of visualization, with full results for all algorithmic fairness scenarios in Supplementary Tables 3-5.\nFigure 5: Average outcomes over 50 in-simulation years for different parameters defining how likely an individual of a certain socioeconomic index was to access screening (left column), experience treatment delays (middle column), and receive adjuvant treatment (right column), represented here as proportion of the unprivileged group who experienced each of these three stages. Top row (A, B, C): Mortality rates for each group, bottom row (D, E, F): socioeconomic index disparities (difference in mean socioeconomic index score between privileged and unprivileged groups). Error bars representing 95% confidence intervals over 100 simulations are too small to visualize. Only naive and equalized odds scenarios are presented for ease of visualization, with full results for all algorithmic fairness scenarios in Supplementary Tables 3-5.\nCorrected typo in Methods – Screening:\n“Based on these findings, individuals in our simulation who received a false positive diagnosis have the probability of receiving a screen in the following time step reduced by 10%.”\nCorrected typo in Equation (12):\n𝑝 𝑑𝑒𝑙𝑎𝑦 = 1 / (1 + 𝑒 𝑑(0sEl12d))\nCorrected notation in Equations 4, 7, 10, 11:\n𝑝 𝑠𝑐𝑟𝑒𝑒𝑛 = 1 / (1 + 𝑒 𝑠(0sEl12s))\n(4)\n=𝑝 • + (1𝑥 )• , 𝑒 h E [0,1], (7)\n𝑝 𝑑𝑒𝑙𝑎𝑦 = 1 / (1 + 𝑒 𝑑(0sEl12d)) , (10)\n𝑝 𝑎𝑑𝑗𝑢𝑣𝑎𝑛𝑡 = 1 / (1 + 𝑒 𝑡(0sEl12t)) , (11)\nAdded missing acknowledgements:\n“This work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC), the Killam Trusts, Alberta Innovates, the Canada Research Chairs program, the River Fund at Calgary Foundation, and the Canadian Institutes of Health Research. We would like to thank Dr. Elise Fear for her valuable suggestions.”\nWe have also updated the code to be more modular for improved readability and re-use."}, {"round": 2, "reviewer_comments": {"reviewer_2": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.", "reviewer_3": "The authors revision has significantly improved the manuscript. That said, I believe a few minor revisions would further clarify and improve the paper:\nMinor comments:\n-To improve generalizability, I suggest the authors expanding on how the simulation model could be adapted to health systems beyond the U.S.\n-Add a note or reference in the caption of Figure 2 to help readers interpret how the simulated mortality rates compare to real-world benchmarks (e.g., SEER or CDC) data) \n-Clearly acknowledge the ethical trade-offs of fairness strategies that reduce performance for the privileged group, and briefly reference alternative approaches that aim to preserve or improve care across all groups."}, "author_response": "Author’s Response to Reviews\nConnecting algorithmic fairness and fair outcomes: A sociotechnical simulation case study of AI-assisted breast cancer screening\nReviewer #3 (R3): To improve generalizability, I suggest the authors expanding on how the simulation model could be adapted to health systems beyond the U.S.\nWe have expanded on this point as follows in the revised paper:\n“We loosely modeled our simulation after the healthcare system in the U.S., where existing disparities in breast cancer screening access, treatment, and mortality are well documented.37–41 However, even in universal healthcare systems, disparities can be prevalent, particularly in screening uptake. For instance, in Canada, factors associated with lower mammography usage also include lower socioeconomic status, geographical remoteness, ethnic minorities, and immigration status.56 Studies from European countries also revealed lower rates of screening associated with area-level socioeconomic deprivation.57 In a study from Taiwan, another universal healthcare system, top reasons for not getting screened included logistics, lack of family history, and discomfort.58 Therefore, adapting our simulation framework to new contexts would require careful consideration of region-specific and patient-level characteristics, for example, differences in health policy environments, socioeconomic conditions, and cultural practices that shape and influence healthcare engagement. Equally important are features of the healthcare system itself, such as physician payment models (e.g., fee-for-service), referral pathways and waiting times, and the influence of public health initiatives or campaigns. Moreover, the landscape of AI governance, including the extent to which AI technologies are regulated, trusted, and integrated into clinical practice, would be a critical factor. These considerations become especially salient when extending the simulation to different medical institutions that may serve diverse populations with distinct healthcare needs and systemic constraints.”\nR3: Add a note or reference in the caption of Figure 2 to help readers interpret how the simulated mortality rates compare to real-world benchmarks (e.g., SEER or CDC) data)\nThe caption has been updated as follows in the revised paper:\n“Figure 2: Outcomes for each of the four algorithmic fairness scenarios (Naive, EqOppFPR, EqOppTPR, EqOdds) over 50 in-simulation years. A: Annual mortality rate for each group, B: Annual mortality rate ratio (proportion of the unprivileged group to the proportion of the privileged group who died of breast cancer in a given year), C: Annual average socioeconomic index (SEI) disparity (difference in mean socioeconomic index score between privileged and unprivileged groups in a given year). Error bars represent 95% confidence intervals over 100 simulations. Note that these results are unique to the simulation, and do not represent real-world reported breast cancer mortality (e.g., Surveillance, Epidemiology, and End Results (SEER) Program of the National Cancer Institute reported 2019-2023 mortality rate=19.2 per 100k59). Source data are provided as a Source Data file.”\nR3: Clearly acknowledge the ethical trade-offs of fairness strategies that reduce performance for the privileged group, and briefly reference alternative approaches that aim to preserve or improve care across all groups.\nWe have expanded on this point as follows in the revised paper:\n“Additionally, the results of our simulation are closely tied to the assumption that fair representation learning methods were applied to achieve error rate parity across groups, which reduced group-specific performance. While this “leveling down” can be a side effect of methods commonly developed and applied within medical contexts,60 reducing performance for a subpopulation represented in a dataset has been widely critiqued as being contradictory to the medical ethics principle of non-maleficence, which emphasizes avoiding harm to patients.22,61 Our simulation further highlights this ethical conflict by demonstrating that lowering the TPR for a subgroup would practically translate to an increased mortality burden for that population. However, we assume here that the Naive model is able to faithfully capture features associated with malignant breast cancer in mammograms, when in reality, AI models trained on medical image data are susceptible to learning spurious correlations, which could result in artificially inflated performance.62 Thus, leveling down may not always be inherently negative from a clinical perspective, if it means that the model is encouraged to learn generalizable and reliable representations of disease rather than exploiting dataset-specific artifacts.63 Regardless, algorithmic fairness methods that aim to improve worst-group performance while keeping performance for initially higher-performing groups constant (e.g., distributionally robust optimization64, Pareto fairness65) may be a preferable strategy from a medical ethics standpoint. However, it should be highlighted that our simulation illustrated how specific algorithmic fairness interventions had a relatively small impact on outcomes relative to the absolute value of group-wise disparities that stemmed from structural barriers embedded in the healthcare system. This suggests that while algorithmic fairness methods are important for addressing statistical limitations of models, addressing systemic inequities more broadly remains central to reducing group-wise disparities in clinical outcomes.”"}]

Decision letter