Investigating health disparities and AI bias in models to predict development of chronic kidney disease in patients with Type II Diabetes Introduction ●Chronic kidney disease (CKD) is estimated to affect more than 1 in 7 adults in the US and is more prevalent in non- Hispanic Black adults than in other races/ethnicities. ●One of the most common preventable risk factors for CKD is type 2 diabetes (T2DM), also shown to have higher prevalence in racial and ethnic minorities and low-income populations. ●Machine learning (ML) can help to identify patients at risk of developing CKD who may benefit from early interventions. ●However, ML models have been found to demonstrate bias in their predictive performance, which has the potential to exacerbate existing health disparities or create new ones. AIMS: Investigate health disparities in development of CKD in adult patients with T2DM and develop fair models for predicting CKD, with a focus on race groups. Methods ●Criteria: Adult patients in the OCHIN database from 2012-2021 with a diagnosis of T2DM based on ICD9 and ICD10 codes, documented date of T2DM onset and CKD onset where applicable, and no CKD prior to T2DM. ●Multivariable logistic regression to quantify health disparities ●Random Forest (RF) and XGBoost (XGB) for ML predictive modeling. Table 3. XGB model fairness results Conclusion ●In a population of patients with T2DM, Black and Asian patients had a higher likelihood of developing CKD compared to White patients. ●Predictive models did not perform equally across race groups. ●High performing models may not perform equitably for all patient populations, therefore evaluation and reporting should extend beyond overall performance metrics to include fairness assessment and reporting of any biases in model performance across demographic groups. Ongoing / future work: - time to event analysis to evaluate disparities in progression to CKD; - more comprehensive evaluation of model biases, including gender, age, and intersectional biases; - Incorporate ML explainability methods; - investigate potential approaches for bias mitigation to develop fair models that perform equitably across groups. Mary M. Lucas1, Christopher C. Yang1, Mario Schootman2 1Drexel University; 2University of Arkansas for Medical Sciences This work was conducted with the Accelerating Data Value Across a National Community Health Center Network (ADVANCE) Clinical Research Network (CRN). ADVANCE is a CRN in PCORnet®, the National Patient-Centered Clinical Research Network. ADVANCE is led by OCHIN in partnership with Health Choice Network, Fenway Health, and Oregon Health & Science University. ADVANCE’s participation in PCORnet® is funded through the Patient-Centered Outcomes Research Institute (PCORI), contract number RI-OCHIN-01-MC. This research was, in part, funded by the National Institutes of Health (NIH) Agreement NO. 1OT2OD032581-01. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the NIH. Results Table 1. Patient cohort distribution by race and CKD Table 2. Multivariable logistic regression results for race Figure 1. Predictive models performance Race Overall N = 343,347 No CKD N = 308,933 CKD N = 34,414 White 199,729 (58%) 180,749 (59%) 18,980 (55%) Asian 20,707 (6.0%) 18,152 (5.9%) 2,555 (7.4%) Black/African American 70,525 (21%) 62,392 (20%) 8,133 (24%) Other or Not Reported 52,386 (15%) 47,640 (15%) 4,746 (14%) Race aOR 95% CI p-value White — — Asian 1.10 1.05, 1.16 <0.001 Black/African American 1.39 1.35, 1.44 <0.001 Other or Not Reported 0.96 0.92, 0.99 0.024 FNR Difference FPR Difference White-Black/AA 0.094 (p=3.69e-05) -0.085 (p=1.159e-09) White-Asian 0.118 (p=7.84e-05) -0.108 (p=1.684e-07) Black/AA-Asian 0.024 (p=0.099) -0.023 (p=0.001)