GlyMan: Glycemic Management using Patient-Centric Counterfactuals

Authors: asiful  arefeen
saman  khamesian
maria adela grando
bithika  thompson
hassan  ghasemzadeh
Venue: IEEE_org_EMBS_BHI_2024_Conference
Published: 2024-07-04 23:12:55

Abstract: Prolonged and frequent exposure to elevated blood glucose levels (hyperglycemia) significantly increases the likelihood of developing chronic complications, such as neuropathy, nephropathy, and cardiovascular disease, along with acute symptoms like fatigue and blurry vision. While current technologies, such as continuous subcutaneous insulin infusion (CSII) and continuous glucose monitors (CGMs), can forecast adverse events like hypoglycemia and deliver small insulin doses to counteract hyperglycemia, progress in developing tailored AI-driven interventions remains limited, which poses a barrier to optimal diabetes care. To address this gap, we propose leveraging counterfactual explanations that guide patients in making targeted adjustments to their carbohydrate intake and insulin dosing to avoid abnormal glucose levels. We introduce *GlyMan*, a novel method that generates counterfactual behavioral recommendations aimed at helping patients and caregivers make small, informed changes to prevent hyperglycemia, thus substantially reducing both its frequency and duration. Additionally, GlyMan incorporates user preferences into its intervention process and ensures more customized and patient-centered guidance. We rigorously evaluated GlyMan using real-world data from 21 type 1 diabetes (T1D) patients using automated insulin delivery (AID) systems. Results indicate that GlyMan surpasses existing methods, delivering 76.6% valid explanations and 86% effectiveness when assessed against historical data.

Main paper: 
 i. introduction: 
 Postprandial hyperglycemia, defined as blood glucose levels above 180 mg/dl (10 mmol/l) two hours after a meal, affects 22% to 46% of non-critically ill hospitalized patients [1] . Chronic hyperglycemia can lead to severe complications, including retinopathy, kidney failure, neuropathy, and cardiovascular diseases [2] . Additionally, diabetes imposes a high economic burden, with an average annual treatment cost of $12,022 per patient and an estimated total cost of $412.9 billion in the U.S. in 2022 [3] . These costs can be mitigated through behavioral modifications, such as maintaining a proper diet [4] and adhering to optimal medication.For individuals with type 1 diabetes (T1D), insulin therapy is essential due to the body's inability to produce insulin. Managing optimal insulin dosing is complex and requires constant decisions regarding food intake and insulin administration. Therefore, despite advancements in automated insulin delivery (AID) systems, few T1D patients achieve recommended glycemic targets. While AI offers potential to improve glycemic control, its use in predicting and preventing hyperglycemia and hypoglycemia in T1D patients on AID systems remains limited. AI driven interventions to target dysglycemia have potential to improve glycemic control and reduce the burden of disease in patients with T1D [5] , [6] .Explainable AI (XAI) presents an opportunity to design better interventions by providing transparency in model decisionmaking [7] . Traditional XAI techniques, such as LIME, TIME, and SHAP, primarily focus on ranking important features. Feature relevance has proved to be important in building trust in a model that works with computer vision [8] , time-series [9] or even tabular data [10] but may lack actionable insight for behavioral interventions. For instance, while understanding the relevance of features in diabetic retinopathy prediction can be useful in healthcare [9] , practical interventions in digital health require more precise and actionable information. Often, these explanations are provided in view of low-level features that are hardly understandable from a human perspective, which undermines the main objective of XAI.Counterfactual explanations (CFs) are more effective for generating actionable insights, as they suggest the smallest feature changes needed to achieve a desired outcome. For example, a CF might suggest preventing hyperglycemia by reducing carbohydrate intake or delaying a meal until blood glucose reaches a certain threshold. CFs can either be derived from training data [11] or generated using adversarial techniques [12] . Studies have demonstrated CFs' potential in diabetes management, improving fasting blood sugar, systolic blood pressure, and other health metrics [13] . However, most CF applications in diabetes research overlook patient preferences, often leading to unrealistic or infeasible recommendations.In this regard, our solution, GlyMan, stands out by incorporating stakeholders' (e.g., patients, physicians) preferences, such as feature importance, into the CF generation process. This ensures that patients are playing a role in the interventions by specifying certain behaviors should remain unchanged. GlyMan's key contributions include:• A novel, model-agnostic, patient-centric algorithm that generates CF-based interventions to prevent postprandial hyperglycemia through behavior changes (e.g., adjusting meal timing, carbohydrate intake, insulin dosage). Assume that D = {(X 1 ,y 1 ), (X 2 , y 2 ), . . . , (X n , y n )} be a dataset of n instances that has longitudinal health observations related to eating events and the corresponding health outcome such as blood glucose level categories. Each instance X i = [x i 1 , x i 2 , . . . , x i d ]f : R d → [1, c]For a test sample X T predicted to result in post-prandial hyperglycemia (i.e., argmaxf (X T ) = hyperglycemia), Gly-Man aims to create an effective intervention plan that guides the patient in making informed behavioral adjustments to prevent the predicted hyperglycemia, while also respecting their personal preferences.To generate such CFs, we have to satisfy several constrains within an optimization process. For example, the CFs must belong to the desired class, must not change too much from the factuals and must reflect user preferences. We assume that the stakeholder's preferences for behavior changes are represented in vector R(X T ) = {r 1 , . . . , r d }, where each r i ∈ [0, 1] represents the relative importance of the i-th feature for modification during intervention. Specifically, a value of r i = 1 indicates that the stakeholders strongly favor modifying the i-th feature, while r i = 0 implies no preference for modification.We formulate the CF generation process using a multiobjective optimization problem as shown in Equation ( 1), where the aforementioned requirements are formalized in the first to third terms, respectively. minEQUATIONHere, CE(•) is the crossentropy loss between model's prediction on the CF and normoglycemia, d(•) is the distance function.To tackle the optimization problem (1) through adversarial perturbation, we use an iterative process that modifies the features of X T incrementally by δ. The adjustments are guided by saliency scores, stakeholder preferences, and the requirement to maintain realistic changes within bounds.A primary goal of CF generation is to introduce minimal changes to the original samples. Feature saliency measures how altering a particular feature affects the model's prediction for the target class. Identifying the most salient feature during each iteration allows GlyMan to determine which feature, when adjusted, will have the greatest impact on shifting the prediction toward the desired outcome. This is done by computing the forward derivative of the model's prediction with respect to each feature.For each modifiable feature in x mod , the saliency score S(x T , y ′ , i) is calculated by perturbing the feature value by a small amount and observing the change in the model's prediction probability for the target class. This change is captured through the forward derivative of the prediction with respect to the feature,S(x T , y ′ , i) = f n (x * i T + δ i ) -f n (x * i T ) δ i ∀x * i T ∈ x mod (2)By combining feature saliency with stakeholder preference weights, a composite score is calculated to identify which feature to adjust. Specifically, the combined score C i for each feature x i T is obtained by adding the normalized saliency score S(x * T , y ′ , i) (scaled to the range [-1, 1]) to the sum of the physician's and user's preference weights (w p and w u ). This score is then used to determine which feature should be modified i.e., the feature with highest composite score is selected for adjustment within the feature limits. Therefore, the index of the feature to modify isi ′ = arg max i |S(x * T , y ′ , i) ′ | + (w p + w u )Figure 2 explains the overall framework for GlyMan.A substantial dataset was collected from 100 T1D patients treated at the endocrinology department of Mayo Clinic, Phoenix between December 2023 and April 2024 (IRB #23-003065). Each patient provided around 26 days of continuous recordings in real-world settings, including CGM data from Dexcom G6 Pro, insulin administration logs, carbohydrate intake, and device mode settings (e.g., regular, sleep, exercise) Fig. 2 : Pipeline: The process begins with collecting data from T1D patients on AID technology, followed by data preprocessing and classifier training. Afterward, GlyMan is initialized with the specified preference weights and iterations are run to generate the counterfactual outcomes. from Tandem T:SLIM X2 Pump. Of these, 21 randomly selected patients (Age: 57.4 ± 16.2 years, 11 female, A1C: 5.0-8.2%, 20 White, 1 Hispanic) contributed data that was processed further to develop and validate GlyMan.1) Basal rates and device modes: The hourly basal rates and device modes in the PDF files downloaded from Tandem are extracted by cropping the informative areas and then using an Optical Character Recognition (OCR) technique.2) Time between meal and food bolus, ∆t: Improving ∆t may play a key role in improving glycemic control. Following prior research [14] , we estimated ∆t from the timeseries data3) Total bolus: The total bolus is the sum of all bolus intakes taken between the time when the glucose level peaks (t max ) and the time when either the meal or the first bolus was taken, whichever occurred first, i.e., min(t meal , t f b ).4) Total basal: Sum of all basal units taken between t meal -90 and t meal falls under Total basal feature.5) Pre-meal glucose level and slope: The CGM reading at t meal is the pre-meal blood glucose level. A linear trend-line is fitted using glucose readings between t meal -30 and t meal to calculate the pre-meal slope.6) Filtering out carb sizes: Patients often try to manage high blood sugar by taking extra food boluses instead of correction boluses. This can result in multiple carb intakes near the primary meal. In such cases, we consider only the largest carb size and ignore the others that occur between min(t meal , t f b ) and t max .Finally, we have 1361 factual samples after data curation. Two of them are depicted in Table I as examples.The fully-connected binary classifier for hyperglycemia classification is of 3 layers with 64, 32 and 32 neurons, respectively. Each layer has a relu activation and dropout rate of 0.4. The model is trained using 85% factual samples for 400 epochs with a learning rate of 0.001.For preventing hyperglycemia, we set target class y ′ = normoglycemia, the corresponding prediction confidence (γ) at 0.6, maximum iterations to N = 200 and consider Carb size, Total bolus, ∆t, and Pre-meal BGL as the modifiable features. Their corresponding perturbation size, δ values are 5 grams, 0.5 unit, 5 minutes, and 10 mg/dL, respectively.We assess the CFs using some standard metrics found in the literature:Validity assesses whether the produced CFs genuinely belong to the desired class. High validity indicates the technique's effectiveness in generating valid CF examples.Nearest Neighbor Test (NN Test) validates the effectiveness of the CFs by comparing them against historical data to determine their likely outcomes (e.g., hyperglycemia or normoglycemia) based on past similar instances. We implement it using a k-nearest neighbor (k-NN) algorithm.Proximity is the L 2 norm distance between X T and X T * . A low Proximity ensures we are making small change to the factual sample by preserving the details and not overcorrecting the user.Sparsity is the average number of feature changes per CF. A low sparsity ensures better user understanding of the CFs.Violations quantifies how frequently non-modifiable features (e.g. age, gender, insulin etc.) are changed.Plausibility estimates the fraction of explanations that fall within the feature ranges derived from the data.The simulator is an XGBoost model trained with real data to validate the CFs. With a max-depth of 13, learning rate of 0.1, 100 estimators and 85% trainig data, the XGBoost simulator achieves 80.14% accuracy.The dense net classifier trained on the Mayo Clinic data achieves 81% prediction accuracy and 80.4% F1-score. The dataset is slightly imbalanced between the two classes.The summary in Table II outlines the quality of CFs generated by GlyMan and NICE [11] . GlyMan achieves an average validity of 0.766, a proximity score of 0.327, 2.34 sparsity and perfect violation and plausibility scores, meaning that it does not modify features which are non-modifiable and all CFs We perform several ablation studies to understand how changing certain parameters of GlyMan impacts different aspects of the produced CFs. Reducing the number of modifiable features leaves less flexibility for GlyMan to operate. In fact, it gets harder to toggle the class by modifying a fewer number of features. When GlyMan is given 4 modifiable features, it can convert all the factual samples into CFs and achieve 100% conversion rate. However, as we reduce the number of modifiable features to 1, average conversion rate drops to as low as 41%. While the conversion rate drops, GlyMan still ensures that the CFs produced are of high quality. Hence, the validity remains close to 0.75 in spite of a dip in conversion rate with just one modifiable feature. Figure 3 depicts the aforementioned analysis. Fig. 3 : Monitoring the changes in validity when the number of modifiable features is varied.We designed GlyMan to actively involve stakeholders in the CF generation process, eliminate the need for an additional generative model, and thereby reduce training complexity. Built on real-world data from uncontrolled environments, GlyMan generates valid, fair, realistic, and minimal CFs. We evaluated and compared these CFs extensively using metrics and methods from past studies. However, GlyMan is not yet clinically validated and faces challenges such as high computational overhead due to iterative processes. Moving forward, we plan to address these limitations and trial GlyMan with real patients in a clinical setting. 
 
 Appendix:  



Reviews: review: title: title: The paper introduces GlyMan, a technique that generates patient-centric counterfactual explanations (CFs) to help manage glycemic levels in patients with type 1 diabetes (T1D). By considering patient and physician preferences, GlyMan aims to prevent postprandial hyperglycemia through personalized behavioral modifications. The paper demonstrates the effectiveness of GlyMan through extensive testing on real-world data from T1D patients, showing improvements in generating valid, effective, and realistic interventions compared to state-of-the-art methods.
paper_summary: paper_summary: proposes GlyMan, a tool designed to generate personalized counterfactual explanations (CFs) that guide patients with type 1 diabetes in making small, actionable behavioral changes to prevent hyperglycemia. The tool integrates patient and physician preferences into the decision-making process to create interventions that are both feasible and aligned with individual needs. GlyMan was evaluated on real data from 21 patients with T1D using automated insulin delivery systems. The results showed that GlyMan outperforms existing methods in terms of validity, effectiveness, proximity, and plausibility of the generated CFs.
strength_weakness: strength_weakness: Abstract is too long

The interventions generated by GlyMan have not yet been tested in clinical settings, which limits the understanding of their real-world impact

GlyMan's iterative approach requires significant computational resources, which may limit its scalability and practical use in resource-constrained settings

Most importantly, authors mention that GlyMan may suggest delaying bolus intakes, which could be counterintuitive or risky without a clear explanation or clinical validation
questions: questions: How do you plan to validate the clinical effectiveness of GlyMan’s interventions in real-world settings, especially considering patient adherence to the suggested behavioral changes?

How does GlyMan handle cases where patient preferences conflict with optimal glycemic outcomes, and how are these conflicts resolved?
title: title: A Patient-Centric Approach to Hyperglycemia Prevention through Counterfactual Behavioral Interventions
paper_summary: paper_summary: The research paper introduces GlyMan, a novel approach designed to generate counterfactual (CF) behavioral interventions for patients with Type 1 Diabetes (T1D). GlyMan focuses on reducing the occurrences and duration of hyperglycemic events by suggesting personalized interventions. The approach considers the preferences of patients and stakeholders, integrating them into the decision-making process to enhance personalization. GlyMan was evaluated on real-world data from 21 T1D patients using automated insulin delivery systems, showing promising results in terms of valid explanations and effectiveness when compared to state-of-the-art methods.
strength_weakness: strength_weakness: The study was conducted on a relatively small sample of 21 patients, which may limit the generalizability of the results.
questions: questions: How scalable is the GlyMan approach for a larger, more diverse population of T1D patients ?
title: title: GlyMan: Glycemic Management using Patient-Centric Counterfactuals
paper_summary: paper_summary: This paper introduces GlyMan, a framework designed to generate counterfactual explanations to help patients with Type 1 Diabetes (T1D) manage their glycemic levels more effectively. The framework is validated using real-world data from 21 T1D patients, demonstrating significant improvements in glycemic management over existing methods.
strength_weakness: strength_weakness: - The iterative, multi-step process of generating counterfactual explanations is computationally intensive, which may limit its scalability and speed in real-world applications. Moreover, a Figure presenting the proposed methodology is missing. This can clarify the overall framework structure. 
- As acknowledged by the authors, the system has not yet been tested in clinical trials, so its real-world effectiveness and patient adherence to the recommendations remain uncertain.
- The study is based on data from only 21 patients, which may not be representative of the broader T1D population. This limits the generalizability of the findings. In addition, the proposed framework was not tested over other existing datasets, such as those used by the mentioned state-of-the-art solutions. 
- A basic dataset content presentation is missing. How many samples are missing, and what is their categorization distribution?
- Can the results be stratified based on gender, age, and years from diagnosis to T1D? 

Further comments: 
- Future versions of GlyMan could benefit from improved visualization tools and user interfaces to enhance patient understanding and engagement with the system.
- While focused on T1D, the underlying methodology of GlyMan could be adapted for managing other chronic conditions, highlighting its potential for broader applications in digital health.
- In Table IV provide references to the methods already published in the literature. E.g., DiCE [ref].
questions: questions: ---