A Predictive Model for Real-time Prediction of Intradialytic Hypotension Based on Machine Learning Algorithms

Authors: yuping  jiang
xueqin  bian
xueming  hong
haiyan  si
di  liu
hao  chen
Venue: KDD_org_2024_Workshop_AIDSH
Published: 2024-06-03 12:03:55

Abstract: Objective. Develop a machine learning-based model to predict IDH using pre-dialysis features. And to continuously predict IDH within the next hour during the dialysis session by incorporating real-time monitoring data. This approach helps in timely intervention, potentially reducing IDH rates and improving clinical outcomes for patients.
Methods. Collected maintenance hemodialysis (MHD) patients from October 1, 2021, to July 31, 2022, and divide them into development and validation datasets based on the treatment time point of May 1, 2022. IDH is defined as follows: (1) Nadir90: intradialytic systolic blood pressure (SBP) < 90 mmHg; (2) Fall20Nadir90: intradialytic SBP < 90 mmHg and a drop of ≥ 20 mmHg from pre-dialysis SBP. Analyzed the model’s predictive performance trained with various machine learning (ML) classification algorithms using k-fold cross-validation, evaluated by plotting the receiver operating characteristic curve (ROC) and precision-recall curve (PRC), calculating the area under the ROC (AUROC) and PRC (AUPRC), and computing the true positive rate (TPR), and true negative rate (TNR).The XGBoost algorithm was used to identify the important features required for the warning models.
Results. Data from 644 patients were analyzed, contributing 61,823 HD sessions with 302,942 intradialytic SBP measurements. IDH occurred in 2,659 (4.3%) HD sessions (Nadir90), in 1,706 (2.76%) sessions (Fall20Nadir90). Among various models compared, XGBoost achieved the best performance for predicting IDH before HD session (TPR: 0.6, TNR: 0.99, AUROC: 0.955, AUPRC: 0.686). Key predictive features included historical minimum SBP, average of historical minimum SBP, current SBP, diastolic blood pressure (DBP), IDH incidence rate, interdialytic weight change rate, prescribed dialysis duration, and dialysis vintage. The real-time model for predicting IDH within the next hour showed a TPR of 0.89, TNR of 0.92, AUROC of 0.959, and AUPRC of 0.38, with additional important features being mean arterial pressure (MAP), dialysis time, and ultrafiltration (UF) changes.
Conclusion. The XGBoost model has a high predictive capability for IDH during an ongoing HD session, assisting healthcare providers in assessing IDH risk and making timely decisions.

Main paper: 
 introduction: 
 Intradyalytic hypotension (IDH) is a common and serious complication during hemodialysis (HD), with significant risk implications. IDH not only affects the quality of life of patients but can also lead to severe cardiovascular complications and even death [1] [2] . According to a study on the mechanisms of IDH [3] , the prevalence of IDH during hemodialysis is approximately 10-12%. Additionally, a recent comparative study [4] on the correlation between IDH and increased mortality evaluated different definitions of IDH and found its incidence 11.19% -21.7% of the sessions, and 32.39% -56.34% of the patients. Traditionally, methods to prevent and treat IDH have been varied [5] [6] , such as reducing the ultrafiltration (UF) rate, avoiding significant interdialytic weight gain, increasing weekly treatment time, adjusting the soac output, impaired vascular resistance, and physiological parameters during HD. However, the rapid advancement of artificial intelligence (AI) and machine learning (ML) in recent years has intdium concentration and temperature of the dialysate, and avoiding food intake during dialysis. These methods mainly rely on manual intervention by healthcare professionals. This approach is not only timeconsuming and labor-intensive but may also lead to suboptimal treatment outcomes due to untimely or inappropriate interventions [7] . Therefore, developing accurate and real-time IDH prediction models is of great significance for improving the prognosis of HD patients.The influencing factors of IDH include blood pressure, weight, hemoglobin level, blood glucose level, excessive UF, reduced cardiroduced new prospects for IDH research [8] [9] , and achieving more accurate and real-time IDH prediction [10] [11] [12] . Lee [7] et al. developed a deep learning model using pre-dialysis clinical variables to predict IDH. Allinovi [13] et al. emphasized the assessment of patients' fluid status before dialysis, using noninvasive imaging techniques to provide crucial predictive information. Zhang [14] et al. focused on utilizing real-time intradialysis data, employing ML algorithms to analyze real-time blood pressure and other dynamic variables during the dialysis process, achieving high predictive accuracy. Li [15] et al. used the dynamic changes in intra-dialysis data, employing advanced optimization algorithms and ML models for real-time prediction. Kim [16] et al. utilized deep learning techniques to perform highprecision analysis on real-time data, with particular attention to processing time-series data. Our experiments have demonstrated that both pre-dialysis information and intra-dialysis data hold significant value in predicting IDH.In this study, we developed an early warning model using predialysis information and incorporated intra-dialysis data to enhance prediction accuracy. And we simplified the model while maintaining predictive accuracy. This approach not only improved the model performance but also ensured real-time prediction capability. 
study population: 
 We conducted a retrospective study on HD patients at the Second Affiliated Hospital of Nanjing Medical University from October 1, 2021, to July 31, 2022. The data from 71,664 HD sessions if 1,274 HD patients. The exclusion criteria for HD patients were: (1) Age < 18 years; (2) Insufficient basic information and clinical history; (3) HD treatment duration less than three months. Exclusion criteria for HD sessions were: (1) No laboratory test data available in the past 3 months; (2) Blood pressure monitoring intervals during dialysis exceeding 1.5 hours; (3) Pre-dialysis blood pressure not recorded (Figure 1 ). In the end, we included data from 49,891 HD sessions of 644 patients. Since this project was conducted as an internal quality improvement initiative for the HD system, it did not undergo institutional review board (IRB) review. We divided the dataset based on the dialysis treatment timestamp on May 1, 2022. The data before this date were used for training and testing the model, while the remaining data were used for validating the model performance. The training and testing sets were randomly split in an 8:2 ratio, and a 5-fold cross-validation was applied.The monitored data of each HD session is automatically saved to the HD system database. Apart from the initial HD script setup, monitored data are collected every 20 seconds from the HD machines, including arterial line pressure (AP), venous line pressure (VP), blood flow rate, dialysate flow rate, UF rate, total UF volume, dialysate temperature, and conductivity. Vital signs, including SBP, DBP, MAP, and pulse rate, are recorded by default every hour and aligned with the time points of SBP measurements from the HD machine. Additional blood pressure measurements are taken when patients complain of any symptoms related to blood pressure abnormalities.We defined IDH events: (1) Nadir90: intra-dialysis SBP < 90 mmHg; (2) Fall20Nadir90: intra-dialysis SBP < 90 mmHg, a decrease in SBP ≥ 20 mmHg compared to pre-dialysis SBP. Each IDH definition was treated as a separate binary outcome.The dataset for this study comprises any data recorded by the HD system during the HD treatment process (Supplementary data, Table S1 ). It includes clinical baseline information, medical history, historical HD sessions, current pre-dialysis examinations, laboratory test results (of each HD treatment in the past three months.), and pre-dialysis vital signs, a total of 120 features.Additionally, we constructed new features from the data, including changes in monitoring data during the last HD session, statistics of IDH events and changes in monitoring data over the past week and the past month, changes in monitoring data during the current HD session, and the relationship between UF volume and body weight and dry weight. Specifically, The additional features and their construction methods are shown in Supplementary (Supplementary data, table S2 )The SBP monitoring frequency during HD sessions is once per hour, allowing us to calculate the risk of IDH in the next hour in real-time.Using both training and testing datasets, we trained and tested models using Multilayer Perceptron (MLP) [17] , K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Random Forest (RF), and XGBoost algorithms. To ensure the robustness of model performance, we employed 5-fold cross-validation to analyze and compare the predictive performance of models on the testing dataset. We selected the algorithms with better performance and comprehensively evaluated and analyzed the predictive performance of the models on the validation dataset.The model evaluation metrics include: positive predictive value, negative predictive value, true positive rate (TPR), true negative rate (TNR), F1 score, AUPRC, and AUROC. Additionally, we plotted ROC and PRC (Table 1 ). The probability that an individual who is actually not at risk is diagnosed as not at risk by the test.Youden Index  1 YI Se Sp   The accuracy index.F1     2 * * / PPV Se PPV Se The harmonic mean of the positive predictive value and the true positive rate.We employ Shapley Additive exPlanation (SHAP) [18] to assess the impact of the model on predicting IDH. This is done to gain a deeper understanding of each variable's contribution to the prediction results. SHAP, as a highly interpretable tool, provides an accurate and transparent method to clearly illustrate the effect of each feature on the predictions across different samples.Through SHAP, we can gain a deeper insight into the logic behind the model's decisions, thereby more accurately predicting the risk of IDH. 
characteristics of hemodialysis sessions: 
 We conducted our study analysis on data from 49,891 HD sessions of 644 patients, with a total of 302,942 intra-dialysis SBP.The demographic statistics are summarized in Table 2 , with patients having a mean age of 58 ± 13 years and a mean dialysis vintage of 93.4 ± 84.6 months. The majority of patients were male (62.9%), and most had comorbidities such as hypertension (72.5%) and diabetes (31.4%). The primary causes of chronic kidney disease (CKD) were hypertensive nephropathy (32.6%), glomerulonephritis (28.7%), and diabetes nephropathy (25.9%). Among the 644 patients, 43.5% experienced at least one episode of IDH, with at least 4.3% of HD sessions resulting in an IDH event. (Supplementary data, table S3 ) The XGBoost model outperformed other algorithm models in terms of AUROC (Nadir90: 0.939, Fall20Nadir90: 0.924) (Table 3 and Supplementary data, figure S1 ). Finally, this model was selected for statistical analysis of its predictive performance on the validation set. The model performance on the validation set: (1) Nadir90: an AUROC of 0.955 [95% CI 0.947-0.962] (Figure 2A ), with an AUPRC of 0.686 (Supplementary data, figure S2A ); (2) Fall20Nadir90: an AUROC of 0.933 [95% CI 0.922-0.945] (Figure 2B ), with an AUPRC of 0.440 (Supplementary data, figure S2B ).For Nadir90, Following discussions with healthcare professionals, a 10% false positive rate is considered acceptable [7] . Therefore, we selected the threshold of 0.05, corresponding to a TNR of 0.9, as the classifying threshold for IDH probabilities. This threshold corresponds to the maximum Youden's index, and both TPR and TNR reach 0.90 (Supplementary data, table S4 ). The threshold corresponding to the maximum F1 score for classifying IDH probabilities is 0.48, with TPR and TNR of 0.60 and 0.99, respectively.(Supplementary data, figure S3A ) Similarly, for the Fal20lNadir90, the performance analysis of the prediction model at each threshold is shown in Table S5 and Supplementary data, figure S3B . For Nadir90, an AUROC of 0.963 [95% CI 0.956-0.970] (Supplementary data, figure S4A ), with an AUPRC of 0.389 (Supplementary data, figure S4B ). We selected the threshold of 0.02, corresponding to a TNR of 0.89 and TPR of 0.93, as the classifying threshold for IDH probabilities. While, we selected the threshold of 0.27, corresponding to a TNR of 0.99, as the classifying threshold for IDH probabilities, meeting the requirement that the FPR is less than 10%. This threshold corresponds to the maximum F1-score. (Supplementary data, table S6 and Supplementary data, figure S5)Supplementary data, figure 6 illustrates the importance of each variable in the pre-dialysis IDH prediction model, plotted as a bar graph based on the average SHAP values. The results indicate that the monthly minimum SBP, monthly average low SBP, weekly average SBP, and weekly minimum SBP have the greatest impact on the model predictions. Monitored SBP, DBP, the incidence rate of IDH events, dialysis duration, and dialysis vintage also significantly affect the prediction results, suggesting their relevance to IDH.  
discussion: 
 The definition of IDH typically involves the nadir SBP and its decrease from pre-dialysis SBP. Considering the association between different definitions of IDH and mortality risk [2] , this study defines IDH as Nadir90 and Fall20Nadir90.In this study, we utilized both pre-dialysis and intra-dialysis data to train a lightweight predictive model, achieving accurate prediction of IDH. By comprehensively leveraging pre-dialysis clinical variables and real-time dynamic data during dialysis, we significantly improved the model's predictive performance using XGBoost algorithm. Our model was developed and validated on a diverse patient cohort comprising 61,823 HD sessions. The predialysis model's performance for Nadir90 achieved an AUROC of 0.955 and an AUPRC of 0.686, while for Fall20Nadir90, it achieved an AUROC of 0.933 and an AUPRC of 0.440. The intradialysis warning for Nadir90 had an AUROC of 0.959 and an AUPRC of 0.38. The FPR corresponding to the maximum Youden's index and maximum F1 score were both less than 10%. Compared to models using only a single type of data, our approach can more comprehensively capture patients' health status and physiological changes, thereby improving prediction accuracy and reliability. Additionally, our model predicts two different definitions of IDH events, providing crucial support for personalized prediction and intervention.We divided the sample data based on specified dialysis treatment time points. This approach simulates the IDH warning system's real-world performance. In contrast, a validation set created by randomly splitting the sample data would share the same distribution as the development and testing sets, making the validation results less representative of real-world usage.ML methods, by leveraging various types of data, can more comprehensively capture these complex factors, thereby achieving more accurate predictions. We compared the models performance of several algorithmic models, including MLP, KNN, SVM, RF, and XGBoost. Among these, the model trained with the XGBoost algorithm exhibited the best performance, and supported the calculation of risk probabilities even in the presence of missing feature values. Currently, many studies use ML methods to predict IDH, but most rely solely on either predialysis or intra-dialysis data [7][10] . Our research distinguishes themself by integrating pre-dialysis and intra-dialysis data, achieving superior predictive performance.Pre-dialysis data (such as the patient's baseline health status and initial physiological parameters) provide an assessment of the patient's initial health condition and potential risks. Intradialysis data (such as real-time blood pressure and other dynamic physiological variables during the dialysis process) reflect the patient's dynamic physiological changes during dialysis. Considering the need for real-time predictions, we have streamlined the model. This improvement ensures not only high prediction accuracy but also the model's real-time capabilities.One notable limitation of our data is the lack of records on oral medications taken during dialysis treatment, which prevents us from fully accounting for individual medication regimens when predicting the risk of IDH. Additionally, since our data are from a single center, we cannot perform multi-center validation, thereby limiting our ability to verify the generalizability of the IDH prediction model.Finally, our model shares a common drawback with many ML models: it is challenging to interpret how the model arrives at its predictions or how individual factors influence the outcomes [19] .In summary, ML algorithm-based models can predict the risk of IDH occurrence both before and during dialysis. Further prospective studies are needed to evaluate whether this predictive information helps healthcare professionals intervene early and prevent IDH events, as well as to assess the improvement in IDH event rates and patient outcomes.The formulas for calculating the UF target rate and DF target is as follows:  
 
 Appendix:  



Reviews: review: title: title: The study effectively develops a high-performing predictive model for intradialytic hypotension (IDH) using large-scale clinical data, achieving an AUROC of 0.955 with XGBoost. However, the innovation is limited by the application of existing methods without new theories, insufficient time-series modeling, lack of detailed ethical considerations, and inadequate handling of class imbalance.
main_review: main_review: ## pros
- The study use of pre-dialysis and intra-dialysis data has significantly improved the accuracy of IDH prediction. The model not only considers pre-dialysis clinical variables but also utilizes real-time dynamic physiological data during the dialysis process, more fully reflecting the physiological changes in patients.
- The sample size is large, and the data are diverse. The study included 61823 dialysis records from 644 patients, with a total of 302942 systolic blood pressure measurements during dialysis, ensuring a substantial volume of data.

## cons
-  Although the manuscript has made some improvements in areas such as feature engineering, its overall innovation is not sufficient. The main contribution lies in the application of existing methods to specific problems, without proposing any new theories or methods. 
-  The occurrence of IDH may be related to the patient's historical status and treatment process, exhibiting time-dependence. The paper does not adequately model and utilize time-series information. It primarily involves summarizing historical features through statistical aggregation, without considering the dynamics of time and sequential patterns.
- IDH is an event with a low incidence rate, and the data may have a serious class imbalance issue. The manuscript does not provide detailed descriptions of how the problem of sample imbalance was addressed, such as whether oversampling, under sampling, or cost-sensitive learning methods were used. Sample imbalance could affect the training and predictive performance of the model, necessitating a more comprehensive assessment.
- In the feature engineering section, the paper lists the raw features used and mentions the construction of some new features based on clinical knowledge. However, the paper does not provide detailed explanations of the specific definitions, calculation formulas, and clinical significance of each feature. For some complex derived features, there is a lack of clear construction logic. This could affect the reader's understanding of the features and interpretation of the results.
- The paper compares several machine learning models, but the description of hyperparameter tuning is somewhat brief. It does not specify the hyperparameter settings. Particularly, the parameters of the models used in the paper are quite important, and a more detailed discussion would be appropriate.
title: title: Good paper, accept
main_review: main_review: First of all, this submitted paper is not blindly. 
Pros:
* Rigorous methodology: this study tilizes a range of machine learning models and cross-validation, with detailed evaluation with AUROC, AUPRC, TPR, etc.
* Clinical relevance: potential to improve patient outcomes by predicting IDH and enabling timely intervention.
* Novel approach: combining pre-dialysis and intra-dialysis data for real-time predictions is innovative.

Cons:
* Limited external validation: in this study, data were collected from a single center. The generalizability is uncertain.
* Incremental improvement: builds on existing work rather than introducing completely new concepts.
* Preliminary nature: needs more validation before clinical implementation.
Overall, this paper presents a robust and innovative approach to predicting intradialytic hypotension using machine learning algorithms, leveraging a comprehensive dataset and rigorous evaluation metrics. The authors have structured the paper clearly, providing detailed explanations and visual aids to enhance understanding. Despite the high complexity and dense technical content, which may pose challenges for non-experts, the novel combination of pre-dialysis and intra-dialysis data demonstrates significant clinical relevance. However, the study's limitations, including the lack of external validation and potential biases in data handling, indicate that further validation is necessary before clinical implementation. This work represents a substantial step forward in the intersection of healthcare and machine learning, offering promising insights while highlighting the need for broader applicability and additional research.