# Heart Failure Prediction Based on Bootstrap Sampling and Weighted Fusion LightGBM Model
## 5 Experimental Results and Analysis  
### Feature Selection  
Feature selection can help to eliminate irrelevant or redundant features, thus improving the accuracy and operation efficiency of the model by reducing the number of features. When selecting features, we usually need to consider two aspects:  
1. Whether the feature is divergent: If the difference between samples in this feature is very small, then this feature has no practical effect on distinguishing data samples. Generally speaking, such features can be regarded as invalid or irrelevant features, and they are eliminated in the process of feature selection to improve the prediction performance of the model. 2. Correlation between features and targets: The correlation between features and targets is a very important factor in feature selection. When the correlation between the feature and the target is high, the prediction ability of the feature for the target is stronger. Firstly, the correlation analysis is carried out. In this paper, the XGBoost algorithm, a prominent algorithm within the tree model family, is employed to conduct the screening of feature variables. The essence of deriving the importance scores for each feature hinges on the computation of information gain. The information gain formula is as follows:  
\[IG(X,Y)=H(Y)-H(Y|X)=\sum_{x,y}p(x)\cdot p(y|x)\cdot\log(y|x)-\sum_{y}p(y)\cdot \log p(y) \tag{8}\]  
In the above formula, empirical entropy (\(Y\)) represents the uncertainty of data set \(Y\), while empirical conditional entropy (\(Y\mid X\)) represents the uncertainty of data set Y under given conditions, that is, when the characteristic variables X are known. Information gain indicates the degree to which the uncertainty of data set \(Y\) is reduced under the action of characteristic variables X. Since different feature variables typically yield varying degrees of information gain, by computing the information gains of each individual feature variable and ranking them accordingly, it is possible to identify the features that exhibit the optimal regression performance. By analyzing the correlation in Figure 8 between the classification results of heart disease and the characteristic variables, it is revealed that the correlation between the RestingECG feature and heart disease is nearly zero when the LVH (left ventricular hypertrophy) value is in a certain state. However, in other value states of LVH, there exists a certain degree of correlation. Therefore, it remains necessary to retain the RestingECG feature in the analysis. Furthermore, the importance of features was ranked, and it was found that Cholesterol, MaxHR, Age, and RestingBP had great influence on heart disease in Figure 9.  
Figure 8: Feature correlation analysis. In this feature importance map, Cholesterol, MaxHR, Age, and RestingBP rank higher. This is because they exert direct and crucial influences on heart health. For instance, abnormal cholesterol levels can disrupt blood supply to the heart; an abnormal maximum heart rate may serve as an indicator of underlying heart problems; the aging process is often associated with the deterioration of heart function and the increased susceptibility to chronic diseases; and elevated resting blood pressure can impose additional stress on the heart, thereby increasing the risk of heart-related issues. Nevertheless, features like fasting blood sugar (FastingBS) and exercise-induced angina (ExerciseAngina) have lower rankings. This is due to the fact that their associations with heart disease are not directly predominant, and within the context of the current data set and model, their contributions to the determination of the prevalence of heart disease are relatively minor. In this data set, cholesterol, maximal heart rate, age, and resting blood pressure are the most important predictors of heart failure, which has its inherent physiological and pathological causes. Excessive cholesterol level will lead to atherosclerosis, resulting in vascular stenosis and blocked blood flow.