# Heart Failure Prediction Based on Bootstrap Sampling and Weighted Fusion LightGBM Model
## 4 Experimental Settings
### Result Evaluation Metrics  
In this study, multiple evaluation metrics are comprehensively employed to assess the advantages and disadvantages. Instead of relying solely on a single evaluation metric to gauge the model's performance, a variety of common and crucial evaluation metrics, such as the accuracy, precision, recall, and F1_score, are calculated. Different evaluation indicators reflect the performance of the model from different angles. For instance, the accuracy rate measures the proportion of correct predictions. The precision rate emphasizes both the prediction of positive cases and the proportion of correct predictions among them. The recall rate focuses on the proportion of actual positive cases that are correctly predicted. The F1_score is the harmonic mean of the precision rate and the recall rate, which comprehensively takes into account the balance between these two metrics. By integrating these evaluation metrics, we can conduct a more comprehensive and precise assessment of whether a specific hyperparameter combination enables the model to exhibit superior performance across various aspects. In turn, this helps to select the parameter configuration that best matches the task requirements.  
The accuracy, precision, recall, and F1_score are as follows:  
\[Accuracy=\frac{TP+TN}{TP+FP+TN+FN} \tag{1}\]  
\[Precision=\frac{TP}{TP+FP} \tag{2}\]  
\[Recall=\frac{TP}{TP+FN} \tag{3}\]  
\[F1\_score=\frac{2*Recall*Precision}{Recall+Precision} \tag{4}\]  
In addition, the receiver operating characteristic (ROC) curve is usually used to show the performance of the classifier, in which the true positive rate (TPR) is taken as the ordinate and the false positive rate (FPR) is taken as the abscissa. The closer the ROC curve is to the upper left, the better the performance of the classifier. If the ROC curve of one classifier completely covers the other, it can be concluded that the latter is better. If ROC curves overlap or cross, it will be difficult to evaluate the performance of classifiers. At this time, the area under ROC curve can be compared for evaluation and comparison. Among them, the calculation methods of TPR and FPR are as follows:  
\[TPR=\frac{TP}{TP+FN} \tag{5}\]  
\[FPR=\frac{FP}{TN+FP} \tag{6}\]  
The lower area of the ROC curve is the area under the curve (AUC), and its value is between 0 and 1. In the actual modeling process, the generalization ability of the model can be roughly judged according to the range of the AUC value.  
Typically, when evaluating machine learning models, we employ the paired \(t\)-test to compare the performance of different models and determine whether the differences among them are statistically significant.  
The formula of the paired \(t\)-test is as follows:  
\[t=\frac{\overline{d}}{s_{d}/\sqrt{n}} \tag{7}\]  
where \(\overline{d}\) is the average value of the difference between paired samples, that is, \(\overline{d}=\frac{\sum_{i=1}^{n}d_{i}}{n}\). \(d_{i}\) is the difference between the i-th pair of observations. \(s_{d}\) is the standard deviation of the difference between paired samples. The calculation formula is \(s_{d}=\sqrt{\frac{\sum_{i=1}^{n}(d_{i}-\overline{d})^{2}}{n-1}}\). \(n\) is the number of paired samples.  
The t-value computed using this formula is compared with the critical value of the t-distribution corresponding to a specific degree of freedom to ascertain whether there exists a statistically significant difference between the means of the two paired samples. At the same time, the \(p\) value can be calculated from the t value, and the significance of the difference can be judged more accurately. If the value of \(p\) is less than the pre-set significance level, such as 0.05, it is considered that there is a significant difference between the two groups of samples. In a paired sample \(t\)-test, the \(p\) value is the probability of obtaining the current sample data or more extreme data under the premise of the original assumption.