# Heart Failure Prediction Based on Bootstrap Sampling and Weighted Fusion LightGBM Model
## 3 Mathematical Model
### Application Strategy of Bootstrap Sampling in Model  
Bootstrap sampling is a random sampling method with playback. In the process of model training, bootstrap sampling is mainly used to increase the diversity of training data and evaluate the stability of the model. The number of bootstrap sampling iterations is directly related to the number of different training subsets generated by it, which has an important influence on the stability and accuracy of the final comprehensive prediction results. Although more bootstrap sampling times may enhance the stability of comprehensive prediction results, it will also lead to an increase in the calculation cost and extension of the training time. In the research process of the thesis, we can choose a smaller value, such as 3, as the starting point, and then gradually increase it, i.e., trying 5, 7, 10, and so on. In this process, the performance changes of the model on the verification set and the growth trend of training time are closely observed. We observe that with the increase in bootstrap sampling iterations, the performance of the model on the verification set is gradually stable, and the training time is obviously prolonged. On this basis, we can determine the optimal bootstrap sampling iteration number. In this way, while ensuring the accuracy and stability of the results, we can reasonably manage the computing resources and time costs, so as to achieve the optimal balance between them. Figure 1: Overall frame diagram. At each bootstrap sampling iteration, the number of sample indexes with the same length as the original training set subset is randomly extracted from the current training set subset, thus obtaining a new training data subset and its corresponding label subset. In bootstrap sampling, training data from each iteration probably contain duplicate samples. Each sample in the original training set has a well-defined probability of being selected or not during each operation. This probability selection mechanism is applied repeatedly in several sampling rounds. Therefore, this leads to different data distributions, which can have a great impact on model training and generalization performance. In the cycle of parameter optimization, for each group of parameter settings, multiple LightGBM models based on different sampling data can be trained through multiple bootstrap sampling processes. This will help the model to better capture various patterns and characteristic relationships in the data, reduce the model deviation caused by a single training data distribution, and also evaluate the performance of the model under different data distributions, providing more abundant and reliable information for subsequent model fusion. The specific implementation steps are as follows:  
1. Determine the sampling times. 2. Conduct bootstrap sampling in the training set. Bootstrap sampling was carried out on the original data set many times to obtain multiple training subsets, and the verification set was divided at the same time. In this process, it is very important to set the proportion of the verification set, which will affect the sensitivity of the model to the performance of the verification set and the effectiveness of the early stop mechanism. Generally speaking, you can try to choose a value in the range of 0.1 to 0.3, such as 0.15, 0.2, 0.25, etc. A small proportion of the verification set may lead to a lack of data in the verification set, which makes it difficult to accurately present the model performance, thus adversely affecting the effectiveness of early stop mechanism. On the contrary, a larger proportion of the verification set will reduce the data size of the training set, which may have an adverse impact on the training effect of the model. Therefore, it is necessary to try different division ratios, carefully observe the performance of the model on the verification set, and the trigger status of the early stop mechanism, so as to find an appropriate ratio value. 3. Further divide the training set after bootstrap sampling.