# Heart Failure Prediction Based on Bootstrap Sampling and Weighted Fusion LightGBM Model
## 3 Mathematical Model
### Parameter Optimization of Model
#### 3.4.2 Parameter Optimization Strategy  
In this study, we use the grid search strategy to adjust the hyperparameters and find the optimal hyperparameter configuration of the model. The grid search is adopted to traverse the hyperparameter combination, and the grid search strategy is used, that is, every possible hyperparameter combination is traversed in turn according to the value range defined by each hyperparameter through the multi-layer nested cycle. For example, there are several values for the learning rate, several values for the number of leaf nodes, and so on for other hyperparameters. In this way, all possible parameter combinations can be traversed, and the entire hyperparameter space can be searched systematically. Although this method may be time-consuming when the number of hyperparameters is large and the range of values is wide, it can ensure integrity and not miss any possible optimal combination. The specific optimization strategies are as follows:  
1. Determination and range setting of hyperparameters. Firstly, the key hyperparameters that need to be optimized are defined, and a reasonable range of values is defined for each hyperparameter. The specific hyperparameter are as follows. Learning rate: This parameter controls the parameter step size in each iteration, which has an important influence on the convergence speed and performance of the model. We set its range as [0.1, 0.05, 0.01]. Number of leaf nodes (num_leaves): This parameter determines the complexity of the decision tree and affects the model's ability to fit data. The range of values is set to [31; 63]. Min_child_weight: used to limit the minimum weight of samples in child nodes and prevent overfitting. The range of values is [1 \(\times\) 10\({}^{-3}\), 1 \(\times\) 10\({}^{-2}\)]. Bagging_fraction: indicates the proportion of randomly selected samples during training. The range of values is [1, 0.8]. Number of bootstrap sampling (num_bootstrap_samples): determines the number of times that bootstrap sampling operation is repeated. The range of values is [3; 5; 7]. 2. Implementation process of grid search  
Using the grid search strategy, by constructing a multi-layer nested loop structure, each possible hyperparameter combination is traversed in turn in strict accordance with the value range defined by each hyperparameter, specifically. The outermost loop takes values from the range [3; 5; 7] in turn for the number of bootstrap samples (num_bootstrap_samples). The second external loop adopts the values in [0.1, 0.05, 0.01] for the learning rate. Next,take a value from [31; 63] for the number of leaf nodes (num_leaves). Then, for the smallest child node weight (min_child_weight), take a value in [\(1\times 10^{-3}\), \(1\times 10^{-2}\)]. The innermost loop takes one of the values in [1; 0.8] for the sampling bagging fraction. Through this loop structure, all hyperparametric combinations can be traversed comprehensively, and the whole hyperparametric space can be searched systematically. Training and evaluation of the model  
For each hyperparameter combination generated, perform the following. Cross-validation setting: The training data set (X_train) is divided into five different subsets by 50% cross-validation (KFold, n_splits = 5, shuffle = True, random_state = 2). Bootstrap sampling and model training: Bootstrap sampling is carried out in the training process of each cross-verification. That is, samples are randomly selected from the current training subset (X_train_fold) to construct self-help sampling data sets (X_train_bootstrap and y_train_bootstrap). Using the LightGBM model, the model parameters are set according to the current hyperparameter combination, trained on the training set, and monitored on the verification set, and the early stop mechanism (early_stopping_rounds = 200) is set to prevent overfitting. To sum up, this study uses the grid search strategy to adjust the hyperparameters.