## 2 Materials and Methods
### Machine Learning Algorithms
#### 2.6.8 CatBoost  
CatBoost is an improved GBDT toolkit similar to XGBoost. In the process of using the CatBoost algorithm to train the data, the latter tree adjusts its weights according to the previous tree so that the tree with the larger residual is assigned a larger weight. Finally, multiple weak regressors are integrated to form a strong regressor. CatBoost solves the problem of gradient deviation and prediction bias. CatBoost has the following advantages [40,54]: CatBoost has two implementations: CPU and GPU. The GPU implementation allows faster training and is faster than GBDT, XGBoost, and LightGBM on similarly sized sets.  
Target statistics (TS) is a very effective method. In the decision tree, the average value of the labels will be used as the criterion for node splitting. This method is called greedy target-based statistics, or greedy TS for short. CatBoost uses a standard method of improving greedy TS, adding a prior distribution item to reduce the influence of noise data on the data distribution:  
\[\widetilde{x}_{k}^{i}=\frac{\sum_{j=1}^{p-1}[x_{\sigma_{jk}}-x_{\sigma_{pjk}} ]Y_{\sigma_{j}}+ap}{\sum_{j=1}^{p-1}[x_{\sigma_{jk}}=x_{\sigma_{jk}}]+a}, \tag{23}\]where \(p\) is the added prior term and \(a\) is usually a weight coefficient greater than 0. For regression problems, CatBoost performs a random arrangement of the dataset, with the prior term taking the mean of the dataset labels.  
CatBoost replaces the traditional gradient boosting algorithm with the ordered boosting algorithm. This algorithm can effectively handle the noise points in the training set to avoid the gradient estimation deviation, solve the inevitable gradient deviation in the iterative process, and improve the model generalization ability. Oblivious trees were used as base predictors. In such a tree, the segmentation used at each level of the tree must be consistent. Therefore, the structure of the tree is balanced, the unbiased estimation of the gradient is obtained, and then the gradient descent is carried out, thus alleviating the overfitting phenomenon. The pseudocode is shown in Algorithm 1.  
```
Input : \(\{(X_{k},Y_{k})\}_{k=1}^{n}\) ordered according to, the number of trees \(I\) \(\sigma\leftarrow\) random permutation of \([1,n]\) \(M_{i}\gets 0\) for \(i=1\ldots n\) for \(t\gets 1\) to \(I\) do for \(i\gets 1\) to \(n\) do \(r_{i}\gets y_{i}-M_{\sigma(i)-1}(x_{i})\); for \(i\gets 1\) to \(n\) do \(\Delta M\leftarrow\)\(\textit{LeanModel}((x_{j},r_{j}):\sigma(j)\leq I)\); \(M_{i}\gets M_{i}+\Delta M\); return \(M_{n}\)
```  
**Algorithm 1**Ordered Boosting  
To obtain an unbiased gradient estimate, CatBoost will train a separate model \(M_{i}\) for each sample \(x_{i}\), which is obtained by using a training set that does not contain sample \(x_{i}\). The model \(M_{i}\) is used to obtain the gradient estimate of the sample, and the gradient is used to train the base learner and obtain the final model. In CatBoost, a random arrangement of training data is generated. By sampling random permutations and obtaining gradients based on them, multiple permutations will be used to enhance the robustness of the algorithm. These arrangements are the same as those used to calculate classification feature statistics. To train different models, different permutations will be used, so using several permutations will not lead to overfitting.