## 4 Artificial Hummingbird Algorithm (AHA) and XGBoost
### Extreme Gradient Boosting Tree (XGBoost)  
XGBoost is an ensemble learning algorithm that improves upon the Gradient Boosting Decision Tree (GBDT) framework. By introducing regularization terms, parallelization, and second-order derivatives of loss functions, it achieves faster and more effective predictions. The algorithm optimizes iteratively by constructing new regression trees to fit the negative gradient (i.e., residuals) of the current model in each iteration, and achieves the final output by weighting and accumulating the predictions from all regression trees. This iterative optimization strategy not only enhances the prediction accuracy of the model but also effectively controls model complexity through regularization terms, thereby improving generalization capabilities. If there are M sub-trees, the model's output result is:  
\[\hat{y}_{i}=\sum_{m=1}^{M}f_{m}(\mathrm{x}_{i}) \tag{24}\]  
In the formula, \(y_{i}\) is the model output, \(f_{m}\) is the \(m\)-th tree, and \(\mathrm{x}_{i}\) is the \(i\)-th sample. The objective function to be minimized is:  
\[L=\sum_{i=1}^{m}l(y_{i},\hat{y}_{i})+\Omega(f_{i})+\epsilon \tag{25}\]\[\Omega(f_{t})=\gamma T+\frac{1}{2}\lambda\sum_{i=1}^{T}\omega_{j}^{2} \tag{26}\]  
In the formula, \(l\) is the loss function, \(\varepsilon\) is a constant, \(T\) is the number of leaf nodes, \(\omega\) is the weight of leaf nodes, and \(\gamma\) and \(\lambda\) are the regularization parameters that controls the complexity of the model. Compared with traditional GBDT, XGBoost introduces a regularization term to penalize the complexity of the model, making it less prone to overfitting when the regularization parameter increases [11]. The objective function in the \(t\)-th iteration can be expressed as:  
\[\begin{array}{l}L(\mathrm{t})=\sum\limits_{i=1}^{n}l(y_{i},\hat{y}_{i}^{( \mathrm{t})})+\Omega(f_{t})\\ =\sum\limits_{i=1}^{n}l(y_{i},\hat{y}_{i}^{(\mathrm{t}-1)}+f_{i}(\mathrm{x}_{i }))+\Omega(f_{t})\end{array} \tag{27}\]  
It can be obtained by performing a second-order Taylor expansion on \(L(\mathrm{t})\) and deleting the constant term:  
\[L(\mathrm{t})\approx\sum\limits_{i=1}^{n}\biggl{[}g_{i}f_{i}(\mathrm{x}_{i})+ \frac{1}{2}h_{i}f_{i}^{2}(\mathrm{x}_{i})\biggr{]}+\Omega(f_{t}) \tag{28}\]  
In the formula, \(g_{i}\) and \(h_{i}\) are the first and second gradients of the loss function, respectively. Let the sample number set of the \(j\)-th leaf node be \(I_{j}=\{i|q(\mathrm{x}_{i})=\mathrm{j}\}\), where \(q(\mathrm{x}_{i})\) is the leaf label value corresponding to \(\mathrm{x}_{i}\). Then, Equation (28) is rewritten as:  
\[L(\mathrm{t})\approx\sum\limits_{j=1}^{T}\biggl{[}G_{j}\omega_{j}+\frac{1}{2} (\mathrm{H}_{j}+\lambda)\omega_{j}^{2}\biggr{]}+\gamma T \tag{29}\]  
In the formula, \(G_{j}\) and \(\mathrm{H}_{j}\) are the sum of all \(g\) and \(h\), respectively. Let the derivative of \(\omega_{j}\) be 0, and get the leaf node score:  
\[\omega^{*}=-\frac{G_{j}}{H_{j}+\lambda} \tag{30}\]  
The minimum value is obtained as follows:  
\[L^{*}=-\frac{1}{2}\sum\limits_{j=1}^{T}\frac{G_{j}^{2}}{H_{j}+\lambda} \tag{31}\]  
The XGBoost algorithm performs well in structured data processing, with faster training speed and a lower required number of samples, so XGBoost is selected as the classifier [12].