## 3 Proposed Methodology
### Classification
#### 3.4.6 XGBoost  
XGBoost, an ensemble method, is used with k-fold cross-validation. The working of XGBoost and its evaluation with respect to the performance metrics is discussed in the next section. Boosting is an ensemble modeling technique for creating a classifier that is strong from a huge number of classifiers, which are weak. It is performed by putting together a model from a series of weak models. The training data are used to develop a model to start the process. The second model is then developed, with the goal of correcting the original model's flaws. This method is repeated until all of the training data are correctly predicted. The overall functioning of XGBoost is shown in Algorithm 1. ```
A: booster: \(\bullet\) At each iteration, choose the kind of model to run. There are two choices: \(\circ\)  gradboostree: tree-based models. \(\circ\)  gradboostree: linear models. B. silent: \(\bullet\) When this is set to 1, no running messages will be printed. \(\bullet\) It is best to leave it at 0, as the messages may aid in comprehending the model. C. nthread \(\bullet\) Total cores in the system must be put here for parallel processing. If one wishes to run on all cores, leave the value blank and the algorithm will figure it out. ```  
**Algorithm 1**Overall functioning of XGBoost  
Despite the fact that there are two sorts of boosters, we will only consider the tree booster because it consistently performs better than the linear booster, which is why the second one is rarely utilized. The function \(y=f(x)\) is recovered by approximately estimating \(\hat{f}(x)\) while measuring how good the mapping is using a loss function \(l(y,x)\) and then taking the average over all the dataset points to obtain the final cost, given the dataset \(\{(xi,yi)\}i=1\,\ldots\,,\,n\), where \(x\) are the features and \(y\) is the target, to solve any supervised machine learning problem. XGBoost uses \(K\) additive trees to create the ensemble model as shown in Equation (10)  
\[\hat{y}=\hat{f}(x)=\sum_{i=0}^{K}\hat{f}_{l}\ (x),\ \hat{f}_{l}(x)\in F \tag{10}\]  
where \(F=f(x)=\omega_{q(x)}\big{(}q:\ R^{m}\to T,\ \omega\in R^{T}\big{)}\). The tree structure that transfers an input to the relevant leaf index at which it finishes up is represented by \(q\). The number of leaves on the tree is denoted by \(T\). Each leaf of a regression tree consists of a continuous score. The score on the \(i^{\text{th}}\) leaf is represented by \(\omega\)i. Then, minimize the following regularized objective to learn the set of functions employed in the model as shown in Equations (11) and (12). \[l(\varnothing)=\sum_{i}l(\hat{g}_{i},y_{i})+\sum_{k}\Omega(f_{k}) \tag{11}\]  
where  
\[\Omega(f_{k})=\gamma T+\frac{1}{2}\lambda||\omega|k|^{2} \tag{12}\]  
To minimize overfitting, the additional regularization term smooths out the final learned weights. Trees with deeper depth have more leaf nodes, which can lead to overfitting on the training data, as only a few samples end up in each leaf node. As a result, we impose a penalty for the amount of leaf nodes to limit the depth and overfitting. The objective reverts to classic gradient tree boosting when the regularization parameter is set to zero. To minimize the following objective function, as shown in Equation (13), it is necessary to add \(f_{t}\) for the \(t^{\text{th}}\) iteration. \[l^{(t)}=\sum_{i}l(\hat{y}_{i}^{t-1}+f_{t}(x_{i}),y_{i})+\sum_{t}\Omega(f_{k}) \tag{13}\]  
The extension of a real function \(f(x)\) around a point \(x\) = \(a\) is called a one-dimensional Taylor series, and it is given by the Equation (14). \[f(x)=f(a)+f|(a)(x-a)+\frac{f^{||}}{2!}(x-a)^{2}+\ldots+\frac{f^{n}}{n!}(x-a)^{n }+\ldots, \tag{14}\]  
Then, the function can be approximated as shown in Equation (15) by using second order approximation while disregarding higher-order terms. \[l^{(t)}=\sum_{i}^{n}[\,l(\hat{y}_{1}^{t-1},\ y_{i})+g_{i}f_{t}(x_{i})+\frac{1} {2}h_{i}f_{i}^{2}(x_{i})]+\Omega\ (f_{k}) \tag{15}\]  
To make the objective function  
IR are the left and right node instance sets after the split.