## 2 Methods
### ANN and DT Models
#### 2.4.2 DTs  
Boosting technique was introduced by [PERSON] [62] and updated by [PERSON] [63], [PERSON] and [PERSON] [64]. This technique produces a set of element NNs whose training datasets are ascertained by the performance of previous ones. Training examples that are incorrectly forecasted by previous networks will perform more vital tasks in the training of succeeding networks. Bagging is introduced by [PERSON] [65] following the bootstrap sampling procedure [66]. This technique produces various training datasets from the initial training dataset and later trains an element NN from every training set.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **Equation** & **R** & **Std. Error of the Estimate** \\ \hline \multirow{2}{*}{Enter} & E = 25.89 \(\times\) DD + 0.005 \(\times\) Vp + 1.50 \(\times\) Qtz \(-\) 1.22 \(\times\) Kpr & \multirow{2}{*}{0.6} & \multirow{2}{*}{26.620} \\  & \(-\) 0.10 \(\times\) Plg \(-\) 2.18 \(\times\) Chl \(-\) 1.29 \(\times\) Mica \(-\) 36.16 & & \\ Stepwise/Backwards/Forwards & E = 0.000904 \(\times\) Vp + 1.644 \(\times\) Qtz \(-\) 43.93 & & 0.553 & 26.018 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Equations using various multiple regression techniques.  
DT is a kind of supervised learning algorithm suitable for classification and regression problems. With a tree-like formation, it builds the training data from the top down by choosing the most suitable decision node to split first. The subsequent nodes are then produced using entropy and information gain. For every chance node, weights are estimated by conditional (joint) probabilities. Various DTs are successfully implemented in the engineering field, including classification and regression decision tree (CART), Chi-square automatic interaction detector (CHAID), Quick, unbiased and efficient statistical tree (QUEST), and C5. However, these DTs are assumed as weak learners because they are vulnerable to changes in data as well as overfit.  
To remedy the issues with DTs stated above, ensemble approaches can be employed. The same as the application of the ensemble approach in ANN, the ensemble approach integrates various DTs to create a more suitable predictive performance compared to a single DT. Random forest (RF) [67] and extreme gradient boosting tree (XGBT) [68] are well-known bagging and boosting techniques, which use a CART model for creating a stronger learner.  
As earlier discussed, RF is a bagging algorithm that reduces variance. DTs are unstable with regard to the tiny fluctuations in data. RF can offer a strong model, which lessens the diversity caused by the \"bagging\" technique. Bagging is an ensemble method in which several predictors are made and consolidated by applying some averaging systems, including normal average, majority vote, and weighted average.  
Gradient boosting (GB) is an effective technique for boosting. GB applies a gradient descent algorithm that can optimize any differentiable loss function. A combination of DTs is created one by one, and single DTs are summed sequentially. The next DT attempts to improve the loss (variation between real and forecasted values). Extreme gradient boosting tree (XGBT), which enjoys definite assessments to determine the most suitable tree model, is an effective extension of GB. XGBT implements its tree creation process where the similarity score (SS) and gain discover the fittest node divisions. Later, the node division with the most excellent gain is chosen as the most suitable division for the tree.  
Figure 3: Bagging and boosting process.