## 2 Background
### Ensemble Learning Approaches Overview
#### 2.2.5 Ensemble Voting Method  
Hard Voting  
Each model casts a vote for a specific class label, and the class with the most votes is chosen as the final prediction. This technique relies on the dominant class label found in each individual forecast [30].  
Soft Voting  
Each model generates a probabilistic estimation for every class. The final forecast is determined by selecting the class with the highest average likelihood after probability aggregation. This technique accounts for the level of certainty that models possess in their predictions [31]. One of the key advantages of the voting technique is its capacity to enhance accuracy by leveraging the strengths of multiple models, often outperforming individual learners [32]. Furthermore, ensemble methods provide robustness by mitigating the influence of individual model weaknesses, thereby producing more reliable and trustworthy forecasts [33]. Simplicity is another notable benefit, as these methods are generally easy to implement across diverse models without significant complexity [34].  
Figure 2 is a pictorial view of the ensemble learning methods, while Table 2 summarizes the five ensemble learning methods with an example of each one, along with their strengths and weaknesses.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Ensemble Learning Technique** & **Examples** & **Strengths** & **Weaknesses** \\ \hline Bagging & Random Forest & \begin{tabular}{c} Reduces variance, robust \\ against overfitting [23] \\ \end{tabular} & May not reduce bias [23] \\ \hline Boosting & \begin{tabular}{c} AdaBoost, Gradient Boost, \\ XGBoost, and Cat Boost \\ \end{tabular} & \begin{tabular}{c} Reduces bias, improves \\ accuracy [16; 17] \\ \end{tabular} & \begin{tabular}{c} Can overfit, sensitive to \\ noisy data [17; 18] \\ \end{tabular} \\ \hline Stacking & \begin{tabular}{c} A stacking ensemble where a \\ Random Forest, Gradient \\ Boosting, and K-Nearest \\ Neighbors classifier serve as base learners, and a Logistic \\ Regression model is used as the \\ meta-learner to combine their \\ predictions for final classification \\ \end{tabular} & \begin{tabular}{c} Combines strengths of \\ multiple models [27; 28] \\ \end{tabular} & \begin{tabular}{c} Complex to implement, \\ risk of overfitting [27; 28] \\ \end{tabular} \\ \hline Hybrid & Combination of bagging, boosting & \begin{tabular}{c} Superior performance \\ leverages multiple \\ approaches [29] \\ \end{tabular} & \begin{tabular}{c} Highly complex, \\ challenging to tune [29]. \\ \end{tabular} \\ \hline Voting & \begin{tabular}{c} Combining Random Forest, SVM, \\ and KNN, then vote between them \\ \end{tabular} & \
\begin{tabular}{c} Improves prediction \\ accuracy by leveraging \\ diverse classifiers with its \\ simplicity of \\ implementation and \\ interpretation [30; 31; 32; 33; 34]. \\ \end{tabular} &
\begin{tabular}{c} Depending on the \\ performance of individual \\ models, it may not perform \\ well if models are highly \\ correlated [30; 31; 32; 33; 34]. \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 2: Ensemble learning methods’ strengths and weaknesses.  
Figure 2: Ensemble Learning Methods.