## 3 SAL: Proposed Self-Augmentation Labeling Approach
### Hybrid Voting-Based Ensemble Approach  
With the goal of producing accurate results that outperform those of any individual classifier, we propose an ensemble of classifier approach to aggregate their predictions and produce a class label for a new data item [23; 24]. The voting classifier is one of the popular and powerful ensemble classifiers which takes input as two or more estimators and classifies the data based on majority voting [24; 25]. By combining the multiple prediction models, it overcomes the limitations of single base classifiers and outperforms the overall results. Voting classifiers use two voting approaches: (i) majority/hard voting and (ii) soft voting [26]. With the hard voting, each individual classifier has one vote and the class label for a data item is the class with the majority that was predicted by each single classifier, while soft voting is the same as the hard voting in terms of the use of a set of different classifiers to make the final decision of the appropriate class of the data item; however, each individual classifier assigns a probability score for each class and the most probable class is the one that obtains the largest average probability of all the candidate classifiers. **Definition 1**. : _Let us define the decision of the \(c\)th classifier as \(d_{c_{i},w_{j}}\in\) {0,1}, \(i=1,\,\ldots,\,C\) and \(j=1,\,\ldots,\,W\), where \(W\) is the number of classes and \(C\) is the number of classifiers. If the \(c\)th classifier chooses class \(w_{j}\), then \(d_{c_{i},w_{j}}=1\), and 0 otherwise. The final decision of both hard and soft voting can be defined by Equations (1) and (2), respectively._  
\[\hat{y} =\arg\max_{j}\sum_{i=1}^{C}d_{c_{i},w_{j}} \tag{1}\] \[\hat{y} =\arg\max_{j}\sum_{i=1}^{C}P_{c_{i},w_{j}} \tag{2}\]  
We weighed the predictions of each base classifier equally. In Equation (1), class \(w_{j}\) will be chosen as the final prediction \(\hat{y}\) if its sum of votes from all classifiers \(i=1,\ldots,C\) achieve the maximum sum among all classes \(j=1,\ldots,W\). While in Equation (2), the final predicted class \(\hat{y}\) will be the class that achieves the highest probability \(P_{c_{i},w_{j}}\) among all classes where \(P_{c_{i},w_{j}}\) is the probability that is assigned to the \(i\)th class with \(j\)th classifier. Figures 2 and 3 show illustrative examples that demonstrate the calculation processes of both hard and soft voting techniques in ensemble learning. In hard voting, as depicted in Figure 2, there are three classifiers whose individual decisions are combined to predict whether the data item is an instance of class A or class B. As we can see, both classifiers \(c_{1}\) and \(c_{3}\) predicted the data item as class A, which is considered as a final decision because it gains the majority of voting. As shown in Figure 3, the ensemble decision making of soft voting is different from hard voting, whereas a probability value is assigned to each class by each classifier. As seen, both classifiers \(c_{2}\) and \(c_{3}\) assigned 55% of the data item belonging to class A. However, both classifiers \(c_{2}\) and \(c_{3}\) assigned high probability scores for the data item that belongs to class A. Class B is the most probable class for the data item because this class obtains the largest average probability value of all classifiers \(c_{1}\), \(c_{2}\), and \(c_{3}\), which is 60%, while the average probability value for the data item belonging to class A is 40%. Figure 2: Hard ensemble voting algorithm. Although hard voting is the simplest and most common strategy in ensemble-based decision making, the final decision is based on majority votes irrespective of the total agreement of classification models. Soft voting, on the other hand, incorporates each classifier's prediction into the final decision through taking average prediction probability scores.