## 6 Experiments
### Ablation Experiments (Rq2)
#### 6.6.2 Analysis of Self-Training with Majority Voting  
Regarding the second component of our proposed method, which is self-training with majority voting, as previously mentioned, we initiated the training process with only 10 labeled examples per emotion class from the benchmark datasets. In each iteration, additional data were automatically labeled by the base classifiers and incorporated into the training data through majority voting. Specifically, new data with its predictions were considered confident and included in the training dataset for subsequent iterations if at least 5 out of the 6 classifiers reached a consensus on its prediction. Tables 10 and 11 show the performance of self-training with majority voting on the AETD and ArPanEmo datasets. Self-training with majority voting obviously demonstrated improved results compared to most individual base classifiers but fell short of the performance achieved by the fine-tuned BERT model. It yielded a weighted average F1-score of 78.12% on the AETD dataset, whereas the BERT model achieved a higher weighted average F1-score of 81.66%. The same results can be noticed when evaluating self-training on ArPanEmo, where the weighted average F1-score was 67.39%, while the fine-tuned BERT model outperformed it with a weighted average F1-score of 69.08%. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Accuracy**} & \multicolumn{4}{c}{**Weighted**} & \multirow{2}{*}{**Training Data (Size)**} \\ \cline{3-4} \cline{6-6}  & & **F1** & **P** & **R** \\ \hline Zero-Shot Classification & 34.03 & 30.93 & 42.67 & 34.03 & No-training \\ \hline Naive Bayes & 69.67 & 68.94 & 69.8 & 69.67 & \\ Logistic Regression & 66.1 & 63.63 & 63.65 & 66.09 & \\ Data Augmentation & 65.7 & 63.56 & 64.82 & 65.7 & AETD (80 manually labeled examples) + \\ Embedding Lookup & 58.69 & 58.28 & 58.04 & 58.69 & Automatically \\ BERT & **81.65** & **81.66** & **81.73** & **81.86** & Labeled Data (22K) \\ \hline Self-training with & \multirow{2}{*}{77.16} & \multirow{2}{*}{78.12} & \multirow{2}{*}{79.64} & \multirow{2}{*}{76.65} & \multirow{2}{*}{7} \\ Majority Voting & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Performance of self-training with majority voting on the AETD dataset. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Accuracy**} & \multicolumn{4}{c}{**Weighted**} & \multirow{2}{*}{**Training Data (Size)**} \\ \cline{3-4} \cline{6-6}  & & **F1** & **P** & **R** \\ \hline Zero-Shot Classification & 28.86 & 27.837773 & 27.27 & 28.43 & No-training \\ \hline Naive Bayes & 59.85 & 61.58 & 63.94 & 59.39 & \\ Logistic Regression & 58.20 & 59.90 & 62.06 & 57.89 & Entire ArPanEmo \\ Data Augmentation & 57.25 & 58.62 & 60.18 & 57.14 & dataset: \\ Embedding Lookup & 55.83 & 57.05 & 58.30 & 55.86 & 8901 manually \\ BERT & 66.95 & 67.64 & 68.64 & 66.67 & labeled examples \\ \hline Stacking Ensemble & **69.08** & **69.93** & **71.46** & **68.47** & \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Performance of the stacking ensemble model and base classifiers on the ArPanEmo dataset. When comparing self-training with the stacking ensemble, it is evident that self-training with majority voting yielded a lower weighted average F1-score than the stacking ensemble model. This indicates the stacking ensemble model's ability to effectively leverage the strengths of the six base classifiers. However, it is worth noting that the F1-score of 78.12% achieved by self-training is still a good result, especially considering that self-training with majority voting initiates the training process with a limited number of manually labeled examples. On the other hand, the precision of self-training with majority voting is 79.64% on the AETD dataset, which is higher than the recall of 76.65%. This may be partially attributed to our strict condition for accepting a newly annotated label in majority voting, where it must receive the consensus of at least 5 out of the 6 votes from the base classifiers.