## 5 Conclusions  
In this work, we proposed a novel stacking ensemble for distinguishing between hateful and non-hateful tweets from English Twitter datasets of various sizes. We applied the proposed stacking classifiers approach at two levels: Base and Meta levels. The Base-Level classifiers are trained using the features extracted from the training set. Then, the Meta-Level classifier is trained on the predictions of the Base-Level classifiers using the development set and the features extracted from the development set. The \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **State of the Art** & **Proposed Stacking Ensemble** \\ \hline HatEval & 65.1 [9] & **65.5** \\ Davidson & 94.0 [19] & **97.1** \\ COVID-HATE & - & **73.0** \\ ZeerakW & **77.8**[18] & 73.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 13: F1-score (%) of the state-of-the-art and the proposed stacking ensemble on all datasets. Bold entries show the highest performance for each dataset. preprocessed unseen test set was then classified using our model. On all of the datasets studied, we compared the results of the proposed approach to those of the single classifiers, majority voting ensembles, standard stacking, and the state-of-the-art. The results indicated that distinguishing Hateful and Non-hateful tweets is a challenging task. Compared with single classifiers, standard stacking, and majority voting, the proposed technique achieved the highest F1-score on all datasets. Furthermore, the proposed approach outperformed the state-of-the-art in three out of the four datasets using the same combinations of features (word2 vec and USE), and the same combination of Base-Level classifiers (SVM, LR, XGB), with LR as Meta-Level classifier. Even though there are two phases for training the classifiers, this is justifiable as it is not repetitive. Nonetheless, the features that may be useful for correct classification are related to the nature and composition of the data to be classified; thus, fine-tuning the feature engineering stage may improve the performance even further. More comprehensive datasets that include a more extended timeframe would exemplify the time-varying nature of the tweets and allow the classifiers to capture the dynamic nature of hate speech. Future work will address the imbalance problem. The data imbalance where the distribution of samples per class is not equal influences machine learning classifiers in favor of the majority class. This is not desirable especially in cases such as hate speech recognition where identifying the minority class is more important. Therefore, balancing the dataset used for training the base classifiers in our approach is expected to improve the performance of the proposed system. Resampling data for balancing the dataset can be achieved by oversampling the minority class and if needed, undersampling the majority class. During this process it is important to ensure useful information is not lost due to undersampling and the oversampled data do not cause overfitting in machine learning classifiers. Our proposed architecture will be extended by employing alternative resampling methods at the base-classifier training phase of the proposed architecture. Specifically, the effect of the minority oversampling techniques on the final performance will be studied. Furthermore, the proposed stacking approach will be applied to different domains of text classification with binary as well as multiple classes. Conceptualization, N.D. and M.K.A.A. ; methodology, N.D., and M.K.A.A. ; software, M.K.A.A. ; validation, M.K.A.A. and N.D.; visualization, N.D., and M.K.A.A. ; data curation, M.K.A.A. ; formal analysis, N.D. and M.K.A.A. ; writing--original draft preparation, N.D. and M.K.A.A. ; writing--review and editing, N.D., and M.K.A.A. ; supervision, N.D. All authors have read and agreed to the published version of the manuscript. This research received no external funding. This research received no external funding. Not applicable. Informed Consent Statement: Not applicable.