## 3 Experimental Setup and Results
### Results from Single Classifier Experiment  
We trained seven single classifiers with the train set of the four datasets, using the same stages of preprocessing and feature extractions. Table 8 presents the performance of the single classifiers on test sets in terms of the F1-score.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Base-Level Classifier** & **HatEval** & **Davidson** & **COVID-HATE** & **ZeerakW** \\ \hline KNN & 0.6061 & 0.9296 & 0.6982 & 0.7480 \\ LR & 0.6434 & 0.9343 & 0.7761 & **0.8406** \\ SVM & 0.6180 & **0.9517** & 0.7632 & 0.8398 \\ NB & **0.6845** & 0.8669 & 0.7201 & 0.7402 \\ RF & 0.6443 & 0.8671 & **0.7963** & 0.6081 \\ E-tree & 0.5808 & 0.9358 & 0.7745 & 0.7180 \\ XGB & 0.6745 & 0.9500 & 0.7960 & 0.8253 \\ \hline \hline \end{tabular}
\end{table}
Table 6: F1-score of the Base-Level classifiers on the development set. Bold entries show the highest performance for each classifier on each dataset.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Base-Classifiers Combination** & **HatEval** & **Davidson** & **COVID-HATE** & **ZeerakW** \\ \hline SVM, LR, XGB & **0.6551** & **0.9713** & **0.7301** & **0.7392** \\ KNN, SVM, NB & 0.6405 & 0.9613 & 0.6920 & 0.7049 \\ LR, NB, RF & 0.6407 & 0.9601 & 0.7136 & 0.7226 \\ KNN, LR, NB & 0.6410 & 0.9710 & 0.7261 & 0.7150 \\ SVM, LR, E-tree & 0.6428 & 0.9710 & 0.7255 & 0.7253 \\ \hline \hline \end{tabular}
\end{table}
Table 7: F1-score of the stacking ensembles using different base classifiers on test sets. Bold entries show the highest performance for each classifier on each dataset.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Base-Level Classifier** & **HatEval** & **Davidson** & **COVID-HATE** & **ZeerakW** \\ \hline KNN & 0.5885 & 0.9281 & 0.6580 & 0.6037 \\ LR & **0.6407** & 0.9365 & 0.7146 & **0.7179** \\ SVM & 0.6394 & **0.9530** & 0.6843 & 0.7030 \\ NB & 0.6024 & 0.8745 & 0.6539 & 0.5508 \\ RF & 0.6016 & 0.8797 & 0.6710 & 0.6274 \\ E-tree & 0.6031 & 0.9397 & 0.6542 & 0.6747 \\ XGB & 0.6353 & 0.9498 & **0.7246** & 0.7058 \\ \hline \hline \end{tabular}
\end{table}
Table 8: F1-score of the Base-Level classifiers on the test set. Bold entries show the highest performance for each classifier on each dataset.  
It can be seen that the SVM classifier produced the best performance on two of the datasets, Davidson and ZeerakW. For the other two datasets HatEval and COVID-HATE, the single classifiers LR and XGB, respectively, achieved the best F1-score.  
We tested the Base-Level classifiers on each dataset's development and test sets and presented the results in Tables 6 and 8, respectively. The challenge here is that the most commonly used words in the test and development sets are different, which may adversely affect the performance of our Meta-Level classifier leading to misclassification. [PERSON] et al. (2019) also attributed the low performance of their system to the inconsistency between train and test sets, which was also observed by ([PERSON] et al., 2019). It can be observed in Tables 6 and 8 that, as mentioned before, the data in the test, development, and training portion of the dataset vary, and that is reflected in the performance of the Base-Level classifiers. Moreover, it can be argued that training the Meta-Level classifier on development data might be harmful.