## 5 Performance Evaluation
### Character-Level CNN-LSTM

Table 6: Experiments and results of character-level CNN-LSTM (using over-sampled training dataset). Figure 6: Evaluation metrics for 50 epochs of training and testing with 3 convolution layers of 256 filters, 1 LSTM layer of 100 neurons, and dropout rate of 0.2 using imbalanced training dataset: (a) Accuracy and (b) Loss. Figure 7: Evaluation metrics for 50 epochs of training and testing with 3 convolution layers of 256 filters, 1 LSTM layer of 100 neurons, and dropout rate of 0.2 using over-sampled training dataset: (a) Accuracy and (b) Loss. Figure 8 reports the accuracy of prediction (hate/non-hate) of our models and compares it with [PERSON] et al. 's [7] model. The corresponding model for each model number in Figure 8 is listed in Table 7. CNN-LSTM performed slightly better than CNN, whether an imbalanced or over-sampled dataset was used for training the model. Moreover, Figure 8 shows that we outperformed [PERSON] et al. 's [7] model's accuracy by: (1) over-sampling the dataset to avoid the inherent imbalanced class labels and, in turn, biased results, and (2) adding an LSTM layer to the CNN architecture and pairing them for feature extraction and classification. We further performed more experiments by changing hyperparameters such as number of filters, number of neurons, and dropout rate, as reported in Table 5.  
Figure 9 shows a comparison between the time of execution of our models and that of [PERSON] et al. 's [7]. The corresponding model for each model number in Figure 9 is listed in Table 7. Figure 9 reports a significant improvement in runtime in our models. This is mainly due to removing emoticons encoded with UTF-8 DOM (Byte Order Marks) from the tweets. The time of execution of [PERSON] et al. 's [7] model trained with an imbalanced dataset was longer even when compared to our models trained with an over-sampled dataset, which is comparatively larger. Figure 10 shows a comparison between the scalability of our models (Models c and e) and that of [PERSON] et al. 's [7] (Model a). In order to achieve the desired dataset size, we duplicated randomly chosen tweets. The corresponding model for each model number in Figure 10 is listed in Table 7. This experiment shows that all models are scalable because runtime increases proportionately with respect to the number of input records. However, the findings in Figure 10 further vindicate that our models successfully reduce \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Model** & **No. of Epochs** & **Runtime** & **Accuracy** & **Loss** \\ \hline a & Character-level CNN using imbalanced training dataset & 50 & 156 min 18 s & 0.8661 & 0.2704 \\ ([PERSON] et al.’s [7] method) & 50 & 48 min 51 s & 0.8652 & 0.2723 \\ \hline c & Character-level CNN using over-sampled training dataset (**our work**) & 50 & 78 min 57 s & 0.8884 & 0.2661 \\ \hline d & Character-level CNN-LSTM using imbalanced training dataset & 50 & 60 min 8 s & 0.8657 & 0.2688 \\ (**our work**) & & & & & \\ \hline e & Character-level CNN-LSTM using over-sampled training dataset & 50 & 88 min 15 s & 0.8897 & 0.2385 \\ (**our work**) & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison of all the results. Figure 8: Accuracy comparison of different models. runtime without compromising the accuracy of prediction, thanks to the pre-processing step (Section 4.1). In summary, the experimental results on accuracy, runtime, and scalability against a closely related method from the literature indicate that data pre-processing plays a significant role in reducing processing time without compromising the accuracy of prediction.