## 5 Performance Evaluation
### Character-Level CNN-LSTM
\begin{table}
\begin{tabular}{l l c c c c c} \hline \hline
**Exp** & **Model** & **Test** & **Test** & **No. of** & **Best** & **Worst** \\
**No.** & **Model** & **Accuracy** & **Loss** & **Epochs** & **Accuracy** & **LossS** \\ \hline \multirow{4}{*}{1} & 3 conv. layers of 100 filters, & & & & & \\  & 1 LSTM layer of 60 neurons & 0.8520 & 0.2963 & 50 & 24 min 40 s & 0.8584 at 49 th epoch & 0.2808 at 49 th epoch \\  & and dropout 0.5 & & & & & \\ \hline \multirow{4}{*}{2} & 3 conv. layers of 100 filters, & & & & & \\  & 1 LSTM layer of 60 neurons & 0.8369 & 0.3049 & 50 & 26 min 44 s & 0.8608 at 49 th epoch & 0.2800 at 49 th epoch \\  & and dropout 0.2 & & & & & \\ \hline \multirow{4}{*}{3} & 3 conv. layers of 100 filters, & & & & & \\  & 1 LSTM layer of 60 neurons & 0.8562 & 0.2808 & 50 & 20 min 51 s & 0.8593 at 49 th epoch & 0.2885 at 49 th epoch \\  & and dropout 0.4 & & & & & \\ \hline \multirow{4}{*}{4} & 3 conv. layers of 256 filters, & & & & & \\  & 1 LSTM layer of 60 neurons & 0.8644 & 0.2705 & 50 & 54 min 18 s & 0.8644 at 50 th epoch & 0.2705 at 50 th epoch \\  & and dropout 0.2 & & & & & \\ \hline \multirow{4}{*}{5} & 3 conv. layers of 256 filters, & & & & & \\  & 1 LSTM layer of 60 neurons & 0.8582 & 0.2952 & 50 & 60 min 52 s & 0.8582 at 50 th epoch & 0.2952 at 50 th epoch \\  & and dropout 0.5 & & & & & \\ \hline \multirow{4}{*}{6} & 3 conv. layers of 100 filters, & & & & & \\  & 1 LSTM layer of 100 neurons & 0.8493 & 0.3091 & 50 & 26 min 11 s & 0.8500 at 47 th epoch & 0.3002 at 47 th epoch \\  & and dropout 0.2 & & & & & \\ \hline \multirow{4}{*}{7} & 3 conv. layers of 100 filters, & & & & & \\  & 1 LSTM layer of 100 neurons & 0.8616 & 0.2781 & 50 & 25 min 17 s & 0.8616 at 50 th epoch & 0.2781 at 50 th epoch \\  & and dropout 0.5 & & & & & \\ \hline \multirow{4}{*}{8} & 3 conv. layers of 256 filters, & & & & & \\  & 1 LSTM layer of 100 neurons & 0.8587 & 0.2948 & 50 & 60 min 24 s & 0.8587 at 50 th epoch & 0.2948 at 50 th epoch \\  & and dropout 0.5 & & & & & \\ \hline \multirow{4}{*}{9} & **3 conv. layers of 256 filters,** & & & & & \\  & **1 LSTM layer of 100** & **0.8657** & **0.2688** & **50** & **60 min 8 s** & **0.8657 at 50 th epoch & **0.2688 at 50 th epoch** \\ \cline{1-1}  & **neurons and dropout 0.2** & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Experiments and results of character-level CNN-LSTM (using imbalanced training dataset). The model in experiment 9 has a precision of 67%, recall of 91%, and F1-score of 77%. Although we trained the model using an imbalanced dataset, the model did not favor the majority class (hate). However, in order to improve our results, we created another model with the same hyperparameters as in experiment 9 and trained it with the over-sampled version of the training dataset and evaluated its performance on the test set. The best performance of this model on the test set was an 88.97% accuracy and 23.85% loss in the 50 th epoch with a runtime of 88 min 15 s, which is shown in Table 6. Hence, the accuracy of the model improved by 2.4% when training it with a balanced dataset. The model has a precision of 74%, recall of 88%, and an F1-score of 80%. Figure 7 shows the accuracy and loss for 50 epochs of training and testing of the model. Table 7 reports the results of running our models (Models b-e) and [PERSON] et al. 's [7] model (Model a) on the same dataset. Both our neural network architectures (CNN and CNN-LSTM) performed better (and similar at worse) compared to [PERSON] et al. 's [7] model in terms of accuracy, and significantly better in terms of runtime. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & **Test** & **Test** & **No. of** & \multirow{2}{*}{**Runtime**} & **Best** & **Worst** \\  & **Accuracy** & **Loss** & **Epochs** & & **Accuracy** & **Loss** \\ \hline
3 conv. layers of 256 filters, 1 LSTM layer of 100 neurons and dropout 0.2 & 0.8897 & 0.2385 & 50 & 88 min 15 s & 0.8897 at 50 th epoch & 0.2385 at 50 th epoch \\ \hline \hline \end{tabular}
\end{table}