## 7 Experiments
### Evaluation
\begin{table}
\begin{tabular}{c c} \hline \hline
**Hyper-Parameters** & **Values** \\ \hline Dropout rate & [0.1, 0.15] \\ Learning rate & [0.00002, 0.00003] \\ Batch Size & [16, 32] \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Explored combination of the bare BERT-base hyper-parameters during the fine-tuning. After then, we computed the overall metrics and metrics for particular classes. Tables 17 and 18 show how the performance metrics improved after fine-tuning of the probability thresholds. Regarding the interpretability of the trained models, we further explored, how the classifiers performed when predicting the actual class labels. We focused on examination of particular comments, especially those, which were classified correctly in multiple categories and on the other hand, the comments that were misclassified. Many comments were categorized correctly, based on the grammar and language used, e.g., the comment \"u are a gigantic faggot\" was correctly predicted as toxic, obscene, insult, and identity hate. On the other hand, we observed an inconsistency in comments labelling, which is a frequent issue in many hand-labelled datasets. For example, the comment \"Bull... Your mom kicking your ass for not studying is influence of Custom. but then again, without all that comic mischief (such as a dead guy and a crying ilha), life would be pretty fukin boring...\" was predicted by the models as toxic and obscene, because of using an explicit language. However, in the testing data, the comment was assigned just with the toxic label. In similar fashion, a comment \"oh, shutup, you douchey Wikipedia rent-a-cop\" is clearly an insult and was correctly predicted as toxic and insult, but in the testing data was assigned only with the first label. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Class** & **Precision** & **Recall** & **F1 Score** & **MCC** & **Support** \\ \hline
**toxic** & 0.58 & 0.87 & 0.70 & 0.67 & 6090 \\
**severe\_toxic** & 0.40 & 0.23 & 0.29 & 0.30 & 367 \\
**obscene** & 0.60 & 0.80 & 0.69 & 0.67 & 3691 \\
**threat** & 0.63 & 0.37 & 0.47 & 0.48 & 211 \\
**insult** & 0.66 & 0.73 & 0.69 & 0.67 & 3427 \\
**identity\_hate** & 0.73 & 0.51 & 0.60 & 0.61 & 712 \\ \hline \hline \end{tabular}
\end{table} 
Table 16: Results of the individual toxicity classes of bare BERT uncased model after fine-tuning. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Class** & **Precision** & **Recall** & **F1 Score** & **MCC** & **Support** \\ \hline
**toxic** & 0.58 & 0.85 & 0.69 & 0.66 & 6090 \\
**severe\_toxic** & 0.35 & 0.50 & 0.41 & 0.42 & 367 \\
**obscene** & 0.66 & 0.75 & 0.71 & 0.69 & 3691 \\
**threat** & 0.48 & 0.07 & 0.12 & 0.18 & 211 \\
**insult** & 0.67 & 0.68 & 0.68 & 0.66 & 3427 \\
**identity\_hate** & 0.73 & 0.33 & 0.46 & 0.49 & 712 \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Results of the individual toxicity classes of biLSTM + CNN model with custom-trained word2 vec embeddings without standard pre-processing. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Class** & **Precision** & **Recall** & **F1 Score** & **MCC** & **Support** \\ \hline
**toxic** & 0.58 & 0.85 & 0.69 & 0.66 & 6090 \\
**severe\_toxic** & 0.35 & 0.50 & 0.41 & 0.42 & 367 \\
**obscene** & 0.66 & 0.75 & 0.71 & 0.69 & 3691 \\
**threat** & 0.48 & 0.07 & 0.12 & 0.18 & 211 \\
**insult** & 0.67 & 0.68 & 0.68 & 0.66 & 3427 \\
**identity\_hate** & 0.73 & 0.33 & 0.46 & 0.49 & 712 \\ \hline \hline \end{tabular}
\end{table} 
Table 15: Results of the individual toxicity classes of biLSTM + CNN model with pre-trained GloVe Twitter embeddings without standard pre-processing.