# A Multitask Learning Framework for Abuse Detection and Emotion Classification
## 4 Experiment
### Results  
We tested the performance of the proposed framework from the two aspects of test accuracy and weighted F1 value. The specific results are shown in Tables 3 and 4. **Accuracy and weighted F1**. First, we note that compared to character-level methods, word-level methods could achieve better results regardless of accuracy or weighted F1 value. We attribute its effectiveness to the semantic information contained in the pretrained static word vectors (GloVe), whereas in the character-level approaches, we needed to randomly assign the character vector. This inspired us to seek more effective word/sentence feature representation, so we used a pretrained BERT as our encoder. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Datasets** & **HatEval** & **Davids** & **OffEval** & **FNUC** & **StormW** \\ \hline Epoch & 20 & 20 & 20 & 20 & 20 \\ Mmax length & 50 & 50 & 50 & 50 & 50 \\ Learning rate & \(2\times 10^{-5}\) & \(2\times 10^{-5}\) & \(2\times 10^{-5}\) & \(2\times 10^{-5}\) & \(2\times 10^{-5}\) \\ Batch size & 256 & 64 & 64 & 64 & 64 \\ \(\lambda\) & 0.9 & 0.6 & 0.9 & 0.5 & 0.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Parameter settings on different datasets. Second, we found that the MTL architectures with emotion classification tasks had obvious advantages over the RNN-based approaches. Although hyper-RNN can learn more about abuse detection through character-level features than ordinary word-level RNN methods, it was obvious that the emotion classification auxiliary task helped more. This inspired us to use additional multitasking frameworks to facilitate the abuse detection results. At the same time, it also verified the validity of our proposed emotion classification label generation method. Thirdly, our method achieved optimal results in all datasets regardless of accuracy or weighted F1 values, especially compared with BERT. This showed that the proposed multitask learning method can be further be expanded on the basis of BERT. In addition, due to the small scale of FNUC, BERT was easy to overfit, so its effect was even inferior to CNN, RNN and other methods. However, the addition of the emotion classification task can effectively improve the overfitting problem of pretrained models on small datasets, which further explains the necessity to introduce multitasking. **Macro F1. ** We also compared our model with other multitask abuse detection models and HateBERT, a pretrained model for abuse detection. Since macro F1 was used as the evaluation in [10], we also used macro F1 to reevaluate our model. First, for the OffEval dataset, our multitasking approach was better because the BERT encoder was more advantageous. Compared with HateBERT, our method was also competitive, suggesting that sentiment analysis can indeed promote the results of abuse detection, even comparable to the domain-pretraining model. Therefore, we also explored the ability of our method to combine with HateBERT in subsequent analysis. It should be noted that HateBERT achieved a significant improvement on HateEval due to the usage of gradient clipping. We tried to remove the gradient clipping in the actual experiment and achieved a result of 53%, which was similar to the result of [10].