# Abusive Content Detection in Arabic Tweets Using Multi-Task Learning and Transformer-Based Models
## 6 Conclusions  
In this work, we present a framework for automatic abusive content detection in Arabic using the multi-aspect annotated dataset. Three main approaches were applied (ML, DL, and pretrained Arabic LMs) and we then comprehensively evaluated the performance for each approach and compared the obtained results with the previous study. TF-IDF feature representation was used with ML baseline models (SVM and NB). For the DL approach, a word embedding model called AraVec was used with two DL models (the CNN and the LSTM). A fine-tuning strategy was applied in five pretrained Arabic LMs in DA, MSA, and mixed language types. The pretrained Arabic LMs used in this study were MARBERT, ArabicBERT, CAMeLBERT, QARiB, and AraBERTv0.2. Additionally, due to the low resource scenario of the selected dataset, a data augmentation technique was applied using the NLPaug library with pretrained Arabic LMs. The MARBERT and CAMeLBERT show better performance among the other pretrained Arabic LMs. The obtained results outperformed four out of five attributes achieving macro-F1 results of 0.84 for the group attribute, 0.82 for the target attribute, 0.56 for the annotator attribute, and 0.44 for the hostility attribute. Moreover, we investigated the impact of using MTL with a multi-aspect annotated dataset. Therefore, we developed a MTL based on MARBERT, and four different architectures were examined. We then compared the archived performance with the previous study. The results demonstrate that MTL enhanced the performance compared to the existing DL model proposed in the literature. The obtained result outperformed four out of five attributes by achieving macro-F1 scores of 0.71 for the target attribute, 0.91 for the group attribute, 0.34 for the hostility attribute, and 0.23 for the annotator attribute. Finally, our experiments attempted to address the two issues in automatic abusive content detection in Arabic using multiple approaches. However, further investigation is necessary to address one issue in the used dataset: the highly imbalanced dataset. Therefore, applying some algorithms for an imbalanced dataset may enhance the performance, such as the synthetic minority oversampling technique (SMOTE). Nevertheless, increasing the size of the dataset may enhance the detection performance, especially with DL approaches and pretrained LMs, due to the fact that those approaches are data-hungry and require much more data. This study was limited to focusing on the Arabic language only, but it could also use the multi-aspect dataset with multilingual information by applying similar methods or, for example, the BERT multilingual base model. Figure 5: The precision and recall results of all MTL architectures. The performance of the models based on: (**a**) the directness attribute; (**b**) the hostility attribute; (**c**) the target attribute; (**d**) the group attribute; and (**e**) the annotator attribute. **Author Contributions**: Conceptualization, B.A., A.J. and A.A.; methodology, B.A., A.J. and A.A.; software, B.A. ; validation, B.A., A.J. and A.A.; investigation B.A., A.J. and A.A.; writing--original draft, B.A. ; writing--review and editing A.J. and A.A.; supervision, A.J. and A.A.; funding acquisition, none. All authors have read and approved the submission of this paper. All authors have read and agreed to the published version of the manuscript. **Funding**: This research received no external support. **Institutional Review Board Statement**: Not applicable. **Informed Consent Statement**: Not applicable. **Data Availability Statement**: The data are available in the GitHub repository [[https://github.com/HKUST-KnowComp/MLMA_hate_speech](https://github.com/HKUST-KnowComp/MLMA_hate_speech)]([https://github.com/HKUST-KnowComp/MLMA_hate_speech](https://github.com/HKUST-KnowComp/MLMA_hate_speech)) accessed on 12 November 2021. **Acknowledgments**: The authors express their gratitude towards the anonymous reviewers, whose comments greatly contributed to improving this paper.