Automated Detection of Cyberbullying Against Women and Immigrants and Cross-domain Adaptability (2020)

Abstract: Cyberbullying is a prevalent and growing social problem due to the surge of social media technology usage. Minorities, women, and adolescents are among the common victims of cyberbullying. Despite the advancement of NLP technologies, the automated cyberbullying detection remains challenging. This paper focuses on advancing the technology using state-of-the-art NLP techniques. We use a Twitter dataset from SemEval 2019 - Task 5 (HatEval) on hate speech against women and immigrants. Our best performing ensemble model based on DistiBERT has achieved 0.73 and 0.74 of F1 score in the task of classifying hate speech (Task A) and aggressiveness and target (Task B) respectively. We adapt the ensemble model developed for Task A to classify offensive language in external datasets and achieved ~0.7 of F1 score using three benchmark datasets, enabling promising results for cross-domain adaptability. We conduct a qualitative analysis of misclassified tweets to provide insightful recommendations for future cyberbullying research.

## introduction and motivation cyberbullying is "the repetitive use of aggressive language amongst peers with the intention to harm others through digital media" @xcite . due to the surge of social media technology use, cyberbullying has become a prevalent and growing social problem. unlike in the physical environment, cyberspace, in particular, online social platforms are not yet evolved sufficiently to prevent people from communicating without disclosing identities, spreading rumours, and harassing others. the risk of and potential consequences caused by cyberbullying are critical including both physical and mental health risk to victims. the impact and consequences are common to all generations (e.g. young, elderly) including emotional and psychological dis-tress, decline in personal/academic development, anti-social behaviour, and, potentially, suicide. the criticality of this societal problem is demonstrated from a study by yale university, commenting "cyberbullying victims are 2 to 9 times more likely to consider committing suicide" across the globe. 1 within australia, the esafety commissioner comments "one in every five australian children aged eight to seventeen are victims of cyberbullying (2018)". 2 adolescents, minorities (e.g. refugees, lgbtqi) and women are among common targets of cyberbullying. according to bullying statistics foot_2 , over half of adolescents are victims of cyberbullying and about the same percentage are involved in bullying. despite recent research advancement in hate speech detection @xcite , automated identification of cyberbullying attempts (i.e. repetitive hate speech against an individual or a group) remains as a challenging subtask of nlp. due to diverse variants of language (e.g. hate, intimidation, sarcasm, metaphors) used by bullies and the evolution of language (e.g. slang), particularly among adolescents, the automated detection of cyberbullying is extremely challenging. the example below appears to be misogynistic as it includes the term 'b***h'; however, it is manually classified as not misogyny since the slang 'gay a*s b***h' is commonly used for a male or gay person. to mitigate the research and social problem of cyberbullying, this paper focuses on advancing the technology to classify cyberbullying using stateof-the-art nlp techniques. as a case study, we focus on cyberbulling against women and immigrants. accordingly, our first research question (rq1) asks, can we build machine learning models to outperform current cyberbullying classification systems on women and immigrants?. the findings of rq1 will lead us to explore the limitations of our models and explanations for misclassification. hence, our second research question (rq2) investigates, what is the content of misclassified tweets and how can we categorise them?. finally, to evaluate the validity of our models across external cyberbullying/hate speech datasets, our third research question (rq3) investigates, can we successfully validate machine learning models developed for cyberbullying detection within the context of women and immigrants for other benchmark datasets?. to answer our research questions, we utilise a twitter dataset developed for semeval 2019 -task 5 (hateval) @xcite that includes labels for three sub tasks: 1) hate speech, 2) aggressiveness, and 3) target (individual or group). we adopt a mixed-method study, using a combination of the building of machine learning models (rq1 & rq3) and qualitative content analysis (rq2) as our methodology. we make the following main contributions: cyberbullying is a complex phenomenon that needs multiple psychological, linguistic, and social theories to understand its nature. the identification of cyberbullying is inherently more complex even for humans (except victims) as it involves repetitive behaviour, peer-oriented nature, and intentionality to harm. therefore, we utilise a definition stated in a recent systematic literature review on cyberbullying @xcite as "repetitive use of aggressive language amongst peers with the intention to harm others through digital media". some recent studies @xcite ) including woah foot_3 (previously known as alw) workshop @xcite have focused on hate speech detection as a more general field. despite recent advancement in hate speech detection, recognising cyberbullying in everyday problems is primarily manual based on victim reports or manual moderation. recent studies rely on contextual features such as demography, social network, and sentiments/emotions as features to train cyberbullying classifiers @xcite . conversely, some related workshops such as trac @xcite and challenges such as hateval @xcite , offenseval @xcite contributed to advance the research field by developing systems using cuttingedge nlp techniques like universal encoder -fermi @xcite , lt3 @xcite , ensemble of deep learning models like openai's gpt and transformer models (team nlpr@sapol @xcite ), and bert (nuli @xcite ). some of these systems have surpassed baselines and earned recognition as the best-performing systems in specific subtasks (e.g. nuli achieved 0.82 of f1 score and ranked 1st place in subtask a to classify offensive language while it ranked only in 18th place for subtask c to classify targets such as individuals, group). despite the promise of current systems, these models are not consistent enough to perform reasonably well within all sub tasks of cyberbullying (i.e. hate speech, aggressiveness and target). additionally, these models were not validated across other cyberbullying-related tasks to ensure generalisability. related literature also lacks comprehensive contributions to draw implications on why machine learning models fail to improve further. our work focuses on addressing these three drawbacks. ## research methodology research questions. our research is guided by three research questions, dataset. we utilise a dataset collected from twitter during july to september 2018 for semeval 2019 -task 5 (hateval) challenge @xcite . this challenge was organised to advance the technology to classify cyberbullying against women and immigrants. tweets were collected both from english and spanish language. we utilise only the english dataset in this paper. the dataset contains a set of tweets and their labels; hs -hate speech (0 -no, 1 -yes), tr -target range (0 -generic group, 1 -individual), ag -aggressiveness (0 -no, 1 -yes). the challenge was divided into two subtasks, task a -classification of hs, and task b -classification of ag and tr. the dataset was labelled via allcloud crowdsourcing platform and added two more experienced annotators to determine the final labels. inter-rater reliability for hs, tr, and ag is 0.83, 0.7, 0.73 respectively. the dataset consists of a total of 13,000 tweets with 10,000 for training set (5,000 each for women and immigrant) and 3,000 for test set (1,500 each for women and immigrant). ## results and discussion evaluation metric. to calculate the classification effectiveness, we use different metrics in each subtask. task a uses the macro-averaged f1 score while task b uses exact match ratio (emr) along with macro-averaged f1 score @xcite . the calculation of emr is discussed in basile et al. (2019). baselines. to evaluate our models (see section 5.1), we utilise top-ranked hateval systems and a system used by basile et al. (2019) as our baselines, 1. task a. fermi (indurthi et al., 2019) using the svm model with google's universal sentence encoder (cer et al., 2018) (refer as 'svm+use') surpassed svc and mfc baselines of hateval challenge. 2. task b. lt3 (bauwelinck et al., 2019) ranked top in task b. ## discussion the ultimate goal of our work is to advance the technology to detect and classify cyberbullying using state-of-the-art nlp techniques, with the longterm aim of enabling social media as a safe space for all users. we developed distilbert-based ensemble model per task as a basis to answer our rq1. with an initial poor performance using a test set of 'original' hateval dataset, we suggest developing a novel version of the original dataset (i.e. 'adjusted') through merging, shuffling and splitting. the 'adjusted' dataset contributed to better performance of f1 score of 0.73 and 0.74 for task a and b respectively. the six categories of misclassified tweets that emerged from our qualitative analysis (rq2) build a point of reference for the content of such misclassifications. this initial categories can help researchers to understand the grounds to improve automated cyberbullying classification. also, the categories identified through this research can serve as a guide which could extend as a conceptual framework for future qualitative and quantitative cyberbullying research. additionally, the categories along with the frequencies that we report in this work provide implications for researchers to collect, annotate, and revise their datasets that could minimise the likelihood of misclassification produced by machine learning models including providing additional contextual information about data. conversely, this raises new research questions on whether we could improve the performance of machine learning models further without relying on demographic data such as gender and data on language evolution such as out-of-vocabulary slang and abbreviations. the findings from our rq3 on generalisability of pre-trained models on other cyberbullying-related tasks demonstrated reasonable accuracy (∼0.7). a possible explanation of not achieving more could be due to pre-trained models might biased within women and immigrant context (e.g. specific hashtags, misogyny) and not be the best option for classifying 'general' offense-related tasks. as a solution, future models could augment data from gen-eral as well as specific datasets (e.g., racial @xcite , gendered @xcite ). in addition to the lack of contextual information that limits our model improvement further, this research is subject to implicit bias of annotators when judging categories to answer rq2. as a solution, our future work will incorporate a semiautomated approach for misclassification annotation by reusing readily available lexical resources like mrc psycholinguistic database @xcite , liwc @xcite to obtain initial codes and employ at least three annotators to refine the codes. furthermore, our 'adjusted' dataset may not provide a robust solution in terms of replicability. therefore, we intend to create a couple of 'adjusted' datasets and report the average of performance in our future works. we also share our current 'adjusted' dataset to enable replication of experiments. in summary, we propose that future cyberbullying classification models need to concentrate on incorporating state-of-the-art solution to common nlp problems like language evolution, sarcasm detection, and pronoun resolution. additionally, future research should also focus on advancing the prediction of demographic information such as gender, age, and personality from data within an ethical framework without reidentifying twitter profiles. ## conclusions due to massive participation in social media, manual moderation of cyberbullying is an extremely labour-intensive task which leads to delay in taking action against bullies while protecting victims. accordingly, automated classification of cyberbullying emerged and remains as a challenging nlp task. this research contributes to develop machine learning models for cyberbullying classification. through a qualitative content analysis, we also contributed to develop a coding schema to deepen the understanding of misclassifications produced by models, enabling future researchers to minimise the impact of data for poor model performance. when social media platforms are equipped with effective cyberbullying detection models, victimised communities will be able to discuss their concerns openly, without harassment.