Automatic cyberbullying detection : A systematic review

Automatic cyberbullying detection is a task of growing interest, particularly in the Natural Language Processing and Machine Learning communities. Not only is it challenging, but it is also a relevant need given how social networks have become a vital part of individuals' lives and how dire the consequences of cyberbullying can be, especially among adolescents. In this work, we conduct an in-depth analysis of 22 studies on automatic cyberbullying detection, complemented by an experiment to validate current practices through the analysis of two datasets. Results indicated that cyberbullying is often misrepresented in the literature, leading to inaccurate systems that would have little real-world application. Criteria concerning cyberbullying definitions and other methodological concerns seem to be often dismissed. Additionally, there is no uniformity regarding the methodology to evaluate said systems and the natural imbalance of datasets remains an issue. This paper aims to direct future research on the subject towards a viewpoint that is more coherent with the definition and representation of the phenomenon, so that future systems can have a practical and impactful application. Recommendations on future works are also made. 1. Defining and detecting cyberbullying online With the development of new Information and Communication Technologies (ICT) and the vast proliferation of Social Network Systems among adolescents to communicate online, interpersonal relationships have gained a new medium through which communication is established. In these online intercommunications, it is common to see social interactions involving offensive online content, since this is one of the main expressions of aggression in cyber harassment situations, such as cyberbullying (Souza, Veiga Simão, Ferreira & Ferreira, 2017). Cyberbullying has been defined in various ways but there is a large consensus in the literature that it involves intentional, cruel and repeated behavior among peers, by means of electronic media (Olweus, 2012; Wright, 2017). The content used by adolescents in online interactions has been examined in the literature with the intent of understanding what determines how they communicate with data gathered from self-reported data (e.g., Veiga Simão, Ferreira, Francisco, Paulino, & Souza, 2018). Automatic cyberbullying detection can be used within digital tools to prevent and intervene in cyberbullying along with messages to promote self-reflection of behavior (Van Royen, Poels, Vandebosch, & Adam, 2017). Thus, classifiers need to be as accurate as possible to reduce the incidence of cyberbullying. It is still difficult however, to capture these incidents online almost in realtime, depending on the operationalization that is given to cyberbullying by automatic cyberbullying detection systems, among other factors. In light of this concern, we propose to understand whether cyberbullying has been automatically detected according to the criteria that constitute its definition and characteristics in order to improve current classifiers, and develop more effective cyberbullying digital interventions. To reach this objective, this study presents an in-depth analysis of research that has focused on automatic cyberbullying detection through a quantitative systematic review approach. In addition, we complement this approach with an extensive experiment to assess current practices, namely, with the use of feature engineering. Moreover, this study provides guidelines for future research and suggests improvements to current datasets and classifiers in automatic cyberbullying detection in line with the findings from the systematic review presented. Research has shown that cyberbullying may affect and be https://doi.org/10.1016/j.chb. 2018.12. 021 Received 2 October 2018; Received in revised form 5 December 2018; Accepted 11 December 2018 ∗ Corresponding author. Faculdade de Psicologia, Universidade de Lisboa, Alameda da Universidade, 1649-013 Lisboa, Portugal. E-mail address: nadia@campus.ul. pt (N. Pereira). Computers in Human Behavior 93 (2019) 333–345 Available online 12 December 2018 0747-5632/ © 2018 Elsevier Ltd. All rights reserved. T determined by social relationships, such as the sense of belonging to a social group (Glover, Gough, Johnson, & Cartwright, 2000; Spears, Slee, Owens, & Johnson, 2009), since the latter cannot be developed without the interference of the social technological world (Spears et al., 2009). Different forms of cyber harassment such as cyberbullying are increasing in online social interactions, particularly among youth, and may affect adolescents' mental health and well-being (Fridh, Lindström, & Rosvall, 2015; Nixon, 2014). Therefore, it is crucial to detect its occurrence almost in realtime so that effective intervention to resolve cyberbullying incidents is developed. In addition, effective intervention could include the phenomenon's detection online firstly and foremost so that victims may be aided in a timely fashion. Cyberbullying can be considered a form of bullying in a new context (Li, 2006). Bullying refers to repeated abusive behavior in which there is an imbalance of power among peers with the intent of harming others in a prolonged manner (Olweus, 1993). Some of the main characteristics of bullying include the intention to physically, psychologically or socially harm the victim, the repetition of aggressive behavior over time, and the physical, mental and/or social imbalance of power between the bully and the victim (Olweus, 1993; Smith & Brain, 2000). Due to the negative psychological and physical consequences that may affect adolescents (Anderson & Sturm, 2007), and which have been reported in the literature as being suicide ideation, depression, anxiety, cutting, negative emotions, and psychosomatic symptoms (Fridh et al., 2015; Miller, 2017; Nixon, 2014), it is fundamental that its occurrence be tracked in order to prevent this type of harm on the victims. Moreover, the consequences of cyberbullying may affect victims in a continuous manner, and therefore, research must focus on the severity of the incident within the specific context in which it occurs and the circumstances surrounding it (Hinduja & Patchin, 2009). Since this paper aims to provide an in-depth analysis of research that has focused on automatic cyberbullying detection, it is crucial to identify how it has been operationalized in various studies focusing on this type of detection, according to the definitions provided in the cyberbullying literature. For instance, several studies in the field of automatic cyberbullying detection (e.g., Bayzick, Kontostathis, & Edwards, 2011; Reynolds, Kontostathis, & Edwards, 2011) have used Patchin and Hinduja’s (2006, p.152) definition of cyberbullying which consists of “willful and repeated harm inflicted through the medium of electronic text”. In addition, Nahar, Li, and Pang (2014) and Van Hee and colleagues (2015), used Smith and colleagues' (2008, p.376) definition, where they consider cyberbullying as “an aggressive, intentional act carried out by a group or individual, using electronic forms of contact, repeatedly and over time, against a victim who cannot easily defend him or herself”. Al-Garadi, Varathan, and Ravana (2016) used Salmivalli's definition of cyberbullying (2010, p.112), mentioning that it could “be a sub-type of aggressive behavior in which an individual or group of individuals repeatedly attack, humiliate and/or exclude a relatively powerless person”. Most of these definitions include the use of ICT, and the aggressors' engagement in harmful behavior. Some also refer to the intentionality and repetition characteristics of cyberbullying, and to the fact that the behavior occurs among peers. Lastly, some authors provided their own definitions of cyberbullying with a focus also on intentionality, repetition and harmful behavior (e.g., Dadvar, de Jong, Ordelman, & Trieschnigg, 2012; Dadvar, Trieschnigg, Ordelman, & de Jong, 2013; Huang, Singh, & Atrey, 2014), while others highlighted harassment through malicious comments (Chavan & Shylaja, 2015), and the bully's persistence, power imbalance and visibility in the community (Dinakar, Reichart, & Lieberman, 2011). In this study, we consider cyberbullying as an individual or group's repeated intentional aggressive behavior towards other peer(s) with the intent of harming them by sending offensive content or engaging in other forms of social aggression through the use of digital technologies (Belsey, 2006; Hinduja & Patchin, 2009; Willard, 2005). Accordingly, characteristics such as repetition, intentionality to harm others, the occurrence among peers, and hostile language will be considered as key factors which the current literature in the field often misrepresents, and which could help identify the phenomenon during online interactions among adolescents. In line with the aims of this study, we propose to answer the following research questions: Has cyberbullying been automatically detected in previous research according to the criteria that constitute its definition and characteristics? Which guidelines for future research in automatic cyberbullying detection can be brought forward from the findings of the systematic review presented? To answer the research questions of this study, we will perform a quantitative systematic review of the related work in the area of automatic cyberbullying detection focusing on the following key aspects: a) criteria used to classify cyberbullying; b) methodological concerns which provide validity to the results (e.g., inter-rater reliability), and ensure privacy of data (i.e., users' consent); c) performance metrics evaluation used to develop classifiers of automatic detection. With these research questions, this study contributes towards a better representation of the phenomenon under study, and, consequently, to the development of more accurate classifiers of automatic cyberbullying dete