Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach (2021)

Abstract: The element of repetition in cyberbullying behavior has directed recent computational studies toward detecting cyberbullying based on a social media session. In contrast to a single text, a session may consist of an initial post and an associated sequence of comments. Yet, emerging efforts to enhance the performance of session-based cyberbullying detection have largely overlooked unintended social biases in existing cyberbullying datasets. For example, a session containing certain demographic-identity terms (e.g., “gay” or “black”) is more likely to be classified as an instance of cyberbullying. In this paper, we first show evidence of such bias in models trained on sessions collected from different social media platforms (e.g., Instagram). We then propose a context-aware and model-agnostic debiasing strategy that leverages a reinforcement learning technique, without requiring any extra resources or annotations apart from a pre-defined set of sensitive triggers commonly used for identifying cyberbullying instances. Empirical evaluations show that the proposed strategy can simultaneously alleviate the impacts of the unintended biases and improve the detection performance.

## introduction cyberbullying has become a prevalent adverse behavior in online social interactions. recent findings indicate that over 35% of young people have been victims of cyberbullying and roughly 15% have admitted to cyberbullying others @xcite . the detrimental consequences of cyberbullying have motivated considerable efforts in various fields to combat cyberbullying. for example, in computational studies of cyberbullying detection -which have been largely aimed at classifying text posted on social media * equal contribution p(@xmath0. ## related work cyberbullying detection. the growing prevalence of social networking sites and convenient access to digital devices and the internet have substantially expedited information-sharing processes. a byproduct of this, however, has been the increased vulnerability of young people, in particular, to one of the most serious online risks -cyberbullying. to help combat cyberbullying, researchers have used various techniques in machine learning and nlp to automate the process of cyberbullying detection. this is also evidenced by a number of recent competitions and workshops for related tasks such as detection of hate speech against immigrants and women @xcite , offensive language identification @xcite , and toxic spans detection @xcite . early works simplified the task as text classification, the input of which are content-based features (e.g., cyberbullying keywords) extracted from a single text (e.g., a tweet) and labels denoting whether the text is relevant to cyberbullying, see, e.g., @xcite . to better leverage the rich information included in social media data, many studies proposed to augment textual features with emotion/sentiment @xcite , social network information such as relational centrality and ego networks @xcite , and other multi-modal information such as location and time @xcite . extensive experimental results revealed that the improvement of these approaches is significant. from the data perspective, research in cyberbullying detection has shifted from modeling a single text to multi-modal data and social media sessions. underpinning these transitions is an increased recognition of two distinct characteris-tics of cyberbullying behavior -repetitiveness and power imbalance @xcite . to address these characteristics, studies such as @xcite proposed to model the structure of a session and temporal dynamics among the comments using han. yet, whereas numerous studies have focused on achieving better prediction performance, these approaches tend to carry or reinforce the unintended social biases in the datasets @xcite . our work thus complements earlier research by examining and mitigating unintended bias in cyberbullying detection models. ## preliminaries cyberbullying is often characterized as a repeated rather than a one-off behavior @xcite . this unique trait has motivated research that focuses on the detection of cyberbullying in entire social media sessions. in contrast to a single text, e.g., a facebook comment or a tweet, a social media session is typically composed of an initial post (e.g., an image with a caption), a sequence of comments from different users, timestamps, spatial location, user profile information, and other social content such as number of likes @xcite . session-based cyberbullying detection presents a number of characteristics such as multi-modality and user interaction @xcite . in this work, because our goal is to mitigate bias in natural language, we focus on text (i.e., a sequence of comments) in a social media session. we formally define session-based cyberbullying detection as follows: definition (cyberbullying detection in a social media session). we consider a corpus of n social media sessions @xmath1, ∫ 2 , ..., ∫ n }, in which each session consists of a sequence of comments denoted as {c 1 , ..., c c }. a session is labeled as either @xmath2. let d be the dimension of extracted textual features (e.g., bag of words) x i for c i . session-based cyberbullying detection aims to learn a binary classifier using a sequence of textual data to identify if a social media session is a cyberbullying instance: ## proposed method an unbiased model for cyberbullying detection makes decisions based on the semantics in a social media session instead of sensitive triggers potentially related to cyberbullying, such as "gay," "black," or "fat." in the presence of unintended bias, a model may present high performance for sessions with these sensitive triggers without knowing their semantics @xcite . in this section, we first discuss how to define and assess bias in the context of session-based cyberbullying detection. we then present the details of our bias mitigation strategy. ## evaluation in this section, we conduct both quantitative and qualitative evaluations to examine the efficacy of our debiasing strategy. 1 in particular, we show that our method can effectively mitigate the impacts of unintended data biases without impairing the model's prediction performance by answering: (1) can we mitigate the unintended bias of machine learning models for detecting cyberbullying sessions by leveraging techniques in rl? two benchmark datasets for cyberbullying detection -instagram (hosseinmardi et al., 2015) and vine (rafiq et al., 2015) -are used for empirical evaluation. the number of sessions in instagram and vine is 2,218 and 970, respectively. both datasets were crawled using a snowball sampling method and manually annotated via the crowdsourcing platform crowdflower. foot_1 sessions containing less than 15 comments were removed to ensure data annotation quality. annotators were asked to examine the image/video, associated caption, and all of the comments in a session before making the final decisions. instagram: instagram foot_2 is a social networking site ranked as one of the top five networks with the highest percentage of users reporting experiences of cyberbullying (the label anti bullying charity, @xcite . each social media session consists of image content, a corresponding caption, and a sequence of comments in temporal order. in total, this dataset is composed of 2,218 sessions, with an average number of 72 comments in each session. vine: vine foot_3 was a mobile application that allowed users to upload and comment on six-second looping videos. each social media session consists of video content, the corresponding caption, and a sequence of comments in temporal order. this dataset contains 970 sessions and each session contains, on average, 81 comments. ## conclusion and future work in this work, we examined unintended biases in datasets for session-based cyberbullying detection. in contrast to conventional data for bias mitigation in text classification, social media sessions consist of a sequence of comments with rich contextual information. to alleviate these unintended biases, we propose an effective debiasing strategy by leveraging techniques in rl. our approach is contextaware, model-agnostic, and does not require additional resources or annotations aside from a predefined set of potentially sensitive triggers related to cyberbullying. empirical evaluations demonstrated that our approach can mitigate unintended bias in the data without impairing a model's prediction accuracy. other types of decisions in sequential decisionmaking processes can impact the underlying user population, thereby influencing future comments generated by users. future research can be directed towards studying the long-term impact of the debiasing strategy, as well as investigating different types of biases in session-based cyberbullying detection, such as gender bias, racial bias, and language bias. our approach can also benefit from integrating previous studies that use data augmentation or swapping methods to counteract bias. due to the challenges of data collection and labeling, validating our approach on datasets across different social media platforms is also an important avenue for future work.