HENIN : Learning Heterogeneous Neural Interaction Networks for Explainable Cyberbullying Detection on Social Media (2020)

Abstract: In the computational detection of cyberbullying, existing work largely focused on building generic classifiers that rely exclusively on text analysis of social media sessions. Despite their empirical success, we argue that a critical missing piece is the model explainability, i.e., why a particular piece of media session is detected as cyberbullying. In this paper, therefore, we propose a novel deep model, HEterogeneous Neural Interaction Networks (HENIN), for explainable cyberbullying detection. HENIN contains the following components: a comment encoder, a post-comment co-attention sub-network, and session-session and post-post interaction extractors. Extensive experiments conducted on real datasets exhibit not only the promising performance of HENIN, but also highlight evidential comments so that one can understand why a media session is identified as cyberbullying.

## introduction in recent years, cyberbullying has become one of the most pressing online risks among youth and raised serious concerns in society. cyberbullying is commonly defined as the electronic transmission of insulting or embarrassing comments, photos or videos, as illustrated in figure 1 . harmful bullying behavior can include posting rumors, threats, pejorative labels, and sexual remarks. research from the american psychological association and the white house has revealed more than 40% of young people in the us indicate that they have been bullied on social media platforms @xcite . such a growing prevalence of cyberbullying on social media has detrimental societal effects, such as victims may experience lower self-esteem, increased suicidal ideation, and a variety of negative emotional responses @xcite . therefore, it has become critically important to be able to detect and prevent cyberbullying on social media. research in computer science aimed at identifying, predicting, and ultimately preventing cyberbullying through better understanding the nature and key characteristics of online cyberbullying. in the literature, existing efforts toward automatically detecting cyberbullying have primarily focused on textual analysis of user comments, including keywords @xcite @xcite and sentiments analysis @xcite . these studies attempt to build a generic binary classifier by taking highdimensional text features as the input and make predictions accordingly. despite their satisfactory detection performance in practice, these models largely overlooked temporal information of cyberbullying behaviors. they also ignore user interactions in social networks. furthermore, the majority of these methods focus on detecting cyberbullying sessions effectively but cannot explain "why" a media session was detected as cyberbullying. given a sequence of comments with user attributes, we think sequential learning can allow us to better exploit and model the evolution and correlations among individual comments. besides, graph-based learning can enable us to represent and learn how users interact with each other in a session. this work aims to detect cyberbullying by jointly exploring explainable information from user comments on social media. to this end, we build an explainable cyberbullying detection framework, heterogeneous neural interaction networks (henin), through a coherent process. henin consists of three main components that learn various interactions among heterogeneous information displayed in social media sessions. a comment encoder is created to learn the representations of user comments through a hierarchical self-attention neural network so that the semantic and syntactic cues on cyberbullying can be captured. we create a postcomment co-attention mechanism to learn the interactions between a posted text and its comments. moreover, two graph convolutional networks are leveraged to learn the latent representations depicting how sessions interact with one another in terms of users, and how posts are correlated with each other in terms of words. specifically, we address several challenges in this work: (a) how to perform explainable cyberbullying detection that can boost detection performance, (b) how to highlight explainable comments without the ground truth, (c) how to model the correlation between posted text and user comments, and (d) how to model the interactions between sessions in terms of users, and the interactions between textual posts in terms of words. our solutions to these challenges result in a novel framework henin. our contributions are summarized as follows. (1) we study a novel problem of explainable cyberbullying detection on social media. (2) we provide a novel model, henin foot_0 , which jointly exploits posted text, user comments, and the interactions between sessions and between posts to learn the latent representations for cyberbullying detection. (3) experiments conducted on instagram and vine datasets exhibit the promising performance of henin, and the evidential comments and words highlighted by henin, for detecting cyberbullying media sessions with explanations. ## related work relevant studies can be categories into social contexts-based and user comment-based approaches. social contexts-based approaches utilize three categories of features, user-based, post-based, and network-based. (a) post-based features rely on text analysis to identify cyberbullying evidences (e.g., profane words) on social media @xcite @xcite . @xcite point out latent semantic analysis(lsa) and latent dirichlet allocation (lda) can be used to learn latent representations of posts. in addition, sicd @xcite further models post sentiments for cyberbullying detection. (b) user-based features are extracted from user profiles to measure their characteristics. gender-specific features, user's past posts, account registration time, and frequently-used words are useful user-based features @xcite . (c) existing studies @xcite @xcite also prove that network-based features are effective in detecting cyberbullying. these features are learned by constructing propagation networks or interaction networks that depict how posts are spread and how users interact with each other. user comment-based approaches utilize the sequence of user comments to detect cyberbullying of the source post. concise @xcite is a sequential hypothesis testing method conducted on the comment sequence to select the significant comment features. @xcite detect harassment-based cyberbullying by identifying expert-provided key phrases from user comments. ## problem statement let @xmath0, s 2 , ..., s m } denote a corpus of m social media sessions. each media session contains the posted text and its subsequent comments. let p be a posted text, consisting of n words {w i } n @xmath1. let @xmath2, c 2 , ..., c t } be a set of t comments related to the post p , where each comment c @xmath3, w j 2 , ...w j q j } contains q j words. let g @xmath4, e s ) be a session-session weighted graph, in which we consider each media session as a node @xmath5,s j ) ∈ e s . let g @xmath6, e p ) be a post-post weighted graph, in which we consider each posted text as a node @xmath7,p j ) ∈ e p . we treat the cyberbullying detection problem as the binary classification problem, i.e., each media session is associated with a binary label @xmath8, 1} with 1 representing a bullying session, and 0 representing a non-bullying session. at the same time, we aim to learn a rank 𝐋 : product 𝐇 : sum 𝐚 : softmax 𝐜: product 𝐋: product 𝐇 : sum 𝐚 : softmax 𝐩 : product concatenate softmax 𝐲 : prediction 𝐰 source post 𝐰 𝐰 𝐰 ... 𝐰 comment 𝐰 𝐰 𝐰 ... ... ... 𝐰 comment 𝐰 𝐰 𝐰 ... comment encoding 𝐩 𝐩 𝐩 ... gru gru gru ... 𝐰 𝐰 𝐰 ... gru gru gru ... 𝐜 𝐜 𝐜 ... 𝐰 𝐰 𝐰 ... average pooling self attention ... 𝐜 𝐜 𝐜 𝐬 average pooling self attention 𝐫 𝐫 𝐫 ... 𝐱 𝐱 𝐱 ... gcn gcn gcn 𝐫 𝐫 𝐫 ... 𝐱 𝐱 𝐱 ... gcn gcn gcn session-session interaction extractor post-post interaction extractor cyberbullying prediction word encoder ... sentence encoder post word embedding list rc from all comments in {c j } t @xmath9, according to the degree of explainability, where rc k denotes the k th most explainable comment. the explainability of comments denotes the impact degree of detecting the media session is cyberbullying or not. formally, we can represent the problem as explainable cyberbullying detection. problem: given a posted text p , a set of related comments c, the session graph g ss and the post graph g pp , the goal is to learn a cyberbullying detection function f : f (p, c, g ss , g pp ) → (ŷ, rc), such that it maximizes the prediction accuracy with explainable comments ranked highest in rc. ## the proposed henin model in this section, we present the details of the proposed henin, which jointly learns the hierarchical self-attention and graph convolutional neural networks for cyberbullying detection. it consists of four major components (figure 2 ): (1) a comment encoder (including word-level and sentence-level), (2) a post-comment co-attention mechanism, (3) session-session and post-post interaction extractors, and (4) a cyberbullying prediction component. the comment encoder component depicts the modeling from the comment linguistic features to latent representation features through hierarchical word-level and sentence-level self-attention net-works. the explainability degree of comments is learned through the attention weights within sentence-level self-attention learning. the postcomment co-attention mechanism is performed in the level of word embeddings. the mutual interactions between the posted text and comments can be learned through the post-comment co-attention. on the other hand, the session-session interaction extractor and the post-post interaction extractor aim at modeling how users interact across media sessions, and how words are correlated across posts, through two graph convolutional neural networks. finally, the cyberbullying prediction is made by concatenating the representations of the aforementioned three components. ## conclusion cyberbullying detection on social media attracts growing attention in recent years. it is also crucial to understand why a media session is detected as cyberbullying. thus we study the novel problem of explainable cyberbullying detection that aims at improving detection performance and highlighting explainable comments. we propose a novel deep learning-based model, heterogeneous neural interaction networks (henin), to learn various feature representations from comment encodings, post-comment co-attention, and graph-based interactions between sessions and posts. experimental results exhibit both promising performance and evidential explanation of henin. we also find that the learning of graph-based session-session and post-post interactions contributes most to the performance. such results can encourage future studies to develop advanced graph neural networks in better representing the interactions between heterogeneous information. in addition, it is worthwhile to further model information propagation and tem-