Conversation Derailment Forecasting with Graph Convolutional Networks (2023)

Abstract: Online conversations are particularly susceptible to derailment, which can manifest itself in the form of toxic communication patterns like disrespectful comments or verbal abuse. Forecasting conversation derailment predicts signs of derailment in advance enabling proactive moderation of conversations. Current state-of-the-art approaches to address this problem rely on sequence models that treat dialogues as text streams. We propose a novel model based on a graph convolutional neural network that considers dialogue user dynamics and the influence of public perception on conversation utterances. Through empirical evaluation, we show that our model effectively captures conversation dynamics and outperforms the state-of-the-art models on the CGA and CMV benchmark datasets by 1.5\% and 1.7\%, respectively.

## introduction the widespread availability of chat or messaging platforms, social media, forums and other online communities has led to an increase in the number of online conversations between individuals and groups. in contrast to offline or face-to-face communication, online conversations require moderation to maintain the integrity of the platform and protect users' privacy and safety @xcite . moderation can help to prevent harassment, trolling, hate speech, and other forms of abusive behavior @xcite . it can also help to prevent and address conversation derailment. conversation derailment refers to the process by which a conversation or discussion is redirected away from its original topic or purpose, typically as a result of inappropriate or off-topic comments or actions by one or more participants. in online conversations, derailment can be exacerbated by the lack of nonverbal cues and the perceived anonymity that can be provided by the internet. conversation derailment can lead to confusion, frustration, and a lack of productivity or progress in the conversation. in this research, we examine the problem of forecasting conversation derailment. the ability to predict conversation derailment early has multifold benefits: (i) it is more timely, as it allows for proactive moderation of conversations (before they cause any harm) due to early warning, (ii) it is more scalable, as it allows to automatically monitor large active online communities, a task that is otherwise time-consuming, (iii) it is more cost-effective, as it may provide enough aid to limit the number of human moderators needed, and (iv) it may identify upsetting content early and prevent human moderators from being exposed to it. turn user text label n -3 a "proper use of an editor's history includes fixing errors or violations of wikipedia policy or correcting related problems on multiple articles." n -2 b "it's very clear that you just go to my contributions list and look to see what biography articles i've worked on, then you go and look to see if you can find something wrong with them. " n -1 a "so, what is wrong with fixing things? at the top of my talk page, it says to keep it on your watchlist. " n b "you cannot possibly be too stupid to understand the warning i'm giving you. i'm not going to repeat it." ? there exist two popular benchmark datasets typically employed for the problem of interest: conversations gone awry (cga) @xcite and reddit changemyview (cmv) @xcite . both datasets contain multi-party conversations, including the text and an anonymous user id of each utterance, along with a label annotation on whether the con-versation will derail or not. cmv also includes a public vote on each of the utterances that provides the public perception of individuals towards it. figure 1a shows an example multi-party conversation from cga that is not sequential in nature. note as well that some participants are in either agreement or disagreement of a heated argument. graph models are more accustomed to represent such dialogue dynamics. figure 1b shows an example conversation from cmv that shows sample voting scores on each utterance in the conversation that could be related to derailment. graph neural networks have been successfully used to model conversations for downstream classification tasks. for example, they have shown promise in forecasting the next emotion in a conversation @xcite , a problem similar to that of interest in this work. this motivated us to explore this line of research and make the following contributions: the remainder of the paper is organized as follows. section 2 reviews related work. the technical problem of interest is presented in section 3. section 4 presents the proposed models. section 5 presents the experimental setup, and section 6 presents the results and a discussion. we conclude in section 7. ## related work there has been considerable research attention on the problem of detecting various forms of toxicity in text data. there are methods for identifying cyberbullying @xcite , hate speech @xcite , or negative sentiment @xcite or lowering the intensity of emotions @xcite . these methods are useful in filtering unacceptable content. however, the focus of these models is on mostly analyzing or classifying already posted harmful texts. the craft models introduced by @xcite are the first models to go beyond classification of hate speech to addressing the problem of forecasting conversation derailment. the craft models integrate two components: (a) a generative dialog model that learns to represent conversational dynamics in an unsupervised fashion, and (b) a supervised component that fine-tunes this representation to forecast future events. as a proof of concept, a mixed methods approach combining surveys with randomized controlled experiments investigated how conversation forecasting using the craft model can help users @xcite and moderators @xcite to proactively assess and deescalate tension in online discussions. extending the hierarchical recurrent neural network architecture with three task-specific loss functions proposed by @xcite was shown to improve the craft models. after pretrained transformer language encoders such as bert @xcite proved to be successful at various nlp tasks, @xcite explored how they can be used for forecasting derailment. the model in this work consists of a bert checkpoint with a sequence classification (sc) head. similarly, de kock and vlachos (2021) evaluated feature-based and neural models to predict whether disagreements in wikipedia talk page conversations will be escalated to mediation by a moderator. @xcite studied the relationship between structure and toxicity in conversations on twitter at individual, dyad, and group level, and found that social relationships among users influence their behaviors. @xcite also studied the differences between toxic and non-toxic conversations on twitter, highlighting important differences between user engagement and toxicity. while these recent works stress the importance of user characteristics in conversation modeling, to our knowledge, models that incorporate such signals for the task of predicting derailment have remained unexplored. here we propose graph-based models for leveraging user-specific information. graph convolutional neural networks have been used for conversation classification. in one popular application, emotion estimation, the graph model is used to ac-count for speaker related information @xcite . other work have used similar graph neural networks to forecast emotions @xcite @xcite . ## problem definition in this section, we formally define the problem of forecasting conversation derailment. for a conversation @xmath0, t 2 , ..., t n } , {u 1 , u 2 , ..., u n } , {s 1 , s 2 , ..., s n }} consisting of n turns, the last turn (i.e., the n 'th turn) is the potential site of derailment where @xmath1, personal attack} denotes the label of this turn. for the ith turn, t i denotes its text, u i denotes its user, and s i denotes an optional score, e.g., number of votes (up-vote/down-vote). an up-vote is a positive impression and a down-vote is a negative impression on the turn utterance. the goal is to forecast the derailment label l of the n 'th turn given a conversation c up to n -1 turns (i.e., without any information about the n th turn). ## model for forecasting conversation derailment in this section, we describe our proposed forecasting graph convolutional neural network model, (fgcn), visualized in figure 2 . ## results and discussion in the forecasting conversation derailment experiment we report the results of the static and dynamic training of our model and its variants and compare with baselines. in the analyzing mean forecast horizon experiment we show how early each model can forecast derailment. figure 4 : forecast horizon on the cga dataset with a model drawn at random from among the 10 available ones. a horizon of 1 means that an upcoming derailment was only predicted on the last turn before it occurred. ## conclusion unlike previous models which were based on simpler sequence models, fgcn is built on a graph convolutional neural network and is able to capture the dynamics of multi-party dialogue, including user relationships and public perception of conversation statements. fgcn performed significantly better than state-of-the-art models on two widely used benchmark datasets, cga and cmv. conversation derailment is a significant issue that frequently and severely impacts our online social interactions, whether in casual settings or more formal contexts such as online learning or remote work. the ability to accurately predict derailment has the potential to enhance the effectiveness of moderation and thus protect individuals who are vulnerable to emotional abuse or harm and improve the overall quality of online interactions.