“Are you kidding me?”: Detecting Unpalatable Questions on R eddit (2021)

Abstract: Abusive language in online discourse negatively affects a large number of social media users. Many computational methods have been proposed to address this issue of online abuse. The existing work, however, tends to focus on detecting the more explicit forms of abuse leaving the subtler forms of abuse largely untouched. Our work addresses this gap by making three core contributions. First, inspired by the theory of impoliteness, we propose a novel task of detecting a subtler form of abuse, namely unpalatable questions. Second, we publish a context-aware dataset for the task using data from a diverse set of Reddit communities. Third, we implement a wide array of learning models and also investigate the benefits of incorporating conversational context into computational models. Our results show that modeling subtle abuse is feasible but difficult due to the language involved being highly nuanced and context-sensitive. We hope that future research in the field will address such subtle forms of abuse since their harm currently passes unnoticed through existing detection systems.

## introduction abusive language and other antisocial behaviour is omnipresent in online discourse. according to a recent survey, 41% of americans have personally experienced some form of online harassment @xcite . to counter abusive behaviour online, different social media platforms implement their own mechanisms such as content moderation, muting or blocking users from posting etc. it is, however, infeasible to manually moderate online communities due to the sheer enormity of content produced every day -twitter, for example, receives over 500 million tweets per day. manual moderation in such a scenario would require humans to read millions of tweets daily which would take an impractical amount of time and other resources. consequently, many computational models have been proposed by the natural language processing (nlp) community to detect online abuse and facilitate automatic content moderation. abuse is an umbrella term which can cover several types of negative expressions. there exists a plethora of abuse detection studies employing different terminology: personal attacks @xcite , bullying @xcite , hate speech @xcite @xcite , nastiness @xcite , harassment @xcite , hostility @xcite , racism or sexism @xcite , abusive language @xcite , aggression @xcite , and others. however, extant work in abuse detection has largely focused on detecting overt abuse ignoring the more subtle forms of abuse which can be just as damaging. this is also noted in a recent survey calling on the nlp community to rethink and expand what constitutes abuse @xcite . in this work, we make three contributions to address this gap in the literature. first, inspired from the theory of linguistic impoliteness, we propose a novel task of detecting a subtler form of abuse called unpalatable questions (uq). it is one of the conventionalized impoliteness formulae introduced by @xcite . we define the uq task as detecting a negatively phrased question designed to antagonise its recipient in online discourse. second, we collect, annotate, and make publicly available a context-aware dataset for the uq task. 1 the data comes from a diverse set of online communities (or subreddits) on the popular social media site reddit. most existing datasets used in abuse detection @xcite @xcite @xcite only include annotations for stand-alone comments or tweets. in comparison, we explicitly consider conversational context during annotation and preserve contextual information in our dataset (see section 4.4 for a detailed comparison). a major limitation of existing abuse detection studies -also pointed out in @xcite @xcite -is that a comment is treated as a single-utterance in isolation, ignoring any conversational context provided by other comments in the discussion. this is problematic since abuse is inherently contextual, and it becomes a major issue when working with subtler forms of abuse such as unpalatable questions. to this end, our third contribution is that we implement a wide array of learning models to detect unpalatable questions and investigate the benefits of incorporating conversational context in our computational models. we adopt the term unpalatable question (uq) from the conventionalised impoliteness formulae introduced by @xcite . although culpeper did not formally define uq, several examples were laid out: 'why do you make my life impossible?', 'which lie are you telling me?', 'what's gone wrong now?'. we find that uqs tend to be rhetorical in nature in that they are usually asked not to elicit an answer but to make a point. in particular, they have a close resemblance to epiplexis: a type of rhetorical question which is asked not to elicit information but to reproach, upbraid, or rebuke @xcite . this can be seen in the examples listed in why uq? @xcite outline a spectrum of abusive behaviour highlighting that existing work only focuses on overt abuse ignoring both the subtler forms and extreme behaviours. as can be seen in figure 1 , our task of detecting a subtler form of abuse is a step towards addressing this gap. moreover, studies in linguistics show that being asked an unpalatable question puts the recipient in a vulnerable position to receive further verbal attacks @xcite . ## data the aim is to detect unpalatable questions in online discourse. for this, we construct a dataset using comments from reddit and annotate them for whether they contain an unpalatable question or not. we also preserve conversational context in the dataset by including the preceding comment in the discussion; therefore, our data consists of (pc i , r i , y i ) tuples denoting the preceding comment, reply foot_1 (or main comment), and the corresponding label respectively. the task is formulated as a binary classification problem where y @xmath0. we collect data from a diverse set of 15 online communities (or subreddits) belonging to different genres: politics, sports, hate and toxic. 4 the subreddits were carefully selected to prevent the dataset from being heavily skewed towards not unpalatable samples since these topics are more likely to involve opinionated and antagonistic discussions. ## methodology in this section, we introduce our methodology for detecting unpalatable questions. ## experiments and results we conduct experiments across two dimensions: we evaluate our computational models on several classification metrics: precision, recall, f1score, and area under receiver operating characteristic curve (auroc) and precision-recall curve (auprc). all reported values are averaged over stratified five-fold cross-validation runs. the empirical results on all-data and high-agreement-data are presented in despite the performance gains observed with using more sophisticated deep learning models, the performance is still poor to be used for any practical applications. this is not surprising given how linguistically nuanced our dataset is and the complexity associated with abusive language detection on noisy social media text @xcite . specifically, learning models struggle to deal with implicit abuse -language which does not immediately convey abuse @xcite . @xcite find that their toxicity classifier fails on data where there were instances of sarcasm, toxicity without employing swear words, and rhetorical questions. we qualitatively examined a random sample of hundred unpalatable samples from our dataset, and found that 65% do not contain swear words and 20% involve sarcasm. similarly, from a random sample of hundred mis-classified unpalatable samples, 72% do not contain swear words and 30% involve sarcasm. moreover, since unpalatable questions are rhetorical in nature, it is not surprising that learning models performed relatively poorly on the task. ## conclusion in this work, we addressed an important gap in the abuse detection literature by introducing a novel task of detecting unpalatable questions. we also released a context-rich dataset for the task and implemented a number of learning models to automatically detect unpalatable questions. our results show that it is difficult to model subtle abuse due to the language being nuanced and context-sensitive. this calls for advancements in natural language understanding methods that can identify such implicit signals and take pragmatic context into account. we hope that future research would explore other forms of abuse and draw inspiration from related fields such as linguistic impoliteness. detecting abuse -both overt and subtle -on the internet would help enhance user's experience online and facilitate civil and productive discussions.