Leveraging Conflicts in Social Media Posts: Unintended Offense Dataset (2024)

Abstract: In multi-person communications, conflicts often arise. Each individual may have their own perspective, which can differ. Additionally, commonly referenced offensive datasets frequently neglect contextual information and are primarily constructed with a focus on intended offenses. This study suggests that conflicts are pivotal in revealing a broader range of human interactions, including instances of unintended offensive language. This paper proposes a conflict-based data collection method to utilize inter-conflict cues in multi-person communications. By focusing on specific cue posts within conversation threads, our proposed approach effectively identifies relevant instances for analysis. Detailed analyses are provided to showcase the proposed approach efficiently gathers data on subtly offensive content. The experimental results indicate that incorporating elements of conflict into data collection significantly enhances the comprehensiveness and accuracy of detecting offensive language but also enriches our understanding of conflict dynamics in digital communication.

## introduction social media platforms enable communication that transcends physical boundaries and temporal limitations, allowing people from diverse backgrounds to interact regardless of direct connections. however, this diversity can also lead to challenges. conflicts often arise from varying interpretations of responses, result in misunderstandings or offense. while the definitions of offensive language vary, most focus on profanity and the receiver's emotional reaction rather than the sender's intention @xcite . consequently, offensiveness is often interpreted as subjective to the receiver. however, many datasets on offensive language consider only individual texts in their construction, without taking subsequent responses into account. additionally, numerous studies on offensive language employ "intended affective datasets", using intention-related language markers such as "#hate" and "#bully" for data collection. these markers indicate the sender's subjective emotional intent, not necessarily how the receiver's interprets these messages. consequently, recipients may not perceive the message with the intended emotional intensity. furthermore, a message not intended to offend by the sender but perceived as offensive by the receiver constitutes an unintended offense. examples of datasets that overlook such unintended offenses include the waseem dataset or known as hshp (hateful symbols or hateful people) dataset @xcite , davidson dataset or known as ahsd (automated hate speech detection) dataset @xcite , founta dataset or known as ayr (are you a racist or am i seeing things) dataset @xcite , kumar/trac 2018 dataset @xcite , offense/olid dataset @xcite and social bias inference corpus (sbic) @xcite . unintended offensive expressions are understudied compared to the intended one. often, these unintended expressions are not meant to harm and may even appear neutral or positive @xcite ; however, recipients can still perceive offensive implications that were not intended by the sender. the interpretation of such expressions can vary greatly depending on the context. despite widespread acknowledgment that context influences perceived offensiveness @xcite , existing datasets typically isolate offensive expressions from their conversation context. this approach limits the analysis of offensive language and the development of detection models, prompting on an overly simplistic understanding of how offensiveness occurs in real interactions @xcite . therefore, it is crucial to incorpo-1 4512 rate contextual information, such as conversation replies, when creating datasets focused on unintended and implicitly offensive language. one method to address this is by capturing instances from the perspective of the reader, specifically through perceived emotion. @xcite introduced reactive supervision, a method to gather posts based on the perspective of a second or third person, by utilizing cue responses. these are replies that underscore the perceived emotion in a previous post. building on this reactive supervision method, our study proposes a human-interactionbased approach for collecting offensive language. this method extends beyond the individual who initiates the post, re-conceptualizing human interactions in light of a common phenomenon, human conflict. we identify two primary types of conflicts: intra-conflict, which represents internal inconsistency within an individual, and inter-conflict, which relates to perceived disagreements between individuals. figure 1 illustrates an example of interconflict in a conversation, where user a takes offense at remarks made by user b, despite the lack of intended offense. this paper introduces a novel data collection method and a corresponding dataset for studying inter-conflict, called the unintended offense dataset 1 . exploring human conflict deepens our understanding of the dynamics of offensive language. by analyzing inter-conflict instances, we gain insights into how offensive language emerges within social interactions. our data collection strategy captures a broader spectrum of implicit offensiveness than existing datasets and shows less topic bias. reevaluating human interactions through the lens of conflict allows for a more thorough exploration of the complex interplay between human expression 1 the dataset is available at https://github.com/ idea-nthu-taiwan/unintended-offense-tweets and perception. our main contributions are threefold: (1) as far as our knowledge, this work is the first to focus on identifying expressions of conflict using innovative data collection methods for inter-conflict based on perceived emotion. (2) we developed the unintended offense dataset, which includes dialogue context and exhibits reduced sampling bias in topic distribution. (3) we demonstrate that models trained on the existing datasets are limited in their ability to identify instances of unintended offense. ## related works research on offensive language has been widely conducted, including studies on dataset construction. the data collection methods for these datasets vary and include distant supervision, crowdsourcing, data aggregation, and context-or reactionbased approaches. distant supervision methods employ language markers-such as keywords, hashtags, emoticons, and emojis-to capture intended affective behaviors. these markers are often explicit and subjective. for instance, waseem dataset @xcite collected the twitter (currently x) data by searching common slurs and terms often used in religious, sexual, gender and ethnic minorities such as "mkr" and "asian drive". the davidson dataset @xcite utilized the hatebase lexicon for search queries and later annotated as hate speech and offensive. similarly, @xcite collected data using derogatory terms, hashtags, and phrases related to race and religion. a hindi-english code-mixed offensive dataset was created using popular hashtags on sensitive topics like "beef ban" and "election result" @xcite . for evalita 2018's automatic misogyny identification task, the dataset was collected using explicit words like "bi**h" @xcite , later extending to form the hateval dataset through keyword filtering @xcite . the olid dataset @xcite , collected data by searching for topic-related keywords such as "maga" and topic-unrelated keywords (e.g. "she is") and was later expanded to distinguish between implicit and explicit offenses @xcite . a popular offensive public dataset by @xcite was created by randomly sampling twitter data, then extracting negative sentiments and offensive words. kumar et al. (2018) also 2 4513 gathered data from facebook fanpages discussing controversial topics, alongside twitter data. data aggregation is another method used in offensive dataset collection, exemplified by sap et al. (2020), where data from various existing sources and datasets like the waseem dataset was compiled. the ishate (ocampo et al., 2023) dataset enhanced seven existing datasets to address implicit and subtle hate speech. another data collection approach involves identifying hate accounts, as done in @xcite , where data was collected from selfidentified or perceived racist accounts, and in elsherief et al. (2021), which gathered tweets, retweets, and replies from three hate groups with the most followers. some research employed crowdworkers to generate offensive language instances, resulting in implicitly abusive comparison @xcite and euphemistic abuse @xcite datasets. additionally, a toxic dataset leveraging large language models has also been created @xcite . the datasets mentioned earlier typically consist of standalone posts and do not encompass conversations or consider responses as contextual information. an exception is the reddit conversation corpus augmented with automatic responses, called toxichat @xcite . another offensive conversational dataset is conan @xcite , which pairs hate comments with their counter-narratives. @xcite incorporated context by extracting previous messages from posts in the founta dataset. however, some messages might have been deleted or could not be recovered, resulting in varied context sizes. the dataset known as the identification of conversational hate-speech in code-mixed languages (ichcl) @xcite ) is claimed to be the first to incorporate conversation context from the outset. however, while these datasets do take context into account, their collection methods primarily concentrate on intended offenses, leaving unintended understudied. ## datasets analysis several wide-adopted offensive datasets were compared: (1) @xcite ;; ## conclusion this work proposes a data collection framework that considers the context based on post reactions. the framework captures unintended offensive tweets by leveraging the concept of interconflict. using this approach, a dataset of unintended offense posts was successfully compiled. comprehensive analyses and experiments conducted across various datasets revealed that our dataset exhibits rich emotional depth, higher implicitness, and reduced topic bias. the findings also indicate that models trained on existing datasets have difficulty to accurately recognize unintended offense. we believe that our open-source dataset and methodology will facilitate more comprehensive data analysis and research opportunities in this domain.