Abusive Language on Social Media Through the Legal Looking Glass (2021)

Abstract: Abusive language is a growing phenomenon on social media platforms. Its effects can reach beyond the online context, contributing to mental or emotional stress on users. Automatic tools for detecting abuse can alleviate the issue. In practice, developing automated methods to detect abusive language relies on good quality data. However, there is currently a lack of standards for creating datasets in the field. These standards include definitions of what is considered abusive language, annotation guidelines and reporting on the process. This paper introduces an annotation framework inspired by legal concepts to define abusive language in the context of online harassment. The framework uses a 7-point Likert scale for labelling instead of class labels. We also present ALYT – a dataset of Abusive Language on YouTube. ALYT includes YouTube comments in English extracted from videos on different controversial topics and labelled by Law students. The comments were sampled from the actual collected data, without artificial methods for increasing the abusive content. The paper describes the annotation process thoroughly, including all its guidelines and training steps.

## introduction the increased use of social media can worsen the issue of online harassment. nowadays, more than half of online harassment cases happen on social media platforms @xcite . a specific popular form of online harassment is the use of abusive language. one abusive or toxic statement is being sent every 30 seconds across the globe foot_0 . the use of abusive language on social media contributes to mental or emotional stress, with one in ten people developing such issues @xcite . automatic detection tools for detecting abusive language are used for combating online harassment. these tools are mainly based on machine learning algorithms that rely on training data. therefore, there is a need for good quality datasets to create high performing algorithms to alleviate online harassment. there are various datasets in the field of online harassment research. however, there is a lack of standards for developing these resources. these standards include the definitions used to determine what content is abusive and the steps of the annotation process (including the annotators). the lack of standards leads to conflicting definitions, which ultimately results in disagreement within the field regarding which tasks to solve, creating annotation guidelines, and terminology. ## related work creating dataset for online harassment research, including abusive language, has been a challeng-ing task. @xcite review a wide range of datasets -and their creation process -within the field and identify many issues. various approaches have been explored over the years, including different data collection strategies, labelling methodologies and employing different views and definitions of online harassment. in terms of annotation methods, crowdsourcing is a popular option for labelling abusive language data @xcite @xcite @xcite . however, in some instances, a small group of non-experts in harassment research @xcite @xcite or domain experts annotate the data @xcite . the definitions used to label the data can vary as well. at times, definitions are derived from literature @xcite on the topic, or existing social media platform's guidelines @xcite . in other instances, annotators decide by themselves when abuse is present in the text @xcite . a recent direction in the field has been applying legal provisions to decide whether content should be removed, given criminal law provisions on hate speech or incitement to hatred. these approaches represent legal provisions as decision trees that guide the annotation process. @xcite apply this methodology focusing on the german provision related to incitement to hatred. two nonexpert annotators label the data, guided by the created decision tree. the experiments show that there was little difference between using expert and nonexpert annotators in this case. ## data collection we aimed to include a representative sample of abusive language on social media in our dataset. therefore, we did not search directly for abusive content. instead, we chose topics likely to contain abusive comments. we chose three different topics before the video selection: gender identity (gi), veganism (vg), and workplace diversity (wd). the topics generate controversial videos on youtube while not being limited to one type of controversy (e.g. gender identity, diet choices, socioeconomical issues). the videos in gi focus on the disclosure of transgender identity and the impact of transgender people in sports. the videos in the vg category concentrate on describing the vegan movement and influencers deciding to become vegan. in wd, the videos illustrate the gender wage gap and its implications. we searched for content in one language; therefore, the videos and majority of the comments are in english. we manually searched for videos using the topics as keywords. we selected popular videos (considering the number of views) made by well-known influencers posting controversial content. we included three types of videos: personal videos (posted by influencers on the topic), reaction videos (videos in which the author reacts to another video) and official videos (posted by news and media channels). to create our dataset, we retrieved all comments from the selected videos, excluding replies. we removed comments containing urls because these are often spam or make reference to external content. we also removed comments with fewer than three tokens. in total, we obtained 879,000 comments after these steps. out of this sample, we selected 20,215 to annotate. we randomly sampled comments from the total distribution, not attempting to balance the data according to content. we aimed to balance video topics and types equally, but as the total number of comments was not even per video category, the final sample was not perfectly balanced. ## annotation framework datasets presented in state-of-the-art research mainly use several definitions for abusive language or focus on specific phenomena -such as hate speech, racism, and sexism. what constitutes abusive content is often not precisely defined. dataset creators -in an attempt to make these concepts clear -rely on various definitions, ranging from platform guidelines to dictionary entries. our goal is to develop an annotation framework inspired by legal definitions and to define abusive content concretely in light of these concepts. using legal definitions as inspiration can provide a consistent and stable background for deciding which content is abusive, since most occurrences of abusive language are already covered in legal systems. ## annotation process throughout the data annotation process, annotators encode our framework and apply it to label the data. a proper annotation methodology, therefore, is fundamental to ensure data quality. yet, most works presenting abusive language datasets fail to report this process in details. we follow the corpus annotation pipeline proposed by @xcite and thoroughly describe how we conducted its main steps when creating alyt. to measure the reliability of the annotation, we use krippendorf's alpha (α) with ordinal level of measurement @xcite and majority agreement to calculate the overall inter-annotator agreement. @xcite highlight that α is a reliable metric for ordinal annotations, such as the 7-point scale from our framework. ## dataset the labelled dataset includes 19,915 comments, excluding the comments used in the training phase and the sample used for calculating the interannotator agreement. each comment has a label ranging from 1 to 7, corresponding to the scale used in the annotation framework. we also aggregate the labels into classes: values from 1 to 3 correspond to non-abusive content; 5 to 7, abusive; and 4, uncertain. in this section, we analyse the aggregated labels. the annotators labelled 2274 comments as abusive. this number represents 11.42% of the total distribution, showing a low prevalence of abusive content. considering that we selected random ## conclusion this paper presented a dataset of youtube comments in english, labelled as abusive by law students, using a 7-point likert scale. the comments were collected from videos on three controversial topics: gender identity, veganism, and workplace diversity. the dataset includes a sample of the actual amount of extracted comments, without any artificial balancing of the abusive content distribution. we developed an annotation framework that includes legally inspired labelling rules based on european provisions and case law. our annotation process includes developing and refining guidelines through various training sessions with active discussions. our data sampling analysis shows that not purposefully searching for abusive content still leads to a considerable amount of abusive comments, while maintaining the characteristics of the social media platform's data distribution. the content analyses show that alyt contains various expressions of abuse, ranging from different topics and targets. the abusive content is not limited to specific keywords or slurs associated