Annotating Online Misogyny (2021)

Abstract: Online misogyny, a category of online abusive language, has serious and harmful social consequences. Automatic detection of misogynistic language online, while imperative, poses complicated challenges to both data gathering, data annotation, and bias mitigation, as this type of data is linguistically complex and diverse. This paper makes three contributions in this area: Firstly, we describe the detailed design of our iterative annotation process and codebook. Secondly, we present a comprehensive taxonomy of labels for annotating misogyny in natural written language, and finally, we introduce a high-quality dataset of annotated posts sampled from social media posts.

## introduction abusive language is a phenomenon with serious consequences for its victims, and misogyny is no exception. according to a 2017 report from amnesty international, 23% of women from eight different countries have experienced online abuse or harassment at least once, and 41% of these said that on at least one occasion, these online experiences made them feel that their physical safety was threatened @xcite . automatic detection of abusive language can help identify and report harmful accounts and acts, and allows counter narratives @xcite @xcite . due to the volume of online text and the mental impact on humans who are employed to moderate online abusive language -moderators of abusive online content have been shown to develop serious ptsd and depressive symptoms (casey newton, 2020)it is urgent to develop systems to automate the detection and moderation of online abusive language. automatic detection, however, presents significant challenges @xcite . abusive language is linguistically diverse (vidgen and derczynski, 2020), both explicitly, in the form of swear words or profanities; implicitly, in the form of sarcasm or humor @xcite ; and subtly, in the form of attitudes and opinions. recognizing distinctions between variants of misogyny is challenging for humans, let alone computers. systems for automatic detection are usually created using labeled training data @xcite , hence, their performance depends on the quality and representativity of the available datasets and their labels. we currently lack transparent methods for how to create diverse datasets. when abusive language is annotated, classes are often created based on each unique dataset (a purely inductive approach), rather than taking advantage of general, established terminology from, for instance, social science or psychology (a deductive approach, building on existing research). this makes classification scores difficult to compare and apply across diverse training datasets. this paper investigates the research question: how might we design a comprehensive annotation process which results in high quality data for automatically detecting misogyny? we make three novel contributions: 1. methodology: we describe our iterative approach to the annotation process in a transparent way which allows for a higher degree of comparability with similar research. 2. model: we present a taxonomy and annotation codebook grounded in previous research on automatic detection of misogyny as well as social science terminology. 3. dataset: we present a new, annotated corpus of danish social media posts, bajer, foot_0 annotated for misogyny, including analysis of class balance, word frequencies, inter-annotator agreement (iaa), annotation errors, and classification baseline. since research has indicated that misogyny presents differently across languages, and, likely, cultures @xcite , an additional contribution of this work is that it presents a dataset of misogyny in danish, a north germanic language, spoken by only six million people, and indeed the first work of its kind in any scandinavian/nordic culture to our knowledge. in denmark an increasing proportion of people refrain from online discourse due to the harsh tone, with 68% of social media users self-excluding in 2021 @xcite , making this study contextually relevant. further, the lack of language resources available for danish @xcite coupled with its lexical complexity @xcite make it an intricate research objective for natural language processing. ## background and related work abusive language is as ancient a phenomenon as written language itself. written profanities and insults about others are found as old as graffiti on ruins from the roman empire @xcite . automatic processing of abusive text is far more recent, early work including e.g. @xcite and @xcite . research in this field has produced both data, taxonomies, and methods for detecting and defining abuse, but there exists no objective framing for what constitutes abuse and what does not. in this work, we focus on a specific category of online abuse, namely misogyny. ## methodology and dataset creation the creation of quality datasets involves a chain of methodological decisions. in this section, we will present the rationale of creating our dataset under three headlines: dataset, annotation process, and mitigating biases. ## a taxonomy and codebook for labeling online misogyny good language taxonomies systematically bring together definitions and describe general principles of each definition. the purpose is categorizing reference lang. labels abusive language zampieri et al. (2019 on one hand, it should be an aim of a taxonomy that it is easily understandable and applicable for annotators from various background and with different expertise levels. on the other hand, a taxonomy is only useful if it is also correct and comprehensive, i.e. a good representation of the world. therefore, we have aimed to integrate definitions from several sources of previous research (deductive approach) as well as categories resulting from discussions of the concrete data (inductive approach). our taxonomy for misogyny is the product of (a) existing research in online abusive language and misogyny (specifically the work in the labeling scheme (figure 1 ) is the main structure for guidelines for the annotators, while a codebook ensured common understanding of the label descriptions. the codebook provided the annotators with definitions from the combined taxonomies. the descriptions were adjusted to distinguish edge-cases during the weekly discussion rounds. the taxonomy has four levels: (1) abusive (abusive/not abusive), (2) target (individual/group/others/untargeted), (3) group type (racism/misogyny/others), (4) misogyny type (harassment/discredit/stereotype & objectification/dominance/neosexism/benevolent). to demonstrate the relationship of misogyny to other in-stances of abusive language, our taxonomy embeds misogyny as a subcategory of abusive language. misogyny is distinguished from, for instance, personal attacks, which is closer to the abusive language of cyberbullying. for definitions and examples from the dataset to the categories, see appendix a.1. we build on the taxonomy suggested in @xcite , which has been applied to datasets in several languages as well as in semeval @xcite . while parikh et al. (2019) provide a rich collection of sexism categories, multiple, overlapping labels do not fulfill the purpose of being easily understandable and applicable for annotators. the taxonomies in @xcite and @xcite have proved their application to english, italian and spanish, and offer more general labels. some labels from previous work were removed from the labeling scheme during the weekly discussions among authors and annotators, (for instance derailing), because no instances of them were found in the data. ## conclusion and future work in this work, we have documented the construction of a dataset for training systems for automatic detection of online misogyny. we also present the resulting dataset of misogyny in danish social media, bajer, including class balance, word counts, and baseline as an indicator. this dataset is available for research purposes upon request. the objective of this research was to explore the design of an annotation process which would result in a high quality dataset, and which was transparent and useful for other researchers. our approach was to recruit and train a diverse group of annotators and build a taxonomy and codebook through collaborative and iterative annotatorinvolved discussions. the annotators reached good agreement, indicating that the taxonomy and codebook were understandable and useful. however, to rigorously evaluate the quality of the dataset and the performance of models that build on it, the models should be evaluated in practice with different text types and languages, as well as compared and combined with models trained on different datasets, i.e. @xcite . because online misogyny is a sensitive and precarious subject, we also propose that the performance of automatic detection models should be evaluated with use of qualitative methods @xcite , bringing humans into the loop. as we found through our continuous discussions, online abuse can present in surprising forms, for instance the denial that misogyny exists. the necessary integration of knowledge and concepts from relevant fields, e.g. social science, into nlp research is only really possible through thorough human participation and discussion.