Fine-grained Sexism Detection in Italian Newspapers
Authors: Federica Manzi, Leon Weber-Genzel, Barbara Plank (Ludwig-Maximilians-University Munich, IT University of Copenhagen, 2024)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
In recent years, tasks revolving around hate speech detection have experienced a growing interest in the field of Natural Language Processing. Two main trends stand out in the context of sexism recognition: the focus on overt forms of sexism such as misogyny on social media and tackling the problem as a text classification task. The main objective of this work is to introduce a new approach to tackle sexism recognition as a sequence labelling task, operating on the token level rather than the document level. To achieve this goal, we introduce (i) the FGSDI (Fine-Grained Sexism Detection in Italian) corpus, containing Italian newspaper articles annotated with fine-grained linguistic markers of sexism, and (ii) a two-step pipeline that sequentially performs sexism detection on the sentence level and sexism classification on the token one. Our primary findings include that (i) tackling the task of sexism recognition as a sequence labelling task is possible, however, a large amount of labelled data is needed; (ii) leveraging few-shot learning for sexism detection proves to be an effective solution in scenarios where only a limited amount of data is available; (iii) the proposed pipeline approach allows for better results compared to the baseline by doubling the overall precision and achieving a better F1-score.

Introduction
According to the Sapir-Whorf hypothesis AUTHOR AUTHOR, language shapes the way we think and interact with the world. It becomes therefore crucial to analyse our usage of linguistic expressions to reveal the intricate dynamics of societal norms, power structures, and cultural values embedded within our belief system. In this context, language can also become a vehicle for different forms of bias and discrimination, including sexism. Sexism in language encompasses a variety of phenomena, ranging from more subtle ones, nested within the grammar and semantics choices we make when talking about women, to more overt instances of misogyny, characterized by aggressiveness and violence against individuals based on their gender identity. \newline In recent years, sexism and misogyny detection and classification have witnessed a growing interest in Natural Language Processing (NLP), especially after the advent of transformers models AUTHOR, which unravelled new possibilities in nearly every NLP task. However, these efforts have mainly focused on misogyny and hate speech in general, tackled as text classification tasks on the document level, and specifically within the context of social media platforms such as Twitter and Facebook. \newline The main contributions of this paper are as follows. First, we concentrate on specific linguistic markers of sexism introducing more fine-grained classes than those usually considered in the sexism detection and classification tasks. %we propose the FGSDI (Fine-Grained Sexism Detection in Italian) corpus of Italian newspaper articles that we annotated following the guidelines in appendix sec:appendix_b. Inspired by linguistic work by Alma Sabatini AUTHOR, we propose a new annotation scheme and corpus for fine-grained sexism detection, resulting in the FGSDI (Fine-Grained Sexism Detection in Italian) corpus of Italian newspaper articles with the annotation guidelines released in appendix sec:appendix_b. Second, we address the recognition of linguistic markers of sexism as a token-level classification task, assigning a label to each token according to the fine-grained classes introduced before. This constitutes an innovation in that, to the best of our knowledge, no other work---in Italian or other languages---has tackled this task at such a granularity. \newline In particular, we compare two different approaches. The first one, which we used as baseline, consists of fine-tuning a RoBERTa AUTHOR model on the token classification task using whole texts as input. The second, novel one is a two-step pipeline approach inspired by AUTHOR which performs sexism detection and classification subsequently. The sexism detection task is tackled as binary classification applied at the sentence level. Sentences classified as potentially containing linguistic markers of sexism will then undergo the second step of the pipeline, which again involves classification on the token level.%This method constitutes a further novelty of the current work

Conclusion
This work aimed to bridge a gap in the research area of sexism detection and classification in Italian by the following contributions. First, we proposed the FGSDI (Fine-grained Sexism Detection in Italian) corpus for which we, importantly, provided new in-depth annotation guidelines. They are based on foundational linguistic work by AUTHOR and can be applied to other text genres in the future. Second, differently from previous research, we modelled the task of sexism classification as a sequence labelling instead of a text classification task. To achieve this goal, we compared two approaches, the baseline and the two-step pipeline, which allowed for a better overall performance on the task. \newline Working on enriching the corpus with new articles annotated with relevant labels would be the biggest contribution to bring this project forward. At the same time, having multiple annotators could enhance insights on the annotations and lower the risk of bias and subjectivity related to having a single annotator. Moreover, the modularity of the pipeline makes it open for further experimentation, especially in scenarios where more relevant data are available. One example could be using the multi-class classification setting of SetFit, which was excluded from the final pipeline since it performed slightly worse than the binary setting we ultimately used. Finally, further improvements can be made to the use of coreference resolution, which in many cases is not accurate in recognizing occurrences of the same referent in text. We would like to thank the reviewers for the feedback and encouraging words. This work is supported by the MaiNLP research lab at LMU Munich.