The Vulnerable Identities Recognition Corpus (VIRC) for Hate Speech Analysis
Authors: Ibai Guillén-Pacho, Arianna Longo, Marco Antonio Stranisci, Viviana Patti, Carlos Badenes-Olmedo (Universidad Politécnica de Madrid, Università di Torino, Aequa-tech, 2024)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
This paper presents the Vulnerable Identities Recognition Corpus (VIRC), a novel resource designed to enhance hate speech analysis in Italian and Spanish news headlines. VIRC comprises 880 headlines, manually annotated for vulnerable identities, dangerous discourse, derogatory expressions, and entities. Our experiments reveal that recent large language models (LLMs) struggle with the fine-grained identification of these elements, underscoring the complexity of detecting hate speech. VIRC stands out as the first resource of its kind in these languages, offering a richer annotation scheme compared to existing corpora. The insights derived from VIRC can inform the development of sophisticated detection tools and the creation of policies and regulations to combat hate speech on social media, promoting a safer online environment. Future work will focus on expanding the corpus and refining annotation guidelines to further enhance its comprehensiveness and reliability.

Introduction
Hate Speech (HS) detection is a task with a high social impact. Developing technologies that are able to recognize these forms of discrimination is not only crucial to enforce existing laws but it also supports important tasks like the moderation of social media contents. However, recognizing HS is challenging. Verbal discrimination takes different forms and involves a number of correlated phenomena that make it difficult to reduce HS to a binary classification. Analyzing the recent history of corpora annotated for HS, it is possible to observe the shift from very broad categorizations of hateful content to increasingly detailed annotation schemes aimed at understanding the complexity of this phenomenon. High-level schemes including dimensions like "hateful/offensiveness" or "sexism/racism" paved the way for more sophisticated attempts to formalize such concepts in different directions: exploring the interaction between HS and vulnerable targets; studying the impact of subjectivity; identifying the triggers of HS in texts. Despite this trend, the complex semantics of HS in texts is far from being fully explored. Information Extraction (IE) approaches to HS annotation have been rarely implemented yet. Therefore, corpora that include fine-grained structured semantic representation of HS incidents are not available. The only notable exception is recent work that treats the identification of HS targets as a span-based task. In order to fill this gap, we present the Vulnerable Identities Recognition Corpus (VIRC): a dataset of 880 Italian and Spanish headlines against migrants aimed at providing an event-centric representation of HS against vulnerable groups. The annotation scheme is built on four elements: Named Entity Recognition (NER): All the named entities involved in an HS expression: "location," "organization," and "person." Vulnerable Identity mentions: Generic mentions related to identities targeted by HS as defined by international regulatory frameworks: "women," "LGBTQI," "ethnic minority," and "migrant." Derogatory mentions: All mentions that negatively portray people belonging to vulnerable groups. Dangerous speech: The part of the message that is perceived as hateful against named entities or vulnerable identities. In this paper, we present a preliminary annotation experiment intended to validate the scheme and to assess the impact on disagreement in such a fine-grained task. The paper is structured as follows. In the related work section, we discuss related work; in the methodology section, we describe the methodology used; in the VIRC corpus section, we introduce the VIRC corpus; and in the conclusion, we present conclusions and discuss possible future work.

Conclusion
The Vulnerable Identities Recognition Corpus (VIRC), created in this work, reveals the challenge of identifying vulnerable identities due to the rapid evolution of language on social media. Our experiments indicate that large language models (LLMs) struggle significantly with this task. VIRC provides a detailed and structured resource that enhances understanding of the extensive use of hate speech in Italian and Spanish news headlines. The corpus is particularly valuable as it includes more annotation dimensions compared to related studies in other languages, such as vulnerable identities, dangerous discourse, derogatory expressions, and entities. This differentiation between vulnerable identities and entities, as well as between dangerous and derogatory elements, enables the development of sophisticated detection tools that can facilitate large-scale actions to mitigate the impact of hate speech (e.g., moderation of messages and generation of counter-narratives that reduce the damage to the mental health of victims). Future work will focus on expanding this resource by doubling the size of annotations for both languages and including non-racism-related phrases to ensure the resource is comprehensive. Additionally, we plan to refine the annotation guidelines to avoid low agreement on the derogatory label, enhancing the overall reliability and utility of the corpus. These efforts will further improve the effectiveness of hate speech detection and contribute to the development of policies and tools for a safer online environment.