Language is Scary when Over-Analyzed: Unpacking Implied Misogynistic Reasoning with Argumentation Theory-Driven Prompts (2024)

Abstract: We propose misogyny detection as an Argumentative Reasoning task and we investigate the capacity of large language models (LLMs) to understand the implicit reasoning used to convey misogyny in both Italian and English. The central aim is to generate the missing reasoning link between a message and the implied meanings encoding the misogyny. Our study uses argumentation theory as a foundation to form a collection of prompts in both zero-shot and few-shot settings. These prompts integrate different techniques, including chain-of-thought reasoning and augmented knowledge. Our findings show that LLMs fall short on reasoning capabilities about misogynistic comments and that they mostly rely on their implicit knowledge derived from internalized common stereotypes about women to generate implied assumptions, rather than on inductive reasoning.

## introduction according to the 7 th monitoring round of the eu code of conduct on countering illegal hate speech online, 1 social media are slowing down the removal of hateful content within 24 hours, dropping to 64% from 81% back in 2021. the prevalence of hate speech phenomena has become a factor of polarization and pollution of the online sphere, creating hostile environments that perpetuate stereotypes and social injustice. previous work on hate speech detection from the nlp community has contributed to definitions @xcite @xcite , datasets @xcite @xcite , and systems @xcite . however, most of these contributions have focused (more or less consciously) on explicit forms of hate. a. muti conducted this work during her visiting period at the university of groningen, under università di bologna's marco polo programme for internships abroad. ## related work hate speech detection is a widely studied research area, covering different targets and linguistic aspects. we discuss literature on implicit misogyny detection with particular attention to contributions in reconstructing implicit content. implicit hate speech detection hate speech detection is a popular research domain, with more than 60 datasets covering distinct targets (e.g., women, lgbtiq+ people, migrants) and forms of hate (e.g., sexism, racism, misogyny, homophobia) in 25 languages, according to the hate speech dataset catalogue. 5 in its early stages, but still predominant nowadays, research in this domain focused on the creation of datasets for detecting explicit cues of hate speech, such as messages containing slurs or swear words (jahan and oussalah, 2023). however, hate speech is often implicit, characterized by the presence of coded language phenomena such as sarcasm, irony, metaphors, euphemisms, circumlocutions, and obfuscated terms, among others @xcite @xcite . for this reason, implicit hate speech detection has progressively gained momentum in recent years, and several efforts have been put into the development 4 both data and code are available at https://github.com/arimuti/ argument-reasoning-for-implicit-misogyny . ## misogyny detection as argumentative reasoning understanding the elusiveness of implicit hate speech is due to its ambiguity. implicit messages could be understood as critiques, opinions, or statements (see figure 1 ) rather than as hateful. hate, in this case, is expressed by assuming social biases, stereotypes, and prejudices against a specific target, women in the case of misogyny. the identification of these assumptions requires access to the reasoning process behind arguments and opinions. argumentative reasoning (ar) offers a solution. ar relies on the notion of an argumentative model or scheme, i.e. a formal representation of arguments into intrinsic components and their underlying relations. it aims at explicating an argument through the identification of its constituent components and relations (lawrence and reed, 2019). for instance, toulmin's ar model organizes arguments into fundamental elements, such as claim, warrant and reason. ar models have been successfully applied in many nlp tasks, from argument mining @xcite @xcite to warrant and enthymeme reconstruction @xcite @xcite @xcite , argumentative scheme inference @xcite , and fallacy recogni-figure 2 : example of a warrant (implicit logical connection) for an implicit misogynous message. example and warrant are extracted from sbic @xcite . grounded on previous work on ar in usergenerated content @xcite , we frame implicit misogyny detection as an ar task @xcite based on toulmin's theory @xcite , with the aim of developing more robust detection tools by explicitly describing the underlying reasoning process in these messages. in order to classify a message, the model is first asked to generate a textual response corresponding to toulmin's warrants. more formally, let c be the claim associated to a given message and @xmath0, . . . , w n } be a set of possible warrants, i.e. logical statement(s) that support c. the model must generate an associated w and, based upon it, classify the message as being misogynous or not. figure 2 represents the approach described above. in this particular case, the generalization that women do not understand sport because it is stereotypically for men is what distinguishes a personal attack from a case of misogyny. while there have been efforts on evaluating llms in argumentative tasks, such as quality assessment @xcite , component detection @xcite , and argumentative linking @xcite , the capability of llms for implicit argumentative reasoning has yet to be explored. to the best of our knowledge, our work is the first to assess llms on implicit misogyny through the lens of ar. ## data this section introduces the datasets used in our experiments. for italian, the newly created implicit-mis corpus ( § 4.1). for english, sbic+, an extended version of the social bias inference corpus @xcite enriched with misogynous texts from implicit hate corpus (elsh-erief et al., 2021) ( § 4.2). ## experimental setup our main goal is to evaluate the abilities of models to generate the implied assumptions for implicit misogynous messages. by doing so, we can also evaluate the implicit knowledge of llms, such as named entities or events mentioned in texts, which, if not known, it would be impossible to understand the misogynistic nature of such texts. each batch of experiments is framed as in section 3, i.e. the model generates the implicit claim c and related warrants w and, based on these, provides a classification label as misogynous or not (toulmin setting). we also experiment with a different strategy by prompting the model to directly reconstruct the implied assumptions, without claim, and afterward classify the text (assumption setting). we address these tasks both in a zero-shot and in a few-shot setting. while implied assumptions are generally broader than warrants, warrants specifically bridge the reasoning gap between claims and evidence. in our prompts, implied assumptions and warrants appear quite similar. nevertheless, the use of these terminologies may significantly impact the model's behavior due to its sensitivity to prompt phrasing, therefore we experiment with both. we experiment with two state-of-the-art llms: llama3-8b and mistral-7b-v02. 6 for both, we select their instruction-tuned version. during preliminary experiments with 50 instances, we also tested italian-specific llms, namely llamantino, 7 fauno, foot_4 and camoscio. 9 since all three models were unable to generate valid implied assumptions, they were discarded. we consider the following baselines: (i) fine-tuned encoder-based models, (ii) zero-shot classification with llms, and (iii) few-shot classification with llms without generating explanations. ## results we report two blocks of results: the first block focuses on classification of the messages. since both the italian and the english datasets contain only positive classes, we only report the recall. the classification task offers an indirect evaluation on the goodness of the ar methods. the second block targets the generation of the implied assumptions/warrants. considering the complexity and the pending issues related to the evaluation of automatically generated text @xcite , we report the results using established automatic metrics (bertscore, bleu, and rouge-l) as well as a manual validation on a subset of 300 messages (150 per language) ( § 6.2). the overall evaluation procedure we have devised allows us to assess both the performance of the models in detecting implicit misogyny and the alignment between llms and human annotators in generating reasoning-based explanations. all answers from llms have undergone postprocessing to evaluate them properly. two main post-processing heuristics concern the treatment of the "refusal to provide an answer" (including the refusal to generate the warrants) and the "need of more context". we considered both cases as if the messages were marked as not misogynous. while llama3-8b tends to return refusals to answers, mostly due to the safeguard layer, mistral-7b-v02 has a tendency towards indecisive answers requiring more context. llama3-8b always provides an answer when applied to the italian data. for completeness, appendix e includes the results considering these cases as correct. ## conclusions we proposed the task of implicit misogyny detection under an argumentative reasoning perspective, since to understand implicit statements, one needs to reconstruct the missing link (the warrant) between the claim and the assumption. our work highlights the complexity of such a task, which paves the way for hate speech detection as a proxy task to probe the reasoning abilities of llms. our prompt-based experiments show that llms fail 68% and 50% of the time in generating implied assumptions in italian and in english respec-tively. the poor relationship between wrongly generated explanations and correctly predicted classes shows llms' over-reliance on their implicit knowledge and spurious correlations rather than reasoning skills. our results are consistent with @xcite : prompting strategies that rely on implicit knowledge in llms often generate an incorrect classification when the generated knowledge (implied assumptions/warrants) is wrong due to lack of references, reasoning skills, or understanding of non-standard language. indeed, verifying the validity of the generated text before injecting it in the prompt in a human-in-the-loop approach would be a next step to undertake. to conclude, our findings show that (i) the performance of the classification task cannot be used as a proxy to guarantee the correctness of the implied assumption/warrant; (ii) llms do not have the necessary reasoning abilities in order to understand highly implicit misogynistic statements. therefore, models for hate-related natural language inference tasks should be improved. one possible approach would be to inject external knowledge in the misogynous texts, in order to fill the gaps related to their lack of implicit knowledge. for instance, had the model known who moana pozzi was, it would have probably inferred that when a person is compared to her, it is a derogatory way to address that woman.