N o H ate B razil: A B razilian P ortuguese Text Offensiveness Analysis System (2023)

Abstract: Hate speech is a surely relevant problem in Brazil. Nevertheless, its regulation is not effective due to the difficulty to identify, quantify and classify offensive comments. Here, we introduce a novel system for offensive comment analysis in Brazilian Portuguese. The system titled “NoHateBrazil” recognizes explicit and implicit offensiveness in context at a fine-grained level. Specifically, we propose a framework for data collection, human annotation and machine learning models that were used to build the system. In addition, we assess the potential of our system to reflect stereotypical beliefs against marginalized groups by contrasting them with counter-stereotypes. As a result, a friendly web application was implemented, which besides presenting relevant performance, showed promising results towards mitigation of the risk of reinforcing social stereotypes. Lastly, new measures were proposed to improve the explainability of offensiveness classification and reliability of the model’s predictions.

## introduction the scenario of hateful comments in brazil is severe and entails the creation of safety and fairness technologies. during the elections in 2018 and 2022, the denunciations against xenophobia content had an increase of 2,369.5%; apology and public incitement to violence and crimes against life, 630.52%, and misogyny and race-ethical, increased by 1,639% and 595% 2 , respectively. hate speech is a particular form of offensive language that considers stereotypes to express an ideology of hate @xcite @xcite . while systems that classify hateful content are undoubtedly relevant, these technologies are being developed with scarce consideration of their potential biases (nadeem et al., 2021; sap et al., 2019; chang et al., 1 demo: http://143.107.183.175:14581/ 2 https://new.safernet.org.br/ 2019; @xcite . these systems may discriminate against the groups they are designed to protect @xcite , reflecting social stereotypes and being able to perpetuate social inequalities when propagated at scale @xcite . to the best of our knowledge, no systems have attempted to analyze text offensiveness in brazilian portuguese. therefore, the main contribution of this paper foot_0 is providing the first web system titled nohatebrazil for brazilian portuguese offensive comments classification. the nohatebrazil system receives two different inputs. the first input consists of a single comment written directly into the initial screen. the second input consists of a file in csv format containing a set of comments. in the following outputs, three pieces of information are exhibited: (i) offensiveness categories; (ii) offensiveness overall score; and (iii) prediction reliability score, which we describe in section 2.1. towards providing a reliable text offensiveness system, we focus on three strong strategies: (i) we provide a contextualized analysis of offensiveness, in which machine learning (ml) models recognize explicit and implicit offensive terms from a specialized lexicon annotated with context information; (ii) we propose and evaluate a framework for offensive comment detection; (iii) we evaluate the potential of our system to reflect social stereotypes through a distinctive analysis of tuples containing stereotypes versus counter-stereotype @xcite . for this purpose, we used a dataset of 300 tuples containing social stereotypes versus counterstereotypes in brazilian portuguese, which consists of a culturally-oriented translation from the crows-pairs @xcite , a benchmark fairness dataset. finally, our system presents 88.8% of f1-score and a low potential of reflecting social stereotypes against marginalized groups (12%). ## offensiveness detection framework in this paper, we propose a new framework that encompasses data collection, human annotation, and the implementation of ml models for offensive comment detection. we used this framework to build the proposed nohatebrazil web system, as shown in figure 1 . b+m: this model uses a generated bagof-words (bow) from the dataset vocabulary. this model embodies labeled context information (context-dependent and contextindependent) from a specialized lexicon of explicit and implicit offensive terms and expressions called mol (see section 3.1). we carried out the match with terms from mol. then, we assigned a weight for each term or expression labeled with context-dependent (weaker weight), and context-independent (stronger weight). according to the b+m model, the value of a term x in the document y is defined as ## final remarks this paper introduces the first system for text offensiveness analysis in brazilian portuguese. the nohatebrazil web system recognizes explicit and implicit offensiveness in context at a fine-grained level. we proposed a friendly design and robust architecture, resulting in a high system performance, besides promising results towards mitigation of the risk of perpetuating social stereotypes against marginalized groups. we also provided a robust framework for offensive comment classification, which encompasses data collection, human annotation, and ml models. finally, two new measures were proposed to improve the explainability of offensiveness classification at a fine-grained level and the reliability of the model's predictions.