Hate Speech Criteria: A Modular Approach to Task-Specific Hate Speech Definitions (2022)

Abstract: The subjectivity of automatic hate speech detection makes it a complex task, reflected in different and incomplete definitions in NLP. We present hate speech criteria, developed with insights from a law and social science expert, that help researchers create more explicit definitions and annotation guidelines on five aspects: (1) target groups and (2) dominance, (3) perpetrator characteristics, (4) explicit presence of negative interactions, and the (5) type of consequences/effects. Definitions can be structured so that they cover a more broad or more narrow phenomenon and conscious choices can be made on specifying criteria or leaving them open. We argue that the goal and exact task developers have in mind should determine how the scope of hate speech is defined. We provide an overview of the properties of datasets from hatespeechdata.com that may help select the most suitable dataset for a specific scenario.

## introduction the surge in online hate speech has resulted in an increased need for its automatic detection. its presence can be highly consequential as it creates an unsafe environment and threatens the freedom of speech @xcite . effects of hate speech range from a personal level (e.g. anxiety or stress @xcite ) to societal level (e.g. discrimination or violence @xcite ) and such speech can disrupt social debate severely @xcite . due to the large volumes of data on social media, automatizing the task is essential as hate speech can violate the law, depending on the country, in addition to its negative consequences in society. this makes automatic hate speech detection a very important task that needs to be carried out responsibly. what is considered hate speech is subjective @xcite , there are a variety of valid viewpoints on what does (not) fall under this concept. current hate speech datasets in nlp reflect this, having similar yet (subtly) different or incomplete definitions. for instance, similar terms are used interchangeably across publications and datasets, e.g. abusive, offensive, or toxic @xcite . we posit that a clear relation to (membership of) a target group of the victim sets hate speech apart from other forms of toxic or abusive language. underspecified definitions and guidelines increase the level of subjectivity in annotations. this subjectivity propagates into the model, which can lead to biased models @xcite . even if annotations are systematic, it may remain unclear which phenomena (e.g. target groups or types of abusive) are covered and thus captured by models. it depends on the task for which a dataset is created whether subjectivity is desired or not. we will argue that, even for scenarios where the goal is to collect multiple viewpoints, it is important to clearly define on what aspects of the phenomenon this subjectivity is sought. at the same time, we must keep in mind that it is impossible to fully remove subjectivity when determining whether something is hate speech or not. even in law, where hate speech is aimed to be defined as objectively as possible, it is inevitable that courts have difficulties interpreting such a context-dependent and sensitive topic in consistent and predictable ways @xcite . nevertheless, a clear definition can help reduce subjectivity to borderline cases. what would then be a good definition of hate speech? we advocate that a good definition of hate speech starts with a good understanding of the intention that it serves. for instance, a social media platform may want a broader consideration of hate speech, as it needs to keep its platform safe, in comparison to a law-enforcing model, which has to take legal action only when there is a clear punishable presence. one may decide to focus on hate speech towards one specific group or to keep this completely open to investigate which groups are considered potential targets by (crowd) annotators. rather than specifying what "the" definition should be, we provide a meta-prescriptive setup to construct definitions and guidelines through a modular approach, where modifications can be made according to the task at hand. concretely, we propose five criteria which should be taken into account when defining hate speech foot_0 and creating annotation guidelines: (1) target groups, (2) social status of target groups, (3) perpetrator, (4) type of negative reference, (5) type of potential effect/consequence. these criteria were developed with insights from law and social science. we provide an overview of all english datasets from www.hatespeechdata.com according to these criteria, so that people working with a specific definition in mind can easily identify which existing english datasets foot_1 may be of direct use or provide a good starting point. ## background and motivation in this section, we describe related work that provided the motivation and insights for the operationalization we propose in this paper. ## proposed hate speech criteria this section provides our proposed procedure to define hate speech. as outlined above, we follow the view that hate speech is characterized by problematic statements that are related to a target's (presumed) membership of a specific group. starting from this assumption, we propose the following criteria, represented in figure 1 , to define the scope of hate speech: ## overview of definitions and datasets to highlight the differences between existing datasets and to comprehend what kind of tasks they would fit, we present an overview of widelyused datasets based on their definitions. our scope is restricted to all english datasets found on hatespeechdata.com that tackle hate speech, since it has the most datasets and a variety of definitions. 10 the overview can be found in for each step in our criteria, we indicate if it is explicitly specified in the definition or not. there is a difference between defining hate speech and specifying a particular focus. we follow the descriptions of the annotations in the dataset for our classification. in cases where only a definition is given, we assume that that is the focus of the dataset as well, unless stated otherwise (e.g. in @xcite ; basile et al. (2019) ). an ✗ signals that the aspect is unspecified in the definition, therefore it is still possible that the specifics are present in the dataset but this cannot be guaranteed. e.g. t nd p explicit ref effects/consequences talat and hovy (2016) ✓ ✓ ✗ stereotype & slur insult, violence, hate, other elsherief et al. (2021) ✓ ✗ ✗ slur & group characteristics insult, violence, hate, discrimination, other kennedy et al. (2022) ✓ ✗ ✗ slur, group characteristics, & stereotypes violence or hate basile et al. (2019) ✓ ✓ ✗ ✗ other kirk et al. (2021) ✓ ✗ ✗ ✗ discrimination, other founta et al. (2018) ✓ ✗ ✗ ✗ insult, hate, other mandl et al. (2019) ✓ ✗ ✗ stereotypes & group characteristics ✗ mollas et al. (2020) ✓ ✗ ✗ ✗ insult, violence zufall et al. (2020) ✓ ✗ ✗ ✗ violence, hate elsherief et al. (2018) ✓ ✗ ✗ ✗ other gao and huang (2017) ✓ ✗ ✗ ✗ other qian et al. (2019) ✓ ✗ ✗ ✗ other ribeiro et al. (2018) ✓ ✗ ✗ ✗ violence, other röttger et al. (2021b) ✓ ✗ ✗ other ✗ chung et al. (2019) ✓ ✓ ✗ ✗ ✗ fanton et al. (2021) ✓ ✓ ✗ ✗ ✗ mathew et al. (2021) ✓ -✗ ✗ ✗ davidson et al. (2017) ✗ ✗ ✗ ✗ insult, violence, hate, other de gibert et al. (2018) ✗ ✗ ✗ other ✗ ousidhoum et al. (2019) ✓ ✗ ✗ ✗ ✗ for the second step, most definitions do not mention anything about dominance. @xcite are the only ones to mention caucasian as a target group, which we mark with a '-' to signal that this paper is explicit about not restricting itself to non-dominant groups. most papers with a ✓for nd specifically define their targets to apply to nondominant groups only @xcite @xcite , with the exception of @xcite , who mention minorities. none of the datasets mention taking perpetrator characteristics into account (column p). similarly, the explicit references are left unspecified in most datasets' definitions (column explicit ref). this means that for such datasets it cannot be guaranteed whether explicit references are present, nor whether they include specifications as to which ones. looking at effects/consequences, violence and hate occur the most. other terminology for negative relations and effects/consequences widely differs and the interpretation with respect to our criteria can be subjective (e.g. is 'humiliate' a form of discrimination, an insult or something else?), we mark such terminology as other. the idea behind the overview is that it can illustrate the need for future datasets that specify their aspects more explicitly and aids in deciding which dataset is suitable for a specific task. for instance, if a task requires a dataset that guarantees a focus on non-dominant groups, then column nd can easily point to the datasets that fit this prerequisite explicitly. if the dominance being specified is not very important but the presence of an explicit reference of a negative relation like a slur is, then @xcite could be fitting for the task. if, in addition, incitement of hate and violence is essential, then @xcite should be considered. if dominance is important as well, one might consider to further annotate samples from these sets that are labeled as hate speech, saving the time to separate 'clean' messages. in combination with the overview of which datasets cover which target groups, we believe these outlines to be helpful for identifying useful datasets. ## discussion the presented hate speech criteria aim to include those aspects of hate speech needed to arrive at clearer definitions and to provide better annotation guidelines, while supporting a wide range of use cases. we are aware, however, that they do not provide a magic solution to all challenges around this complex phenomenon. in this section we briefly discuss (1) possible extensions, (2) possible further specifications and (3) challenges that a clear definition cannot (fully) address on its own. we tried to create an extensive overview of relevant aspects, but are well aware that we may have missed things. moreover, hate speech is strongly connected to culture and what is perceived as hate speech may change. the criteria can thus be extended to cover new target groups, more perpetrator characteristics, or additional potential consequences. this particularly holds for the fourth step: types of references to target groups. we maintain that evidence of the abuse being related to (assumed) group membership is a requirement, but researchers may decide that other clues can also serve as possible evidence for an utterance being hate speech. these clues may include the history of a certain perpetrator, who in the past has uttered instances of hate speech multiple times. as mentioned, the evidence may also come from e.g. what the utterance is responding to. the typology of @xcite is not included in our criteria. we explained in section 2 how our criteria relate to this typology. it is however straightforward to add further specifications to a hate speech definition created through our criteria. note that, even though our criteria relies on the clear presence of group characteristics, stereotypes, and/or slurs, they do not exclude implicit forms of abuse, especially since we also evaluate the potential consequences e.g.: "everything was quite ominous with the train accident. would like to know whether the train drivers were called stereo-typicalname1, stereotypicalname2 or stereotypi-calname3 #refugeecrisis" @xcite . here, the stereotypical names indicate that this falls under hate speech. utterances like "white revolution is the only solution" @xcite may seem problematic due to the lack of an explicit slur, stereotype or target group characteristic. it nevertheless provides direct evidence, since "white" implies that the revolution would be against non-white and the violent nature of the threat. a clear definition can help avoid inconsistencies and unwanted forms of subjectivity in the data, but it cannot address all challenges involved in determining whether an instance exhibits hate speech. first, we mentioned multiple times that some form of subjectivity remains inevitable when dealing with hate speech. though people will always differ as to where they draw the line, instructions on the level of severity that should be included with illustrating discussions can be helpful. even for the seemingly clear case of inciting violence, which is a core aspect, there is a vast difference between uttering "throw tomatoes at them!" and an actual life-threatening "gun them down!". the class of group insult in particular can include a large variety, from merely unkind statements "women really have a horrible sense of fashion with their white sneakers!" to insults that question people's capabilities or attack someone's morals. questioning a groups capabilities can lead to discrimination, especially when uttered by people with authority or in power. the remark "i'm not sexist but female comedians just are not funny!" @xcite may seem relatively harmless when coming from a tweeter with few followers, but when coming from an influential critic or the president of a comedians' union, it can actively harm women's careers. attacks on a group's morals can also have an impact beyond merely insulting. for instance, saying that a non-dominant group are leeches can incite hate or lead to violent ramifications. explanations that illustrate the potential affect on different targets can help annotators to determine the severity of a specific statement and may help them to make more systematic decisions on where to draw the line. this leads to the second challenge that a definition by itself cannot solve: annotator bias. explanations and training may help annotators to tackle their bias and may make them more sensitive to more subtle attacks to groups they are not part of, but the affect of an annotator's background will not be completely eliminated. our criteria are meant to support creating a definition and guidelines. they do not mention gathering annotator information, because we believe that the definition crafted for a specific task does not change based on annotator information. however, we want to emphasize that it is essential that annotator demographics are taken into account and that the goal of the task should be kept in mind when establishing annotators' background. e.g. if the target group considered for the task is gender, it is of utmost importance to have annotators that can capture experiences of all genders. in general, it is vital to include members of potential target groups, since they are more likely to pick up on subtleties. a third issue that can only partially be addressed by means of a clear definition lies in the relation between hate speech and freedom of speech. as mentioned, hate speech can create unsafe environments that hamper freedom of speech. at the same time, opinions can differ regarding whether specific remarks are harmful or should be allowed because they are part of an important debate (and where marking them as hate speech would hamper freedom of speech). law has to clearly define what is punishable hate speech and what on the other hand should be protected by freedom of speech. in dutch law, a distinction is made for e.g. public debates where politicians are given more (but not unlimited) space for controversial statements. another example is that dutch law protects members of a religion from problematic utterances (which is prohibited in all eu countries), but leaves ample space for negative statements about specific religions (decriminalized in many western countries) @xcite . in the context of creating a safe environment for discussion, this distinction between attacking a religion or people can be hard to make sometimes and there are cases where it does not seem to make sense. the example #banislam from @xcite , for instance, might be aimed at religion and not at people, but it can clearly be harmful to muslims and it is hard to see how such a hashtag would contribute to a useful public debate or discussion. in this context, both the potential harm and added value of statements to a debate should be taken into account. though the example of #banislam seems clear, it is easy to imagine that it is not always straightforward to make this call. the challenges mentioned above show that a clear definition may be a good starting point, but cannot solve everything. we provided examples that illustrate the complexity, but full discussions would merit individual papers on each of these topics. a final limitation we want to point out is that our overviews are currently limited to english datasets. our framework leaves variables related to linguistic properties or cultural aspects open and can thus be easily applied to datasets covering other languages. ## conclusion we presented modular criteria to construct definitions and annotator guidelines to address hate speech. these steps include aspects that have, to our knowledge, not been prominent before. we propose five components for defining hate speech: we provided an overview of a large variety of english hate speech datasets based on the dimensions that are present in our criteria. we hope that the criteria and discussions in this paper will motivate nlp researchers working on hate speech to critically think about the tasks they are addressing and evaluate how fitting current definitions and datasets are for their task. the overview can then help select the most suitable datasets that can either be directly used or used as starting points that serve the task after adding further specifications. we particularly hope that the discussion in this work will help those working on new datasets to take these aspects into account from the start.