Targeted Identity Group Prediction in Hate Speech Corpora (2022)

Abstract: The past decade has seen an abundance of work seeking to detect, characterize, and measure online hate speech. A related, but less studied problem, is the detection of identity groups targeted by that hate speech. Predictive accuracy on this task can supplement additional analyses beyond hate speech detection, motivating its study. Using the Measuring Hate Speech corpus, which provided annotations for targeted identity groups, we created neural network models to perform multi-label binary prediction of identity groups targeted by a comment. Specifically, we studied 8 broad identity groups and 12 identity sub-groups within race and gender identity. We found that these networks exhibited good predictive performance, achieving ROC AUCs of greater than 0.9 and PR AUCs of greater than 0.7 on several identity groups. We validated their performance on HateCheck and Gab Hate Corpora, finding that predictive performance generalized in most settings. We additionally examined the performance of the model on comments targeting multiple identity groups. Our results demonstrate the feasibility of simultaneously identifying targeted groups in social media comments.

## introduction the proliferation of hate speech on online platforms continues to be a significant human rights issue, associated with a host a negative consequences @xcite . hate speech distinguishes itself from other types of toxic or offensive content in that it specifically targets an individual or group on the basis of their membership in an identity group, such as race, religion, gender, sexual orientation, etc. @xcite . thus, developing methods that can identify and characterize hate speech, and its targets, is of paramount importance. given the scale of online hate speech, much effort has been made toward the development of automated approaches to classify or measure it given raw text @xcite . while initial efforts used binary labels, subsequent work has introduced additional labels that more finely characterize or measure hate speech @xcite @xcite . these include studies that implicitly specify the targeted identity group, such as labeling speech as racism or sexism @xcite . predicting the identity group targeted by social media content is useful beyond hate speech detection. such algorithms could identify comments that target groups of interest for secondary analyses. these analyses include evaluating the impacts, such as adverse health outcomes, of social media targeting specific communities @xcite . furthermore, leveraging knowledge of the target identity can better inform interventions or moderation of hateful content @xcite . thus, automated approaches to targeted identity prediction could serve these analyses by streamlining the process of labeling new corpora for study. while some efforts have been made to develop algorithms that predict targeted identity groups, they have largely focused on classifying individual vs. group targets @xcite or implicitly characterizing the target @xcite . predictive models capable of identifying a broad range of targeted protected classes have been less studied @xcite . hate speech corpora that include the requisite range of targeted identity annotations have been limited until recently, opening the door to a full examination of this problem @xcite @xcite . in this work, we developed models to predict identity groups targeted by social media comments. using the measuring hate speech (mhs) corpus @xcite , we trained neural networks to predict 8 identity group and 12 sub-group targets of hate speech. we demonstrated that these models exhibited good predictive performance, validating them within the mhs corpus and on external datasets. lastly, we examined model performance on comments with multiple targets, finding that performance depended highly on those targets. ## methods all code used in this work is available on the hate_measure repository foot_0 , which contains a codebase of various models applicable to the mhs dataset, and the hate_target repository foot_1 , which contains the code used for the analyses and figures described in this paper. all datasets were obtained as described by their corresponding entries on the hate speech data website @xcite . ## results our main goal was the multi-label binary prediction of target identity groups. we first trained and evaluated models to predict the targeting of the broad identity groups. we repeated these experiments, but on identity sub-group predictions. we then evaluated the performance of the model on two additional datasets: the hatecheck and gab hate corpora. lastly, we evaluated the performance of the model on samples which had multiple targets. ## discussion we have demonstrated that transformer-based neural network models can achieve good predictive performance on classifying multiple targeted identity groups or sub-groups simultaneously. we additionally validated the models on out-of-corpus data, finding that the results indicated some degree of generalizability. these results largely serve to benchmark this task for future studies, but also raise additional questions on the definition and conceptual framing of "targeting" in hate speech corpora. we evaluated the performance of the model on multiple targets. however, the survey question prompting for identity targets did not distinguish between a single target with multiple identities, or multiple distinct targets. for example, a secondary analysis may be interested in comments that target black women (at the intersection of racial and gender identity sub-groups), which are distinct from comments that separately target a black person and a woman, but would be indistinguishable under the labeling scheme. the distinction is important, as the former setting corresponds to intersectional identity @xcite , on which datasets and machine learning algorithms have been demonstrated to exhibit biased coverage or performance @xcite . thus, the development of new labeling instruments that ask annotators to make the distinction between intersectional and multiple targets is of interest for future work. for example, @xcite developed a hierarchical labeling scheme which allowed for the the identification of intersectional targets in a portugese dataset. in this work, we considered multi-label networks designed to simultaneously predict either identity groups or sub-groups. however, constructing networks that can simultaneously predict multiple sets of sub-groups is of interest, particularly for identifying intersectional targets in social media content. this can be viewed as multi-task problem, which may require adjustment to network architectures in order to achieve desirable performance @xcite . the development of multi-task networks with identity group specific sub-networks is of interest for future work @xcite . such networks could, for example, contain sub-networks predicting racial identity sub-groups, gender identity sub-groups, and others, in parallel. we relied on synthesizing annotator responses into a single label for each comment, while incorpo-rating some knowledge of their disagreement. this approach generally falls in line with the weak perspectivist approach in predictive computing @xcite . however, annotator disagreement on the identity group targets (appendix c) indicates that there is some subjectivity in identifying targeted groups. data perspectivist approaches more strongly incorporating different annotator responses are a viable path forward @xcite @xcite . at the same time, continued improvement in labeling instruments could further ameliorate these issues. for example, instruments that allow annotators to explain their reasoning in a structured fashion could shed light on why annotator disagreement is present. qualitative examination of comments could support additional theorization of the the concept of "targeting". in this vein, following @xcite , it may be possible to develop a measurement scale for "targeting" to facilitate item response theory approaches on this task. extensions to this work could facilitate parsing of the sentence to better elucidate the manner in which hateful comments refer to targets. for example, @xcite develop extraction networks to identify the text corresponding to both the "target" of a comment and its "aspect", or the characteristic attributed to the target. such work could facilitate additional qualitative examination of comments. while hate speech is understood to "target" a person or group based on a characteristic, the notion of "targeting" is slightly different across datasets. for example, we used "target" to mean the identity group that a comment is directed toward, whether the comment exhibited positive or negative valence. this was framed in the context of a measurement scale spanning supportive and hateful speech @xcite . however, other corpora limit their definition to content that is strictly hateful. these subtle distinctions limit the ability of out-of-corpus validation on datasets. for example, in this context, we could only use a subset of the ghc for generalization, since many comments were deemed not hateful (and thus did not have targeted identity annotations), despite referencing an identity group. datasets may also reference the manner in which "targeting" occurs, such as calls to violence, usage of profanity, or implicit rhetoric (e.g., sarcasm or irony). further work is needed to standardize these definitions to better inform the curation of future corpora.