Reconsidering Annotator Disagreement about Racist Language: Noise or Signal? (2021)

Abstract: An abundance of methodological work aims to detect hateful and racist language in text. However, these tools are hampered by problems like low annotator agreement and remain largely disconnected from theoretical work on race and racism in the social sciences. Using annotations of 5188 tweets from 291 annotators, we investigate how annotator perceptions of racism in tweets vary by annotator racial identity and two text features of the tweets: relevant keywords and latent topics identified through structural topic modeling. We provide a descriptive summary of our data and estimate a series of generalized linear models to determine if annotator racial identity and our 12 latent topics, alone or in combination, explain the way racial sentiment was annotated, net of relevant annotator characteristics and tweet features. Our results show that White and non-White annotators exhibit significant differences in ratings when reading tweets with high prevalence of particular, racially-charged topics. We conclude by suggesting how future methodological work can draw on our results and further incorporate social science theory into analyses.

## introduction hateful and racist language is abundant on social media platforms like twitter and a growing body of work aims to develop tools to detect such language in these spaces. such a tool would offer opportunities to intervene, like providing automatic trigger warnings, and would provide a powerful barometer to measure racism. however, these efforts are hampered by low interrater agreement, low modeling performance, and a lack of consensus on what counts as racist language (e.g., @xcite @xcite . these efforts are also largely disconnected from rich understandings of race and racism in the social sciences (but, see @xcite . indeed, social scientists have long acknowledged the difficulties of measuring racism, even when using traditional social science methods (e.g., interviews and surveys), due to social desirability biases and the increasingly covert nature of racism @xcite . in this paper, we reconsider efforts to annotate for racism in light of sociological work on race and racism. instead of generalizing our detection of racism, we narrow our scope to focus on anti-black racism. in other words, we focus on racialized language directed at or centering on black americans. using human annotations foot_0 for the racial sentiment (positive or negative) of 5188 tweets, we describe how ratings vary by annotators' own racial identity. our findings suggest that white raters respond differently to particular types of racialized language on twitter, as identified by structural topic modeling (stm), than non-white raters. failing to account for this systematic difference could lead researchers to consider tweets which non-white annotators identify as including negative sentiment as innocuous because more numerous white annotators rate those same tweets as positive or neutral. we conclude by suggesting several ways in which future work can account for the variability in annotations that comes from annotators' own racial identities. ## hypotheses based on previous work in annotation and a sociologically informed understanding of race and racism, we propose three hypotheses: h 1 : annotations will vary, on average, based on the racial identity of the annotator: white raters will rate tweets as having a more positive racial sentiment on average compared to non-white raters. h 2 : annotations of racial sentiment will vary by the racially charged keywords in the tweet and the latent topics in the tweet. tweets with racialized keywords (e.g., n****r) will be rated as more negative than those without. h 3 : annotations will vary based on the interaction of text features (racially charged keywords and latent topics) and the racial identity of the annotator: compared to non-white raters, white raters will interpret particular topics as having a more positive racial sentiment, and interpret other topics as having a more negative racial sentiment. ## conclusions the goal of our analysis was to determine if and how annotator racial identity influenced perceptions of the racial sentiment of a tweet regarding black americans. when we examine the mean difference between white and non-white annotators (model 1), we find a small but significant difference in sentiment ratings. when we consider white and non-white raters responses to different amounts of topics in the tweets (model 3) we find strong evidence that annotator racial identity does inform perceptions of sentiment towards black people for seven of our twelve topics. our results suggest that white and non-white raters interpret these seven topics differently and as these topics increase in tweets (figures 2-4), this gap in interpretation widens. notably, the topics we identify as most divisive are some of the very topics which social scientists may be most interested in analyzing: references to police brutality, references to historical figures or events, and discussions of anti-racist politics. our descriptive qualitative analysis suggests that white annotators may not be as attuned to the nuances of these topics in tweets. future work might expand on these results to investigate raters' rationale for their ratings. given that perceptions of racism vary by annotator's own race, it is crucial that future work considers whose interpretations are reflected in annotations for racism. indeed, annotators' interpretations end up being a gold standard from which models learn to detect what counts as racist or not. while we focus on the role of annotators' racial identities, many other dimensions of annotators' identities likely also influence their responses on annotation tasks more generally. this issue extends beyond annotation tasks: across the disciplines, there is growing recognition that much of the social scientific knowledge produced to-date is specific to the population from which we most often draw participants @xcite . we suggest several takeaways for future research. first, researchers should use purposeful sampling @xcite to gather annotations from diverse populations of annotators. this may be challenging given that platforms for collecting annotations may include a particular demographic of workers. in particular, young, white, and well-educated workers are over-represented on mturk @xcite . second, research using human annotation might collect (and report) annotator demographics, in order to be explicit about whose interpretations the annotations do (or do not) reflect. third, given the many possible identities, researchers might consider several possible strategies to focus on a particular demographic of annotators. researchers might focus on the populations for whom the gold standard is most important, or might be most divisive. we suggest that the gold standards for racist language should reflect the interpretations of who is impacted most the standard. for example, annotations for anti-black racism should ideally reflect how black individuals interpret the data. perhaps annotations could be weighted when training classifiers to detect racist language, so that annotators whose identities are most affected by the gold standard have stronger influences on the gold standard. human annotation lies at the crux of many advances and tools in computer science. our work also fits into a broader, growing body of scholarship which reconsiders how researchers' choices and assumptions around human annotation shapes the tools and information that annotation is used to produce (e.g., @xcite @xcite ). annotations for racist and hate speech must be reflexively collected and used to avoid contributing to other forms of biases along the way.