Robustness and Confounders in the Demographic Alignment of LLM s with Human Perceptions of Offensiveness (2025)

Abstract: Despite a growing literature finding that large language models (LLMs) exhibit demographic biases, reports with whom they align best are hard to generalize or even contradictory. In this work, we examine the alignment of LLMs with human annotations in five offensive language datasets, comprising approximately 220K annotations. While demographic traits, particularly race, influence alignment, these effects vary across datasets and are often entangled with other factors. Confounders introduced in the annotation process—such as document difficulty, annotator sensitivity, and within-group agreement—account for more variation in alignment patterns than demographic traits. Alignment increases with annotator sensitivity and group agreement, and decreases with document difficulty. Our results underscore the importance of multi-dataset analyses and confounder-aware methodologies in developing robust measures of demographic bias.

## introduction a growing body of literature explores llms as a quick, inexpensive, and reliable alternative to human annotators @xcite @xcite . the eventuality of annotating data with llms is more than speculation: crowdworkers, who are often key to data annotation, already rely on llms for efficiency @xcite . since data annotations are a primary concern for any machine learning application, it is essential to assess the quality of llm-generated annotations. in particular, tasks that reflect annotators' subjectivity, such as the perceived offensiveness of a message @xcite , raise the question of which subjectivities are reflected in llm-generated annotations. recent research finds evidence of demographic bias, that is, systematic alignment between llms' annotations and those from select demographic groups of human annotators. if llms replicate the views of one demographic group over others, downstream applications risk perpetuating structural harms like marginalizing minority views. although llms' demographic bias has been identified for a variety of subjective constructs, including resumé screening @xcite , healthcare @xcite , political opinions @xcite , and offensiveness @xcite , we know little about whether such bias is consistent, since many of these studies focus on different nlp tasks and datasets. even within the same dataset and task, the identified biases are unclear or even seemingly contradictory. for example, for offensiveness annotations in the popquorn dataset, @xcite find that llms align most with white and female annotators, while @xcite do not find such alignment. using a different dataset, @xcite find alignment is highest with asian americans. understanding which demographic biases are consistent in llm annotations is fundamental to tackling them. our work fills this gap with a systematic study of llm alignment on offensiveness labeling with different genders and ethnicities across five datasets. we further investigate factors beyond annotator demographics that might drive human-llm misalignment. first, to exclude that llm bias may simply be attributed to low performance, we verify rq1: to what extent llms can substitute human annotators in detecting offensive language. indeed, llms are strong performers: correlations with aggregate human labels are positive and significant-ranging from 0.2 to 0.8. through permutation and bootstrapping tests, we show that llms match human performance in all datasets and surpass it in three. next, we test rq2: which demographic biases are consistently reproduced in llm-generated annotations across datasets. demographic biases exist within each dataset; however, most of these biases lack consistency. we find that only the difference in bias between the white and black demographics remained consistent across all five datasets. other biases appeared inconsistently or even showed opposite results across datasets. to unpack such differences between datasets, we therefore explore rq3: to what extent confounding factors explain demographic bias. we consider three hypotheses about the dataset annotation process: hpa) documents that are difficult to annotate may be assigned unevenly across demographics; hpb) annotators may show strong individual rather than demographic preferences in their annotation; and hpc) the disagreement between annotators of a same demographic group may affect the measures of alignment. we find that confounders explain a large fraction of the variance of llm-human alignment, and thus can help us unpack cases when demographic bias is inconsistent. overall contributions and novelty. prior studies on human-llm alignment rely on single datasets, leading to non-generalizable findings. we conduct the first large-scale, multi-dataset analysis across five nlp datasets and uncover inconsistent alignments, despite prior claims @xcite . however, we show that demographic factors alone do not explain misalignment. through a robust regression framework, we identify key confounders that drive alignment patterns, a crucial aspect overlooked in past research. finally, we provide actionable recommendations for bias estimation and contribute a harmonized dataset of 220k offensive language annotations, enabling more rigorous bias assessment in future studies. 1 2 related work our work assesses one aspect of alignmentdemographics-within the context of llm labeling. therefore, it lies at the intersection of using llms for annotation, annotation subjectivity, and demographic bias in llms. ## data to fully understand how llm annotations align with human opinions, we leverage five datasets encoding annotators' perceptions of offensiveness-a construct that has been proven to vary according to annotators' sociodemographic characteristics. specifically, we use the following datasets: annotator with attitudes (awa) dataset @xcite , uc berkeley's measuring hate speech corpus (mhsc) @xcite , nlpositionality (nlpos) dataset @xcite , popquorn dataset (popq) @xcite , and social bias inference corpus (sbic) @xcite ). 2 for awa, mhsc, and sbic, participants were recruited from amazon mechanical turk, while for popq, they were recruited through prolific. only sap et al. (sbic) mention that they restricted the annotator pool to people from the us and canada, while @xcite explicitly recruited a diverse pool of annotators from over 80 2 three of the five datasets (awa, popq, and sbic) focus on offensiveness, while the remaining two datasets (nlpos and mhsc) focus on hate speech. past research has looked into the association between offensiveness language and hate speech, concluding that both constructs, while not identical, are often similarly perceived @xcite @xcite . offensive language can be considered a superset of hate speech, where the latter is offensive language targeting protected groups or minorities. we use the annotator backgrounds in these datasets to consider the potential alignment between llms' responses and those of annotators in specific demographic groups ( ## methods next, we outline the models used for annotation, explain our approach to prompting, and describe how we operationalize our research questions. ## results in this section, we address three questions about the capabilities of llms in simulating human judgment when annotating offensive language. first (rq1), we examine the extent to which llms can accurately replicate human annotations overall. second (rq2), we investigate the alignment between these models and the annotations of sociodemographic subgroups of annotators and whether these alignments are consistent across datasets. third (rq3), we model the alignment behavior by considering potential confounding factors. ## discussion and conclusions our findings corroborate some of the previous results on demographic biases in llms' offensiveness ratings. in line with @xcite , llms consistently align better with white than black annotators. this bias replicates across datasets and is measurable even when accounting for several confounders. however, our systematic analysis offer a nuanced picture compared to past single-dataset studies. apparent demographic biases such as those based on annotators' gender @xcite or the asian american demographic @xcite , are statistically significant but contradictory in different datasets. we show that llm-human alignment may appear associated with annotators' demographics, when in fact it may be due to factors that are distributed inconsistently across demographics and datasets. implications for demographic alignment in data annotation. llms showed consistently worse alignment with black people's annotations of offensiveness. therefore, we echo the warnings about the potential negative consequences of using llm-generated annotations without understanding whose points of view they reinforce or neglect @xcite . however, given inconsistent alignment with any other demographic factor, we caution against narratives that anthropomorphize llms and essentialize annotators. recommendations for measuring misalignment. our results show that measuring demographic biases in a single dataset can produce unreliable results. systematic benchmarking across multiple datasets, coupled with replication and meta-analytical studies, is essential to support general claims about demographic bias. moreover, including confounding variables in analyses is crucial to distinguish between demographic bias and other sources of llm-human alignment. the heterogeneity of results also underscores the need for datasets with greater demographic representation, more redundant labeling, and increased data diversity @xcite . while acknowledging the practical and financial constraints of building such datasets, our research provides insights into strategic approaches for enhancing data quality. specifically, we emphasize the importance of accounting for confounders, such as document difficulty and the sensitivity of individual annotators, in their interplay with annotator demographics. since, at present, such confounders only emerge at the end of the annotation process, we see an opportunity for developing annotation solutions that dynamically adjust annotator recruitment and document assignment in response to emerging patterns. finally, while sociodemographic prompting has had some success in simulating human samples (argyle et al., 2023), it might not have the same success in simulating human annotators. but, this is not necessarily an indication of unrepresentativity of llms (though it is possible llms are, in fact, unrepresentative). in line with past research on demographic variation in annotation @xcite , it is crucial to explore whether different demographics correlate with confounding factors, as humans do not conduct content analysis solely driven by their demographic identity. this leads to unexplained variation and misalignment with llm labels when sociodemographic variables are the only ones considered.