The Ecological Fallacy in Annotation: Modeling Human Label Variation goes beyond Sociodemographics (2023)

Abstract: Many NLP tasks exhibit human label variation, where different annotators give different labels to the same texts. This variation is known to depend, at least in part, on the sociodemographics of annotators. Recent research aims to model individual annotator behaviour rather than predicting aggregated labels, and we would expect that sociodemographic information is useful for these models. On the other hand, the ecological fallacy states that aggregate group behaviour, such as the behaviour of the average female annotator, does not necessarily explain individual behaviour. To account for sociodemographics in models of individual annotator behaviour, we introduce group-specific layers to multi-annotator models. In a series of experiments for toxic content detection, we find that explicitly accounting for sociodemographic attributes in this way does not significantly improve model performance. This result shows that individual annotation behaviour depends on much more than just sociodemographics.

## introduction different annotators will not necessarily assign the same labels to the same texts, resulting in human label variation @xcite . previous work finds that this variation depends at least in part on the sociodemographics of annotators, such as their age and gender @xcite @xcite . these results are particularly pronounced for subjective tasks like toxic content detection @xcite @xcite . since human label variation is relevant to a wide range of nlp tasks, recent research has begun to model individual annotator behaviour, rather than predicting aggregated labels @xcite . in this setting, we would expect sociodemographic attributes to help explain annotator decisions. therefore, we investigate whether explicitly accounting for the sociodemographic attributes of annotators leads to better predictions of their annotation behaviour foot_0 . there is a risk of misreading these efforts as an example of the ecological fallacy: aggregate group behaviour does not necessarily explain individual behaviour @xcite . for example, while on average, white annotators may be more likely to label african-american vernacular english as toxic @xcite , that does not mean it is true for every white annotator individually. however, we aim at exactly this distinction to discuss the relevance of sociodemographic groups in models of individual annotator behaviour. likewise, we do not assume prior work to commit ecological fallacies, even if a less-nuanced read might suggest it. davani et al. (2022) introduce a simple multiannotator model, where each annotator is modelled with a separate classification head. we expand their model with group-specific layers, which are activated for each annotator based on their sociodemographic attributes. we compare the two model setups to a control setup where we randomise group assignments. all comparisons use annotator-level toxicity data from @xcite . we find that find that explicitly accounting for sociodemo-graphic attributes does not significantly improve model performance. this result suggests that human label variation happens at a more individual level than sociodemographics, and that annotator decisions are even more complex. contributions 1) we introduce group-specific layers to model groups of annotators with shared attributes in multi-annotator models. 2) we evaluate the effect of group-specific layers for toxic content detection, and show that explicitly accounting for sociodemographic attributes does not significantly improve performance, thus highlighting the risk of the ecological fallacy in annotator modelling. as a corollary, we show that multi-annotator models can be applied to many times more annotators than in prior work. ## related work sociodemographics in annotation behaviour a growing body of research studies how annotator sociodemographics relate to their annotation decisions, for tasks ranging from natural language inference @xcite to the detection of racist @xcite or generally toxic @xcite language. @xcite , for example, find that annotators from certain sociodemographic groups (e.g., lgbtq people) tend to find content attacking their own groups (e.g., homophobic content) to be more toxic. this motivates our research into explicitly accounting for sociodemographics to model annotation behaviour. however, the link between sociodemographics and behaviour is not uncontested. @xcite , for example, do not find significant differences in annotation behaviour between annotators of different genders for four different tasks. ## data we use a sample of the @xcite dataset for our experiments. the full dataset contains 107,620 english comments from twitter, reddit, and 4chan, annotated for toxicity by 17,280 annotators. the annotation process encouraged annotator subjectivity @xcite which is a desired feature for modelling annotator behaviour. for each annotator, there is extensive sociodemographic information, collected with a survey. annotations are given as ratings on a five-point scale which we convert to binary annotations by mapping ratings of 2 to 4 to toxic, and ratings 0 and 1 to non-toxic. we randomly sample comments from the dataset until we reach annotations from more than 5,000 annotators. we then add all other annotations by these annotators. this approach maximizes the number of examples while controlling the number of annotators in our sample. our final sample contains 111,780 annotations from 5,002 annotators on 22,360 comments with 20 to 120 annotations per annotator (mean 22.35). most comments have five annotations. 20 comments have four because we removed any underage annotators before sampling. in total 78,357 annotations (70.10%) are toxic, and 33,423 annotations (29.90%) are non-toxic. we focus on four sociodemographic attributes: gender, age, education, and sexual orientation. group sizes vary by attribute. for gender, 2,450 annotators (48.98%) identify as female, 2,116 (42.30%) as male, 23 (0.46%) as non-binary (rest in residual categories, full statistics in a.1). ## experiments we compare three models. the baseline model is the multi-annotator model by @xcite . we use their multi-task variant: for each annotator, there is a separate classification layer trained on annotations from that annotator. all annotator layers share a pre-trained language model used to encode the input. we use roberta @xcite for this, motivated by computational constraints. the other models in our experiments build on this baseline model. for the sociodemographic models, we add group-specific layers based on sociodemographic attributes of the annotators. a single attribute, e.g., age, implies several groups, e.g., ages 25-34, ages 35-44. we add the group-specific layers between the pre-trained model and the annotator layers. each group of annotators shares a separate group-specific layer. we implement group-specific layers as fully-connected, linear layers, each learning a feature transformation applied for one group of annotators. finally, for the random models, we shuffle the assignment of annotators to groups from the sociodemographic model, retaining the relative group sizes. in other words, the probability of each annotator staying in the same group or being reassigned to another group corresponds to the relative size of each group. this approach keeps the model architecture constant while removing the connection between actual sociodemographic attributes and group assignment. it allows us to distinguish the effects of additional parameters, which groupspecific layers add in comparison to the baseline, from the effects of sociodemographic information. ## results sociodemographics vs. baseline across attributes, the average scores of the sociodemographic model and the baseline are similar. the sociodemographic model often has a slightly higher average macro f1 than the baseline, but no statistically significant gains. where average performance is better by several points, as for homosexual annotators, this gain is offset by a large variance in performance (a consequence of small group sizes). sociodemographics vs. random we also do not find significant performance differences between sociodemographic group-layer models and the corresponding random group assignment models. for most groups, the randomized models achieve the highest average scores, but differences to the sociodemographic model are never statistically significant. ## discussion we do not find strong evidence that explicitly modelling sociodemographics helps to predict annotation behaviour with multi-annotator models. these results might seem counter-intuitive, given the evidence of systematic annotation differences between sociodemographic groups (see §2). this discrepancy, however, echoes the issue highlighted by ecological fallacies @xcite : not every annotator will be a perfect representative of their group, so we will not necessarily learn additional information based on their group identity. this seems especially true if we already have access to individual behaviour (i.e., individual annotations). in contrast to davani et al. (2022), we made sociodemographic information explicit in our experiments, as one of the factors influencing annotation behaviour. group-specific layers can be seen as an inductive bias putting emphasis on the sociodemographic relations between annotators. however, there are potentially many other factors influencing annotation behaviour (e.g., attitudes, moral values, cognitive biases, psychological traits). in light of our results, it seems plausible that multi-annotator models learn about these factors implicitly as part of predicting individual behaviour, so that making one factor explicit does not change prediction quality, at least in the case of sociodemographics. still, we also know that generally group attributes can help predict individual decisions, i.e., as base rates or priors. to avoid ecological fallacies in modelling annotation, we therefore need to better understand when and how modelling sociodemographic information is useful in predicting an individual annotator's decisions. for example, we have only evaluated group-specific layers for single attributes. in contrast, social scientists have long adopted the idea of intersectionality @xcite , which also informs research on fairness in machine learning @xcite . intersectionality means that the effect of interactions between sociodemographic attributes enables specific experiences that are not captured by the attributes in isolation. for example, identifying as a man means something different depending on the person's education. groups derived from single attributes might simply be too coarse to improve classifiers learnt from individual labels, as in multi-annotator models. the dataset we use @xcite has many characteristics which are ideal for our study (see §3). however, it uses a broad notion of toxicity, in contrast to other studies of toxic language @xcite , which match content and analysed groups. when modeling the groups frequently referenced in the datasets themselves, we would expect greater benefits from group-specific layers. similar to us, @xcite who do not find significant differences between annotators of different genders, do so in a more general setting. we can only partially compare to @xcite , despite using the same dataset. in addition to differences in approach (see §2), our and their work also differ in their research questions and thus experimental conditions. @xcite compare their full model (group and individual) against using group information alone. we compare our full model (group and individual) against using individual information alone. so it is unclear if their model would benefit from group information in comparison to individual-level information alone. while they find an improvement from group information it is only in comparison to a baseline predicting not individual but aggregated labels. additionally, the composition of test sets sampled from the full dataset differs between the studies: @xcite use a test set of 5,000 comments, while we use 22,360 comments in a four-fold cross-validation. we leave an explicit comparison to future work. group-specific layers ( §4) are a natural extension of annotator-specific classification layers in multi-annotator models. however, other architectures to predict annotator-level labels use different ways to represent sociodemographic information, e.g., via embeddings in a recommender system @xcite . future work could explore additional representations of annotator attributes (e.g., as part of the input, either textual or as separate features) and other approaches to modelling the relation of individual labeling decisions and attributes (e.g., probabilistic graphical models). ## conclusion we ask how relevant modelling explicit sociodemographic information is in learning from individual annotators. our experiments with group-specific layers for four sociodemographic attributes on social media data with toxicity annotations @xcite show no significant benefit of modelling sociodemographic groups in multi-annotator models. however, as the issue of ecological fallacies highlights, it is not implausible that these models do not learn additional information from group information beyond the inherent variation. however, our results do not refute the usefulness of sociodemographic attributes in modelling annotation, but underscore the importance of their judicious use. different tasks and model architectures will likely benefit to different extents. ultimately, annotation behaviour is driven by complex factors and we will need to consider more than annotators' sociodemographics.