Aggregation Artifacts in Subjective Tasks Collapse Large Language Models’ Posteriors (2025)

Abstract: In-context Learning (ICL) has become the primary method for performing natural language tasks with Large Language Models (LLMs). The knowledge acquired during pre-training is crucial for this few-shot capability, providing the model with task priors. However, recent studies have shown that ICL predominantly relies on retrieving task priors rather than “learning” to perform tasks. This limitation is particularly evident in complex subjective domains such as emotion and morality, where priors significantly influence posterior predictions. In this work, we examine whether this is the result of the aggregation used in corresponding datasets, where trying to combine low-agreement, disparate annotations might lead to annotation artifacts that create detrimental noise in the prompt. Moreover, we evaluate the posterior bias towards certain annotators by grounding our study in appropriate, quantitative measures of LLM priors. Our results indicate that aggregation is a confounding factor in the modeling of subjective tasks, and advocate focusing on modeling individuals instead. However, aggregation does not explain the entire gap between ICL and the state of the art, meaning other factors in such tasks also account for the observed phenomena. Finally, by rigorously studying annotator-level labels, we find that it is possible for minority annotators to both better align with LLMs and have their perspectives further amplified.

## introduction large language models (llms) @xcite @xcite @xcite have become to dominate language processing as generalists that can perform many tasks. this dominance comes from the emergence of methods such as in-context learning (icl; @xcite and chain-of-throught prompting (cot; @xcite , wherein llms perform tasks by leveraging input-output demonstrations and task instructions in the prompt only, without any parameter updates. while icl is often contrasted with traditional in-weights learning (i.e., gradient-based updates of the models' parameters) @xcite , the icl abilities of llms depend on their general, in-weights prior knowledge, allowing them to perform many tasks in a zero-shot or few-shot manner. therefore, studying how llms leverage the context in relation with their existing knowledge is a prerequisite to understanding icl. prior work found evidence that llms may be overly reliant on their prior knowledge, disregarding the demonstrations in the prompt. specifically, @xcite demonstrated that, under certain circumstances, llms ignore the provided signal in their prompt in the form of the mapping between inputs and outputs, and instead act as a database of tasks; they focus on the examples and the labels independently to fetch the underlying task @xcite : @xcite sampled examples and labels independently, and showed very little change in performance. since no annotations are provided, the setting is given the status of "zero-shot" inference, namely task-recognition zero-shot. while follow-up work @xcite @xcite has further studied this phenomenon and questioned its generality, more recent work @xcite has provided further, quantitative evidence for the relative importance of prior and evidence in the posteriors of the models. specifically, in complex subjective tasks like multilabel emotion or morality recognition, llms seem to virtually disregard evidence from the dataset's mapping in their posterior predictions, even in the form of chainof-thought prompting (cot; @xcite , performing significantly worse than traditional algo-rithms @xcite . this may imply an inability to perform annotator modeling, as the same document can receive different but valid annotations by different people. here, we use complex subjective to denote survey settings @xcite , in which people can reasonably disagree about their semantic interpretations, where the notion of ground truth is replaced with crowd truth @xcite . in this work, we question whether we can efficiently use this crowd truth when modeling these tasks with llms. we hypothesize that the aggregation process creates artifacts in the labels provided to the model, and in turn in the prompt, causing the model to ignore the entire input-label mapping as noise. specifically, aggregation can create inconsistent annotations, since, for instance, different annotators can prevail in different examples, which can cause inconsistencies within or across splits (for a toy example, consult section c). we create a carefully crafted experimental setting to test our hypothesis with annotator-level labels using icl and cot, and try to gauge at the magnitude of this effect. we find that llms do indeed tend to consistently favor individual perspectives compared to the aggregate, and in fact favor minority annotators more than majority annotators, who better resemble the aggregate. our contributions are: we advocate for more transparency in data collection and data sharing, and urge releasing and modeling individual annotations and not simply the aggregates in benchmark datasets. ## methodology we closely follow the methodology and notation of @xcite . for a set of examples x , and a set of labels y, a dataset d a defines a mapping f a : @xmath0, where a denotes a specific annotator or the aggregate, as well as reasoning chains r a (x) = r, @xmath1, and therefore d @xmath2, y, r) : @xmath3, @xmath4, @xmath5, from which we can sample demonstrations with p(x, y, r). we do not differentiate between splits for brevity. given cot prompt @xmath6, y i , r i ) : (x i , y i , r i ) ∼ p s , @xmath7, an llm produces its own mapping and predictions for the task, denoted as fk (.; p s ) : @xmath8. when using regular icl, we simply drop the reasoning text. for all our experiments, we set the temperature τ = 0 to derive deterministic predictions, and vary the examples and/or labels between runs. ## conclusion in summary, we see that aggregated labels tend to align less with the llms' prior for each task. furthermore, and in spite of worse aligned annotators receiving larger posterior performance increases, the aggregate posterior appears to collapse to the prior, resulting in significantly worse performance to several annotators. this result indicates that the majority is not necessarily well-aligned with models. then, it is evident that interpretable and consistent mappings can be modeled by llms and improve upon the prior, even though the model might not align with the specific annotator a priori. finally, we see that individuals do indeed result in better performance on the task. given the commonsense reasoning capabilities of llms, the emotional and moral capabilities of llms demonstrated in settings different from traditional machine learning settings (tak and gratch, 2024), as well as best-case results, we conclude that the aggregation process introduces artifacts in the labels that cause llms to ignore the mapping as noise. it is interesting to note that in our prompts, the aggregate in rarely a hodgepodge of disparate opinions, wherein the aggregate does not match any individual, but factors like different annotator mixtures, especially between train and test splits, as well as different annotator groups prevailing in different examples, introduce sufficient noise in the annotations that cannot be modeled with icl and cot. that being said, the performance gap with gradient-based methods remains large, suggesting that other factors like task complexity also majorly account for the observed biasing effects. finally, we question what it means to model these aggregate opinions. namely, since they might not reflect the opinion of any individual, even if in a minority of cases, then what rationale should be provided and how should it be generated in a sound manner? we advocate, therefore, as previous work has done @xcite , for releasing and modeling annotator-level labels instead of aggregates, and suggest that the field of subjective modeling should move away from aggregate modeling in the age of llms and of more elaborate modeling methods such as cot.