The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels (2024)

Abstract: Longstanding data labeling practices in machine learning involve collecting and aggregating labels from multiple annotators. But what should we do when annotators disagree? Though annotator disagreement has long been seen as a problem to minimize, new perspectivist approaches challenge this assumption by treating disagreement as a valuable source of information. In this position paper, we examine practices and assumptions surrounding the causes of disagreement–some challenged by perspectivist approaches, and some that remain to be addressed–as well as practical and normative challenges for work operating under these assumptions. We conclude with recommendations for the data labeling pipeline and avenues for future research engaging with subjectivity and disagreement.

## introduction when developing human-labeled data for machine learning (ml) tasks, labels for each example are often obtained by collecting annotations from multiple annotators, which are then aggregated to provide a single ground truth label per example. however, a line of recent work has illustrated that annotators disagree for many reasons, and that capturing this disagreement can improve model performance and calibration @xcite , surface minority voices @xcite , and uncover task ambiguities @xcite . researchers have begun to ask: what should we do when people disagree? how can (or should) our datasets and models account for different opinions? we argue that this new wave of research-which, following @xcite , we refer to as the perspectivist turn-constitutes a paradigm shift in data collection for ml and offers an opportunity to systematically examine the changing landscape. in this position paper, we examine practices and assumptions across papers regarding how data is collected from multiple annotators, discuss challenges raised by these approaches, and provide recommendations for rethinking data labeling when annotators disagree. we offer our own syntheses of observed practices and assumptions in natural language processing (nlp), as well as observations drawn from meta-analyses of ml research more broadly. we first examine what has changed under this paradigm shift: we examine each paradigm's assumptions about the causes and nature of disagreement, and the practical challenges that arise when operating under each set of assumptions. we then explore what has not changed, identifying normative challenges-questions and assumptions about labeling not yet taken up in this shifting landscape. finally, we offer recommendations for designing data labeling processes that better account for annotator disagreement, and avenues for future research. by charting these shifting assumptions and practices, we aim to surface the ways in which each paradigm succeeds, or fails, to account for the rich tapestry that disagreement can offer. ## the longstanding paradigm we characterize the longstanding paradigm of data labeling as work that collects labels on a data instance from annotators and aggregates them with the goal of capturing underlying ground truth labels @xcite ). 1 by contrast, work in the perspectivist paradigm treats variation among annotator labels as a source of meaningful information @xcite . we first examine assumptions about the causes of disagreement and challenges faced under the longstanding paradigm. ## the perspectivist turn perspectivist efforts argue that longstanding approaches are insufficient when (1) annotators frequently disagree in ways that are important to capture, and (2) even with diverse annotator recruitment, aggregate labels often fail to adequately represent the true population's opinions. approaches in the perspectivist turn include training with annotators' individual labels or pertinent details about the annotators and explicitly modeling individual annotators' behavior (e.g., @xcite @xcite ; training with probability distributions over labels @xcite ; calibrating to variance between annotators @xcite ; collecting labels from many annotators @xcite ; and investigating causes of disagreement (e.g., @xcite @xcite . 2 here, we explore how perspectivist approaches conceptualize the causes and nature of disagreement, as well as emerging practical and normative challenges. ## recommendations for practical challenges annotator disagreement carries implications for all stages of the data labeling process. we provide recommendations for each of these stages: before data labeling begins. if prescriptive decisions are made about acceptable bounds of disagreement when designing a data labeling process, these decisions should be made explicitly. in addition, consider potential axes of disagreement for the specific task at hand, such as linguistic or sociocultural differences, ambiguous labels, or differences of opinion. considering these normative questions-who or what is the data collection for-and potential sources of disagreement before beginning data labeling can help to design the process so that important differences in opinion are captured, and sources of confusion are minimized. recruitment. depending on the extent to which the collected data should reflect the opinions of all stakeholders or focus on experts, different best practices for annotator recruitment apply. if the objective is to reflect the views of a particular population, such as potential users, it is crucial to recruit a representative sample of that group. this may sometimes require additional recruitment efforts to account for different demographics' uneven participation in crowdsourcing. in addition, rather than filtering out "noisy" annotators based on whether they disagree with others, alternative filtering strategies such as checking intra-annotator agreement @xcite or doing multiple rounds of qualification tasks before the main task @xcite can help to reduce spam without discarding minority opinions. an intermediate approach might consider stratifying the recruited sample of annotators based on important axes of disagreement (e.g., different countries where a model will be used) to upsample groups that might otherwise be underrepresented. in addition, for tasks involving different types of expertise (e.g., system summarizing medical or legal documents, or a language model giving advice to specific communities), consider allocating annotators to items based on their expertise. other considerations apply regardless of the recruited population. recruiting a large annotator pool helps mitigate sample error, and capping annotations per annotator can prevent a dataset from primarily reflecting the views of a few annotators. when modeling disagreement, consider collecting annotator data specifically about factors likely to cause disagreement for the task at hand. ## recommendations for normative challenges in this section, we discuss potential avenues for research aiming to engage annotator disagreement. replace implicit normative decisions with explicit ones. majority-vote aggregation captures the average view of the aggregated population with every annotator weighted equally. by contrast, data labeling tasks where some people are clearly betterinformed (e.g., doctors in medical domains, or speakers of a language for translation) have an implicit "expert-driven" framing, in which only some views are solicited. this includes considering lived experience as a form of expertise, which can prove critical to successful annotation. we can imagine a spectrum of practices ranging from "democratic" to "expert-driven," with different points along this spectrum suited to different situations. for a task requiring medical knowledge, it would be unreasonable to use labelers with no medical training; when setting community norms, all community members' views are important. each data labeling task requires choosing a point on this spectrum. making this decision explicitly, rather than defaulting to majority vote, can help to create decision rules that are easier to define and contest. draw on parallel problems from other disciplines. the broader questions of how to capture a population's views, and make decisions based on them, has a long history across a range of traditions involving stakeholder participation, from social choice theory and mechanism design @xcite to valuesensitive and participatory design @xcite : science and technology studies. bowker and @xcite analysis of the political and social dimensions of classification highlights the importance of retrievability, the process of retaining the voices of people conducting classification for systems to maintain "maximum political flexibility." as perspectivist methods examine ways to retain the opinions of individual labelers, this line of work can help to understand how individual voices can be lost, merged, or preserved in systems that draw on them; and understand how we can, as bowker and star (2000) note, "reflect new institutional arrangements or personal trajectories." bowker and star, as well as @xcite and agre (2014), illustrate how technological artifacts embed and reproduce social and political values; drawing on douglas, @xcite and @xcite investigate institutional factors and perspectives of researchers or industry leaders who describe subjectivity as a problem to minimize. together, this literature contextualizes subjectivity and disagreement, and emphasizes the need for critical reflection on practices and assumptions in technological development. elsewhere, critiques of machine learning, including approaches to fairness and ethics, can offer opportunities for perspectivist efforts to reflect on assumptions surrounding representation and inclusion. for example, hoffmann (2020) complicates the notion of inclusion in dataset design by pointing out that such inclusion can forestall calls for more radical change, while @xcite similarly observe that more representative datasets for e.g., facial recognition do not address the more fundamental problem of surveillance. philosophy of mind. literature related to why, despite our understanding of the physical processes involved, we still lack a full understanding of where subjective feelings come from and why they differ, such as discussion of qualia @xcite @xcite , can provide a starting point for discussion of differences of opinion that are not easily situated in terms of the annotator's background. voting and social choice. many issues regarding optimal data labeling resemble issues regarding ideal voting mechanisms, with different constraints. in electoral settings, the full population's opinions may be solicited, and single decisions based on their choices have widespread effects (e.g., electing an official who makes decisions in many policy areas). during annotation, by contrast, it is often infeasible to solicit all stakeholder opinions, and different aspects of model outputs can be decided independently (e.g., decisions on coherence or offensiveness of model outputs). however, overarching themes of how to aggregate preferences while maximizing stakeholder satisfaction and welfare @xcite could provide useful lessons for perspectivist work. pragmatics. the community could take inspiration from the notion of a "common ground" in pragmatics, wherein conversational participants communicate based on a shared understanding of the world. this shared understanding is based on factors that include demographic attributes, but also factors such as the specific speech situation, the participants' professions, online communities, languages spoken, and imagined audiences @xcite . annotation of text functions like a communicative situation in which the annotator interprets language while making assumptions about the speaker, purpose, and audience of the text based on their own background, and a wide variety of factors in their background may be relevant based on the text. focusing on content moderation, thylstrup and talat (2020) draw on @xcite to describe how these assumptions become embedded in datasets: data annotators serve as intermediaries who read on behalf of the intended recipient and often interpret text differently from the specific intended reading that the sender meant to encode, with the result that systems based on those labels encode the intermediary position instead of that of the sender or intended recipient. understanding the range of factors that influence annotator interpretation could disentangle more latent factors behind disagreement in data collection. participatory design. elsewhere, participatory traditions that interrogate power dynamics in order for "non-expert stakeholders to provide direct input on technology design" @xcite can offer practitioners valuable insights for navigating disagreement and reflecting on assumptions about disagreement embedded in their practices @xcite . take advantage of nuanced output spaces to meet diverse stakeholder needs. the existence of disagreement does not rule out the possibility of building systems that produce single outputs affecting a whole population. @xcite note that "any society that protects freedom of thought and expression" experiences "continued disagreement about key normative questions," but we still "find fair terms of social cooperation without requiring everyone to agree." even when providing a single output is unavoidable, treating preferences non-unidimensionally can help to arrive at single outputs that better serve more people. systems that provide for a broad range of potential outputs, including generative models, can help to consider different, non-contradictory values that seem to result in disagreeing preferences. for example, if one annotator prefers a language model output that is non-discriminatory and another prefers one that is concise, they might disagree on their preferences between two outputs, but a non-discriminatory and concise output could satisfy both annotators. revealing that preferences are not unidimensional and exploring the resulting space of potential outputs opens ways to generate greater consensus. ## conclusion assuming that tasks have a ground truth, using majority-vote aggregation, and avoiding a normative stance have long been common practices in data labeling. however, a growing perspectivist literature is recognizing that datasets and models must be designed to account for the full spectrum of human perspectives. we argue that perspectivist approaches can accomplish their goals more fully by considering causes of disagreement beyond demographics, addressing tensions with data quality and research pressures, and reasoning explicitly about normative considerations.