NLP ositionality: Characterizing Design Biases of Datasets and Models (2023)

Abstract: Design biases in NLP systems, such as performance differences for different populations, often stem from their creator’s positionality, i.e., views and lived experiences shaped by identity and background. Despite the prevalence and risks of design biases, they are hard to quantify because researcher, system, and dataset positionality is often unobserved. We introduce NLPositionality, a framework for characterizing design biases and quantifying the positionality of NLP datasets and models. Our framework continuously collects annotations from a diverse pool of volunteer participants on LabintheWild, and statistically quantifies alignment with dataset labels and model predictions. We apply NLPositionality to existing datasets and models for two tasks—social acceptability and hate speech detection. To date, we have collected 16,299 annotations in over a year for 600 instances from 1,096 annotators across 87 countries. We find that datasets and models align predominantly with Western, White, college-educated, and younger populations. Additionally, certain groups, such as non-binary people and non-native English speakers, are further marginalized by datasets and models as they rank least in alignment across all tasks. Finally, we draw from prior literature to discuss how researchers can examine their own positionality and that of their datasets and models, opening the door for more inclusive NLP systems.

## dataset & model positionality: definitions and background a person's positionality is the perspectives they hold as a result of their demographics, identity, and life experiences (holmes, 2020; savin-baden and howell-major, 2013). for researchers, positionality "reflects the position that [they have] chosen to adopt within a given research study" @xcite . it influences the research process and its outcomes and results @xcite . some aspects of positionality, such as gender, race, skin color, and nationality, are culturally ascribed and part of one's identity. others, such as political views and life history, are more subjective @xcite . ## case studies we present case studies of applying nlpositionality to two different socially-aligned tasks: social acceptability and hate speech detection. ## conclusion we introduce nlpositionality, a framework to quantify design biases and positionality of datasets and models. in this work, we present how researcher positionality leads to design biases and subsequently gives positionality to datasets and models, potentially resulting in these artifacts not working equally for all populations. our framework involves recruiting a demographically diverse pool of crowdworkers from around the world on labinthewild, who then re-annotate a sample of a dataset for an nlp task. we apply nlpositionality to two tasks, social acceptability and hate speech detection, to show that models and datasets have a positionality and design biases by aligning better with western, white, college-educated, and younger populations. our results indicate the need for more inclusive models and datasets, paving the way for nlp research that benefits all people.