Subjective Crowd Disagreements for Subjective Data: Uncovering Meaningful C rowd O pinion with Population-level Learning (2023)

Abstract: Human-annotated data plays a critical role in the fairness of AI systems, including those that deal with life-altering decisions or moderating human-created web/social media content. Conventionally, annotator disagreements are resolved before any learning takes place. However, researchers are increasingly identifying annotator disagreement as pervasive and meaningful. They also question the performance of a system when annotators disagree. Particularly when minority views are disregarded, especially among groups that may already be underrepresented in the annotator population. In this paper, we introduce CrowdOpinion, an unsupervised learning based approach that uses language features and label distributions to pool similar items into larger samples of label distributions. We experiment with four generative and one density-based clustering method, applied to five linear combinations of label distributions and features. We use five publicly available benchmark datasets (with varying levels of annotator disagreements) from social media (Twitter, Gab, and Reddit). We also experiment in the wild using a dataset from Facebook, where annotations come from the platform itself by users reacting to posts. We evaluate CrowdOpinion as a label distribution prediction task using KL-divergence and a single-label problem using accuracy measures.

## introduction long term exposure to offensive, threatening, and hate speech posts through any public-facing social media platform can lead to depression or even physical injuries, specially at a younger age @xcite . this is a persistent problem in social and web content where the impact could be not limited to just the targeted parties but expand to anyone in the community consuming the content @xcite , from human annotation for twitter posts on whether they are intended to be offensive. these examples show how offense cannot generalize, and in cases when a majority of the annotators are not offended the input for a classifier is the majority voice. @xcite @xcite . language used by content creators in social media (see figure 1 ) with a subtle tone and syntax can hide the offensive content from the purview @xcite or machine learning classifiers @xcite . this challenge has ethical and legal implications in many countries as these governments have imposed restrictions for platforms to identify and remove such harming content @xcite citing the right for safety. the ml classifiers generally rely on human feedback @xcite . because humans, as content creators or annotators (content moderators), are subjective in their opinions @xcite . their feedback is essential to understanding subjective web or social media content. the standard practice is to ask multiple annotators about each post and then use the majority opinion or ml-based methods to determine the ground truth label (see figure 2 ). typically, minority views are completely removed from the dataset before it is published. yet these views are often meaningful and important 4 ) the prediction of a machine learning model trained on either the empirical or estimated distributions. @xcite @xcite @xcite . figure 1 shows three tweets with offensive language that have been labeled by multiple annotators about the tweeter's intent @xcite . in each case, the majority of annotators considers the offensiveness to be not intended. yet a minority considers it to be intended. a classifier trained on such language data after these minority opinions are removed would not know about them. this is dangerous because abusers often obscure offensive language to sound unintended in case they are confronted @xcite . and so, removing minority opinions could have dramatic impacts on the model's performance if, say, it was trying to detect users creating hateful or offensive content on a social platform. consequently, a growing body of research advocates that published datasets include all annotations obtained for each item @xcite @xcite @xcite . and a substantial body of research is studying annotator disagreement @xcite @xcite @xcite @xcite . unfortunately, most existing datasets are based on 3-10 annotators per label, far too few, statistically speaking, to represent a population. thus, learning over such a sparse space is challenging. @xcite show that clustering in the space of label distributions can ameliorate the sparseness problem, indicating that data items with similar label distributions likely have similar interpretations. thus, a model can pool labels into a single collection that is large enough to represent the underlying annotator population. recent work by @xcite , studying annotator disagreement with majority vote and multi-label learning methods, has called out the need for cluster-based modeling to understand annotator disagreements. the lack of annotator-level labels also hinders studying the annotator behaviors using methods that utilize those granular-level labels @xcite @xcite @xcite . we see this as a benefit to crowdopinion (co) we propose, a technique applicable at a broader level for understanding and predicting annotator disagreements which mitigate granular-level annotations. the motivation behind crowdopinion is to reduce inequity and bias in human-supervised machine learning by preserving the full distribution of crowd responses (and their opinions) through the entire learning pipeline. we focus our methods on web and social media content due to its subjectivity. our contributions to this core problem in ai and nlp is a learning framework foot_0 that uses unsupervised learning in stage 1 on both the labels and data features to better estimate soft label distributions. and in stage 2, we use these labels from stage 1 to train and evaluate with a supervised learning model. we consider the following three questions. q1: does mixing language features and labels lead to better ground truth estimates than those that use labels only? this focuses on the first stage as a standalone problem and is difficult to answer directly, as "ground truth" from our perspective is the distribution of labels from a hidden population of would-be annotators, of which we often only have a small sample (3-10 annotators) per data item. we study four generative and one distance-based clustering methods, trained jointly on features and label distributions, where we vary the amount of weight given to features versus labels. q2: does mixing features and labels in the first stage lead to better label distribution learning in the second? we use the label distributions obtained from the first-stage models from q1 as feedback for supervised learning. we compare our results with baselines from pooling based on labels only @xcite , predictions trained on the majority label for each item without clustering, and predictions trained on the label distribution for each item but without any other first-stage modeling. our results show improvement over unaggregated baselines. q3: do our methods lead to better single-label learning (sl)? since most applications consider only single-label prediction, we measure the model performance on single-label prediction via accuracy. ## methods -crowdopinion in conventional, nondistributional supervised learning, clustering might happen over the feature space only as a form of data regularization @xcite ; the labels, being strictly categorical and nondistributional, would be scalar and thus too simple to benefit from extensive modeling. in our setting, each data item x @xmath0, representing the empirical distribution of all annotator responses, which we view as sample of a larger, hidden population. our approach, crowdopinion (co) is two-staged and summarized in algorithm 1. in stage 1, we cluster together related data items and share among them a label distribution ŷi based on all labels from all items in each cluster. this stage resembles, in function, a deep vein of label estimation research begun by dawid and skene @xcite @xcite @xcite @xcite , except that (a) our output is an estimate of the distribution of label responses by the underlying population of annotators, not a single label, and (b) y i in their models is a vector with one dimension for each annotator. to better handle the label sparseness common in most datasets, our y i has one dimension for each label choice, representing the proportion of annotators who made that choice. stage 2 performs supervised learning on these new item, label distribution pairs (x i , ŷi ). note that nearly any pair of clustering c and supervised learning h algorithms can be used algorithm 1: co-c-h-w 1 parameters: 2 clustering (or pooling) algorithm c 3 hypothesis space h 4 mixing parameter @xmath1, 1] 5 inputs: 6 data features with empirical label distributions (x i , y i ) 1≤@xmath2: 8 stage 1: 9 perform clustering with c on both item features and labels, weighted and concatenated together: (w • x i , (1w) • y i ) 1≤@xmath3, ŷi ) be the centroid of the cluster π j associated with each (x i , y i ) 11 stage 2: perform supervised learning on (x i , ŷi ) over hypothesis space h for stages one and two, respectively. liu et al. (2019) performed the same kind of label regularization only using the label space y, it is a baseline for our methods (@xmath4. our main technical innovation is to perform label regularization based on the weighted joint feature and label space w • x × (1w) • y, where @xmath5, 1] is the mixing parameter that determines the relative importance of x versus y during clustering. we consider four clustering models c used by @xcite : a (finite) multinomial mixture model (fmm) with a dirichlet prior over π ∼ dir(p, γ = 75), where p is the number of clusters and each cluster distribution π j is a multinomial distribution with dirichlet priors dir(d, γ = 0.1), where d is the size of the label space, using the bnpy library (hughes and sudderth, 2013), a gaussian mixture model (gmm) and a k-means model (km) from scikit-learn, and the gensim implementation of latent dirichlet allocation (lda) @xcite . each of these models takes as a hyperparameter the number of clusters p. we perform parameter search (4 ≤ @xmath6, choosing arg min p i kl((x i , y i ) w , (x i , ŷi ) w ), i.e., the p that minimizes the total kl divergence between the raw and clustered label distribution, where, e.g., (x i , y i ) w denotes (w • x i , (1w) • y i ), i.e., the weighted concatenation of x i and y i . we also consider a soft, distance-based cluster-ing method, called neighborhood-based pooling (nbp) in the context of pll @xcite . for each data item i it averages over all data items j within a fixed kullback-liebler (kl) ball of radius r: here, the hyperparameter is the diameter r of the balls, rather than the number of clusters, and there is one ball for each data item. we perform hyperparameter search (0 ≤ @xmath7. the supervised model (cnn) for h is a 1d convolutional neural network @xcite , with three convolution/max pool layers (of dimension 128) followed by a dropout (0.5) and softmax layer implemented with tensorflow. the input to the model is a 384-dimension-vector text embedding, described below. we compare our methods against four baselines. pd is our cnn model but with no clustering; it is trained directly on the raw empirical label distributions (y i ). sl the same model, but trained on one-hot encodings of most frequent label in each y i . ds+cnn uses the dawid and skene (1979) model for c and @xmath8. co-c-cnn-0 is from @xcite , which clusters on labels only. we represent language features for both our unsupervised learning and classification experiments using a state-of-the-art pre-trained paraphrase-minilm-l6-v2 transformer model using sbert (sentence-transformers) library @xcite . we identified this pre-trained model based on sts benchmark scores at the time of writing. the feature vector size for each post is 384. ## discussions and ethical considerations our results for qs 2-3 show that cluster-based aggregation universally improves the performance of distributional learning. this seems to confirm that clustering is a powerful tool for combating label sparseness to predict population-level annotator responses. however, results were mixed for singlelabel learning. also, among the clustering methods in both distributional and single-label learning, there was relatively little variance in performance as w varies. the latter is certainly a negative result with respect to the technical ai question of whether or not to use both data features and label distributions in cases when we do cluster. but it is positive in that, combined with the overall superior performance of clustering for population-level learning, it shows that either label features or label distributions are adequate for realizing the benefits of clustering as a means of label distribution regularization. it also suggests that annotator disagreements are, in fact, meaningful and essential. to gain a better sense of how these methods can be used to address annotator inequality, we extract examples from d jq3 ( here, we see that the predicted distributions seem to differ from the empirical distributions and each other in meaningful ways. this is because in te n d e d n o t in td . p ro b . in td . n o t p ro b . in td . 0.0 0.2 0.4 0.6 label distributions gt (@xmath9. cons. hispanic 33 pi woman mod. cons. white 35 ni woman liberal white 18 (a) d si e4 "bpd is a genetic condition caused by having two x chromosomes. when a man is diagnosed with bpd it's just a professional way of saying he's acting like ae c***". source: r/incels in te n d e d n o t in td . p ro b . in td . n o t p ro b . in td . 0.0 0.2 0.4 label distributions gt (@xmath10: r/darkjokes in te n d e d n o t in td . p ro b . in td . n o t p ro b . in td . 0.00 0.25 0.50 0.75 1.00 label distributions gt (@xmath11. [repeat] the original boys in the hood". source: r/meanjokes in te n d e d n o t in td . p ro b . in td . n o t p ro b . in td . 0.0 0.2 0.4 0.6 label distributions gt (@xmath12." source: gab our models rely on other items with similar label distributions or language to normalize reactions. for instance, in example d fb e4, we see that the heavy annotator response to sad (795 responses) is retained when @xmath13.910), when only labels determine the clusters, but it decreases dramatically (to 0.165 and 0.126) as w increases. these examples show that when we introduce text into the clustering phase, the overall performance may not change, but qualitative differences may be quite significant at the item level. the examples in figure 4 were surfaced by randomly sampling reddit d si for posts whose predictions, using our models, differed from the human annotation. these examples all elicit ways of interpreting social media posts that contrast model predictions, human annotator choices, and our observations about offensiveness and toxicity. example d si e4, (figure 4a ) is an offensive joke that mocks women and people with a mental health disorder called borderline personality disorder ("bpd"). in contrast, the human annotation was split between not intended to be offensive and probably intended to be offensive. no human chose intended to be offensive, yet our algorithm predicted it might be, reflecting the deniability that comes from phrasing offensive speech as a "joke." example d si e5, (figure 4c ) is a joke about rape and older women. it is offensive because it associates rape with sex as opposed to rape with violence and sex with procreation. this is a challenging case for a typical ml classifier-there is no majority, and the label polarities are also opposite. in this case, our prediction correctly identifies the majority label. this may be due to our models grouping similar data items of similar content, supporting items such as this when there is contrasting confidence in human annotators. example d si e6 (figure 4b ) is offensive because it makes light of the hate group kkk wearing hoods by identifying them with an nwa song and film about african american teenagers ("boyz n the hood"). the pll prediction also indicates that this post may have been intended to be offensive. but the human annotator thought it was probably not intended to be offensive. this is another case where our prediction aligns with our judgment. example d si e7, (figure 4d ) is offensive because it alludes to a woman being dead and thus not having agency; it seems threatening. two human annotators chose this to be probably intended to be offensive, and one annotator considered it not intended to be offensive. the prediction finds this intended to be offensive. a commonality among these examples is that they all contain an element of deniability-the poster can always claim they were only joking. one challenge with content moderation is where to draw the line. when does the potential harm of letting an offensive post through outweigh the winnowing of free discourse? the answer often depends on context. the population-level learning approach we advocate here can help provide a more nuanced view into annotator response. it may also provide context on opinions to inform decisions about what should and should not be censored. our work also supports the findings from @xcite , where they studied the underlying reasons why annotators disagree on subjective content, such as offensive language annotation. the examples show how the proposed models can identify offensive content even with unreliable training data (human annotations). ## conclusion human annotation is often an expensive-to-acquire, challenging, and subjective resource for supervised machine learning. the obstacles to using human decisions in ml classification tasks are even more apparent when the problem domain is social media content. the nuance, disagreement, and diver-sity of opinions by humans augment and enrich the complex decisions machine learning attempts to surface. to gain as much utility as possible from this valuable resource, we propose and subsequently crowdopinion to retain these human judgments in the data prediction pipeline for as long as possible. first, this work introduces a novel method for mixing language features and label features into label distribution estimators to improve populationlevel learning. then, we evaluated our approach against different baselines and experimented with datasets containing varying amounts of annotator disagreements. our results suggest that (i) clustering is an effective measure for countering the problem of label sparseness when learning a populationlevel distribution of annotator responses, (ii) data features or label distributions are equally helpful as spaces in which to perform such clustering, and thus (iii) label distributions are meaningful signals that reflect the content of their associated items.