Polarized Opinion Detection Improves the Detection of Toxic Language (2024)

Abstract: Distance from unimodality (DFU) has been found to correlate well with human judgment for the assessment of polarized opinions. However, its un-normalized nature makes it less intuitive and somewhat difficult to exploit in machine learning (e.g., as a supervised signal). In this work a normalized version of this measure, called nDFU, is proposed that leads to better assessment of the degree of polarization. Then, we propose a methodology for K-class text classification, based on nDFU, that exploits polarized texts in the dataset. Such polarized instances are assigned to a separate K+1 class, so that a K+1-class classifier is trained. An empirical analysis on three datasets for abusive language detection, shows that nDFU can be used to model polarized annotations and prevent them from harming the classification performance. Finally, we further exploit nDFU to specify conditions that could explain polarization given a dimension and present text examples that polarized the annotators when the dimension was gender and race. Our code is available at https://github.com/ipavlopoulos/ndfu.

## introduction annotations for subjective tasks are often aggregated, to form ground truth labels and allow supervised learning algorithms to be trained for these tasks. given a text, for example, annotations are averaged to yield binary labels reflecting whether the text is misogynous or not @xcite . these annotations, however, are not always described by a single mode. specific data items may lead to non-unimodal annotations, increasing the interannotator disagreement @xcite . this point is clearer if we consider a post classified as -1 by half of the annotators and as 1 by the other half, assuming a 3-point scale. no point is suitable to represent this item due to the two polarized ratings. current machine learning conventions reduce the annotations for a given text into a single label (most often, the mode) and consider the inter-annotator agreement @xcite as an indicator of the quality of the ground truth, or the task difficulty. in this work, we argue that polarized annotations may be beneficial in machine learning for subjective tasks and that inter-annotator agreement is not necessarily reflective of the ground-truth quality. the negative impact of the information loss due to such aggregations can be higher when the annotators come from different social groups. language that is offensive to specific groups at risk for discrimination will be obscured in datasets with aggregated annotations and consequently be ignored by algorithms trained on those datasets. in this work, we focus on polarized opinions of annotators about the label to be assigned, foot_0 suggesting their detection prior to supervised learning, to remove ambiguous annotations and improve the classification performance. recently, the distance from unimodality (dfu) measure has been found to to have a strong correlation with human judgment when used as an index of polarization @xcite . although effective, this measure is un-normalized, a fact that limits the measure's interpretability. to this end, we propose in this work a normalization which directly improves the measure. by employing the normalized dfu, then, we propose a classification methodology where we introduce a new class comprising data with polarized annotations. despite the fact that, in principle, a new class increases the task's difficulty, our approach outperforms the binary baseline in three datasets for abusive language classification. furthermore, the probability for the added class, assigned for a text, serves as an estimate of the polarization of the text annotations. the contribution of this work is threefold. for many nlp tasks, a diversity of valid beliefs exist about what the correct data labels should be @xcite . such tasks comprise the detection of toxic language @xcite , harassment (al kuwatly et al., 2020), and stance @xcite . due to the lack of measures assessing polarized opinions, however, no published work to date aimed at detecting and classifying polarized annotations, which is the goal of this study. instead, the focus is more broadly on ambiguous instances @xcite , with current approaches reducing the number of classification labels @xcite , or modelling the distribution of annotations using a gaussian distribution @xcite , or learning the histogram of annotations @xcite . reducing the number of class labels @xcite showed that the quality of the ground truth (e.g., the inter-annotator agreement) impacts the performance of machine learning models and should not be taken for granted. the authors studied different ways to yield a single target label from multi-rater settings, which is a common approach in supervised learning for nlp. the standard reduction method is majority voting from crowdsourced opinions or the fraction of raters who said yes (in a yes/no question), binarized. although common, this approach fails to encode uncertainty @xcite . uncertain ground truth uncertainty can be tackled by considering the annotations for a data item as noisy observations that can be modeled by a gaussian distribution @xcite . @xcite attempted to learn simultaneously the mean and the variance of the normal distribution showing that this approach outperforms ground-truthing methods that disregard uncertainty. although encoding uncertainty is useful, the use of a unimodal distribution (e.g., a gaussian) imposes severe limitations, since it disregards the possibility of polarized opinions (multiple modes). such an assumption may be harmful in tasks with subjective opinions, such as sentiment analysis and toxic language detection, where annotators with different personal, cultural, or demographic backgrounds may perceive differently commonsense knowledge (what they will assign as target label) of the same item @xcite . soft labels instead of a noisy unidimensional target label, one may attempt to learn a multivariate probability density function (i.e., the normalized histogram). such a ground truth model allows the maintenance of polarized annotated opinions in the supervised signal when using machine learning algorithms. @xcite , for example, showed that predicting the whole distribution of the class annotations improves robustness in image classification. another example is the work of @xcite , who encoded human disagreement to improve the quality of social computing datasets, building on prior findings showing that annotators' disagreement is not noise @xcite . these studies, however, treat polarized opinions as a special case of disagreement @xcite . ## datasets we investigated one resource comprising what human experts perceive as polarized and three datasets comprising annotations for toxic language, a subjective task with high social impact. regarding the latter, we limited our search to datasets whose annotations are released without any aggregation, i.e., one label per annotator is provided. ## experiments using opgt, we measured the correlation between our proposed ndfu and human judgment. then, we used ndfu to introduce the additional class of polarized opinions in three toxicity datasets, and we compare the performance in toxicity detection with and without the added polarized class. ## polarized class prediction analysis as shown with the reduced class hypothesis ( §6.3), unpolarized learning can lead to a performance improvement in binary toxicity classification. networks trained using the unpolarized learning strategy, also provide an additional benefit, which is the ability to estimate the probability for the k + 1 class. in other words, for a new text input the k +1 output estimates the probability that the text is going to receive polarized annotations. in order to better assess the ability of the model to provide such predictions, we used the polarized class probability along with the model-agnostic explainability framework of @xcite , which has been found to be the best option for text classification tasks @xcite . local interpretable model-agnostic explanations (lime) approximate the effect of features (words) on the model's output by training local surrogate models. the words suggested as explanations were not always easy to interpret or distinguish from the other two classes. one example from the cctk dataset is "i'm not black, but there's a whole lotta times i wish i could say i'm not white -frank zappa", where the words 'black' and 'white' contribute toward the decision for the k +1 class while the surname of frank zappa contributed reversely. to gain more insights into the k + 1 class of polarized annotations, we used error analysis as a proxy. a common mistake of models trained with unpolarized learning (i.e., using posts with non-zero ndfu to define the k + 1 class) concerns the misclassification of k + 1 posts to the civil class (confusion matrices in appendix a.3). this information, however, is not useful on its own. therefore, we focused solely on posts of the k + 1 class, exploring their average toxicity without the step of binarization, which we analyze next. as is shown in fig 5 , posts of the k + 1 class that are misclassified as civil (in blue, on the left) are often annotated as civil by the annotators. on the other hand, posts of the k + 1 class that are misclassified as toxic (in orange, on the right) are annotated more often as toxic. in other words, the generated/scipy.stats.mannwhitneyu.html women men figure 6: synthetic bimodal histogram of annotations, where annotators agree only conditioned on their gender. ## a posteriori unimodality explanations polarization may be due to various reasons, such as political beliefs, social dimensions, gender, age, etc. although ndfu estimates polarization ( §3.2), it does not suggest its cause. as an example, a bimodal annotation histogram is shown in figure 6 , where the colour reflects the gender of the annotator. although polarization is easily estimated in this histogram, its root cause (here, gender) is not revealed. to address this important issue, we next propose an approach that could possibly explain polarization given a dimension. let the set of opinions x of algorithm 1 for a non-zero ndfu post, and let g be the values for a dimension d that characterizes the opinion holder i, that is d @xmath0, ..., d g }. 10 based on the value of d corresponding to each annotator, the set x can be partitioned into g subsets x k (@xmath1, ..., g). this means that each subset x k contains the annotations of the annotators with value d k . then, we consider that d explains the polarization of x (ndf u (x) > 0), if the following a posteriori unimodality conditions hold: in the simple case of figure 6 , where only male and female annotators are considered, let x 1 denote the set of annotations from men (red histogram) and x 2 the set of annotations from women (blue histogram). it is obvious that ndf u (x 1 ) = 0 and ndf u (x 2 ) = 0 since each of them forms a unimodal histogram. consequently, it can be inferred that gender constitutes the source of polarization in the whole set of opinions x. we explored cctk posts that satisfy equation 4 regarding two dimensions, viz. gender and race. 11 source toxic civil text cctk lgbtq cntrl homosexuality, transgender sex-all are deviations from normal human behavior as well. cctk cntrl aa in other words, he's not brown enough for ya to be labeled a terrorist. ## ethical considerations and bias the detection of toxic language can be susceptible to data and algorithmic biases, unfairing underrepresented groups or preferring over-represented perspectives. our work makes a step toward recognising polarized opinions (ndfu) and toward understanding the poles in terms of annotator characteristics (aposteriori unimodality). ignoring polarized instances during inference can improve the classification performance, but it may also perpetuate or reinforce existing biases. in a real-world application bias perpetuation/reinforcement may be addressed by employing methodologically aposteriori unimodality testing. that is, it could be used to highlight topics in which the opinions of annotators from underrepresented groups (e.g., at risk for discrimination) deviate from those of annotators from other groups. for posts related to these topics, then, more annotators could be added (perhaps from focused groups), which can potentially lead also to debiasing. ## conclusions in this study we have focused on dfu, a measure that correlates well with human judgment for the assessment of polarized opinions. we have presented a normalized version, called ndfu, which not only correlates well with human judgment but is also more intuitive and interpretable that is important for tuning purposes. using ndfu, we suggested the unpolarized learning method for text classification, which introduces a new class that contains the items detected as polarized. in this way the original classes are trained using unimodal (unpolarized items) and classification performance is improved. experimenting with toxic language detection, an important and challenging task due to the subjective annotations, we showed that it outperforms the baseline with a statistically significant difference. finally, besides estimating polarization, we have shown that ndfu can also be used to trace the possible cause of polarization, by checking aposteriori unimodality conditions. putting gender and race under the microscope, we presented texts per feature for which annotators were polarized only in an inter-dimension setting. in future work, we will apply aposteriori unimodality to more datasets, developing a corpus of polarized texts, and facilitating the study of polarization. also, extensions of unpolarized learning will be investigated, exploring the path towards more accurate and fair nlp.