Argument from Old Man’s View: Assessing Social Bias in Argumentation (2020)

Abstract: Social bias in language - towards genders, ethnicities, ages, and other social groups - poses a problem with ethical impact for many NLP applications. Recent research has shown that machine learning models trained on respective data may not only adopt, but even amplify the bias. So far, however, little attention has been paid to bias in computational argumentation. In this paper, we study the existence of social biases in large English debate portals. In particular, we train word embedding models on portal-specific corpora and systematically evaluate their bias using WEAT, an existing metric to measure bias in word embeddings. In a word co-occurrence analysis, we then investigate causes of bias. The results suggest that all tested debate corpora contain unbalanced and biased data, mostly in favor of male people with European-American names. Our empirical insights contribute towards an understanding of bias in argumentative data sources.

## introduction social bias can be understood as implicit or explicit prejudices against, as well as unequal treatment or discrimination of, certain social groups in society @xcite . a social group might be described by physical attributes of its members, such as sex and skin color, but also by more abstract categories, such as culture, heritage, gender identity, and religion. a typical, probably in itself biased, example of social bias is the old man's belief in classic gender stereotypes. in most cases, social bias is deemed negative and undesirable. recent research shows that bias towards social groups is also present in machine learning and natural language processing (nlp) models @xcite , manifesting in the encoded states of a language model @xcite or simply causing worse performance for underrepresented classes @xcite . such bias has been studied for different nlp contexts, including coreference resolution @xcite , machine translation @xcite , and the training of word embedding models @xcite . in contrast, computational argumentation (ca) has, to our knowledge, not seen any research in this direction so far. given that major envisioned applications of ca include the enhancement of human debating @xcite and the support of self-determined opinion formation @xcite , we argue that studying social bias is particularly critical for ca. in general, social bias may affect diverse stages of ca: in argument acquisition, for example, researchers may introduce social bias unintentionally, for instance, by collecting arguments from web sources that are only popular in a certain part of the world. this is known as sample bias @xcite . in argument quality assessment, a machine learning model may develop a prejudicial bias and judge arguments made by a certain social group better, for instance, because it considers features inadequate for the task, such as the gender @xcite . and in argument generation, a model might produce arguments that have an implicit bias towards a certain social group, for instance, because the features chosen are based on prior experience of the researchers and may not properly represent the whole population @xcite . as the examples indicate, social bias can, among other reasons, be caused by the source data and how it is being processed. as a starting point, this paper therefore focuses on social bias in the source data underlying ca methods. in particular, we ask the following questions: applications such as those outlined above usually rely on web arguments for scaling reasons. to study the questions, we hence resort to five english debate portals that give access to arguments on versatile topics: 4forums.com, convinceme.net, createdebate.com, debate.org, and changemyview. all five have been deployed in ca corpora @xcite @xcite . first, we analyze the general presence of social bias in each of the five debate portals. to this end, we train three custom word embedding models, one for each available corpus. next, we evaluate the models for social bias using a widely used bias metric, called weat @xcite , and compare the results. we then inspect the debate.org portal more closely with regard to specific social groups. in particular, we group the texts based on the provided user information and apply the same evaluation. lastly, to gain a better understanding of what makes some texts more biased than others, we explore their language by analyzing word co-occurrences with group identity words in the texts. our findings suggest that all three corpora are generally biased towards male (compared to female) and european-american (compared to african-american) people. this bias is not only reflected in the weat results, but also in unbalanced occurrences of identity words for certain social groups. more generally, we observe that the use of names as identity terms for social groups has unpredictable effects. with those insights, we contribute an initial understanding of social bias in sources of dialectical argumentative texts. ## related work in recent years, research on different types of bias in natural language has received a considerable amount of attention. media bias is one prominent example @xcite , particular the political bias of news articles @xcite . in various sub-fields of nlp, studies on media bias are concerned with analyzing techniques utilized by media outlets when reporting news. these include the framing of an event by phrasing the report with positive or negative terms, and selective reporting by including or omitting facts depending on the tone and (political) stance a media outlet wants to convey @xcite @xcite . we do not target media bias here but social bias. social bias can emerge from pre-existing stereotypes towards any social group @xcite , often leading to prejudices and discrimination. stereotypes are understood as "beliefs about the characteristics of group members" @xcite . they may be so powerful that they do not only limit the freedom of individuals, but also cause hate, exclusion, and -in the worst case -extermination @xcite . even if stereotypes, and with that social biases, are individually controllable, they persist to this day; prominent examples of social groups that have historically been subject to bias are ethnicities, genders, and age groups @xcite . a major factor in carrying and reinforcing those biases is language @xcite . in spoken and written argumentation and debates, biased language can be present if, for example, one sides argues in self-interest @xcite to favor a certain social group or uses unbalanced arguments @xcite . as ca methods are receiving more and more attention @xcite , it is important to understand how existing social biases influence them. one possible source is the human-generated data on which automated systems are trained and evaluated @xcite . in ca, one of the main sources of data are online debate portals, both for research and for applications @xcite . prior work has evaluated different properties of dialogical argumentation on debate portals. for example, @xcite evaluated the success rate of users in debates on debate.org based on prior experience, the users' social network in the portal, and linguistic features of their arguments. the authors find that information on a user is more informative in predicting the success compared to linguistic features. al @xcite retrieved all debates and posts from reddit's discussion forum changemyview to analyze the characteristics of debaters there. with this information, they were able to enhance existing approaches to predict the persuasiveness of an argument and a debater's resistance to be persuaded. while the evaluation of debates notably misses work on social biases, other forms of bias have been studied. @xcite , for example, attempted to build a classifier that predicts the presence or absence of myside bias in monological texts. in contrast, this work offers an initial evaluation of social bias in dialogical argumentation by analyzing the posts of debate portals. more generally, many approaches have been proposed to identify different types of social bias in word embedding models. in one of the first studies, @xcite showed that pre-trained models contain gender bias. they found that it is revealed when generating analogies for a given set of words. this suggests that the distance between word vectors can act as a proxy to identify biases held by a model. building on that notion, methods to automatically quantify the bias in a pre-trained word embedding model were presented. @xcite introduce a metric named the word embedding association test (weat) that adapts the idea of the implicit association test @xcite . other methods include the mean average cosine similarity (mac) test @xcite , the relational inner product association (ripa) test @xcite , the embedding coherence test (ect), and the embedding quality test (eqt) @xcite . for a more detailed literature review on detecting and mitigating biases in word embedding models, see @xcite . our approach builds on the weat metric to evaluate social biases in embedding models generated from debate portal texts. probably closest to our work is the study of @xcite . building on a method developed by @xcite , the authors analyzed scientific abstracts of biomedical studies published in a time span of 60 years to quantify gender bias in the field and to track changes over time. for this purpose, they generated separate word embedding models for each decade in their data and evaluated them using the weat metric. using the ripa test in addition, the authors identified the "most biased words" @xcite of each model. while we will apply a similar method to detect social bias in textual data, our study differs in two main regards: first, instead of biomedical abstracts, we evaluate dialogical argumentative structures extracted from online debate portals and compare them to each other. second, we additionally conduct a word co-occurrence analysis of the textual data as an attempt to get more insights into the weat results; something that is notably missing in previous studies. ## data to study social bias in argumentative language, we consider the dialogical argumentation found on online debate portals. as our analysis below is based on training word embedding models, which need sufficient data to find statistically meaningful co-occurrences, we resort to the three previously published corpora described below. the texts in these corpora have a similar argumentative structure, benefiting comparability: iac the internet argument corpus v2 @xcite contains around 16k debates from multiple debate portals, including createdebate.com, convinceme.net, and 4forums.com. the debates tackle various topics that are led by users of the respective portals. as the number of arguments from the single portals are rather small compared to the other two corpora, we only consider this corpus as a whole. cmv the webis-cmv-20 corpus (al khatib et al., 2020) covers posts and comments of roughly 65k debates from the internet platform reddit.com, specifically from its subreddit changemyview. each debate consists of multiple comments and arguments, including the opening post in which a user states an opinion and arguments on a given issue and prompts other users to provide opposing arguments. the most convincing counter arguments can then be awarded by the initiator of the debate. you're wrong. i'm pro abortion. women, as those caring for a child, should have the right to choose. this is not men's business, if they are only at work. ## experiments this section presents the experiments that we carried out on the given data in light of our three research questions, as well as the underlying methodology. we describe how we train embedding models, evaluate their bias, and analyze the word co-occurrences causing the observed bias. figure 1 illustrates the process. foot_0 ## results portal-specific and baseline models the weat results on the embedding models of the complete corpora in comparison to the baseline models can be seen in weat-10 has no result for ethnicity-black and age-below-23 due to too many out-of-vocabulary tokens. the weat-10 value above 2 of the age-23-up corpus is probably caused by a floating point error. the lowest values on average (e.g., 0.1300 for weat-5) and thus seems to be the least biased. in contrast, the debate.org corpus has higher values in almost all tests, with a noteworthy difference in the gender bias tests weat-6 and weat-8. in cases where it does not have the highest values (as for weat-6), it is surpassed by the cmv corpus. compared to the weat scores of the baseline models, the results of all debate corpora are mostly closer to the debiased numberbatch model than to glove commoncrawl. regarding the general direction of the weat scores, we see in group-specific models a similar observation can be made for the debate.org sub-corpora in with some exceptions, the overall direction of the weat results in tables 2 and 3 suggests that the three evaluated debate corpora are all biased towards men (compared to women) and the european-american word co-occurrences the co-occurrence counts in in most cases, however, words from both groups are only rarely mentioned in the same sentences, compared to the overall size of the corpora. for the tests that use names as social group identifiers, e.g. weat-6, this problem is even more noticeable, as indicated by the lower numbers presented in ## discussion to put the results into context, we will discuss two additional observations more closely in the following, namely the low number of occurrences of some identity words as well as the influence of using names for social group lexicons. we will then close this section with limitations observed during the analysis. ## conclusion in this paper, we have analyzed social bias in three debate corpora commonly used in ca research. to this end, we have trained custom word embedding models and evaluated their associations for multiple social groups using weat. further, we have analyzed co-occurrences of terms used to define the social groups. we have found all three corpora to show social bias, mostly towards women and african-american people. according to our evaluation, the smallest corpus, iac @xcite , carries the least social bias, whereas the debate.org corpus @xcite shows mostly the highest. the cmv corpus (al khatib et al., 2020) seems like a middle ground, as it contains the most data and is less biased than the debate.org corpus. in all three corpora, we have found imbalances regarding the representation of the evaluated social groups. future work should investigate additional ways in which social bias may be present in ca methods, as the underlying data is not the only source. other possible causes may be the features selected for a trained model or the way in which a model is applied to a real world problem. we believe that future ca corpora should be evaluated and mitigated for social bias. as data is underlying most ca methods, it is essential that it is as representative for as many social groups as possible in a balanced manner.