Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021)

Abstract: Warning: this paper contains content that may be offensive or upsetting. Commonsense knowledge bases (CSKB) are increasingly used for various natural language processing tasks. Since CSKBs are mostly human-generated and may reflect societal biases, it is important to ensure that such biases are not conflated with the notion of commonsense. Here we focus on two widely used CSKBs, ConceptNet and GenericsKB, and establish the presence of bias in the form of two types of representational harms, overgeneralization of polarized perceptions and representation disparity across different demographic groups in both CSKBs. Next, we find similar representational harms for downstream models that use ConceptNet. Finally, we propose a filtering-based approach for mitigating such harms, and observe that our filtered-based approach can reduce the issues in both resources and models but leads to a performance drop, leaving room for future work to build fairer and stronger commonsense models.

## introduction commonsense knowledge is important for a wide range of natural language processing (nlp) tasks as a way to incorporate information about everyday situations necessary for human language understanding. numerous models have included knowledge resources such as conceptnet @xcite for question answering @xcite , sarcasm generation @xcite , and dialogue response generation @xcite , among others. however, commonsense knowledge resources are mostly human-generated, either crowdsourced from the public @xcite or crawled from massive web corpora @xcite . for example, conceptnet originated from the open mind common sense project that collects commonsense statements online from web users @xcite foot_0 and genericskb consists of crawled text from public websites. one issue with this approach is that the crowdsourcing workers and web page writers may conflate their own prejudices with the notion of commonsense. for instance, we have found that querying for some target words such as "church" as shown in the potentially biased nature of commonsense knowledge bases (cskb), given their increasing popularity, raises the urgent need to quantify biases both in the knowledge resources and in the downstream models that use these resources. we present the first study on measuring bias in two large cskbs, namely conceptnet @xcite , the most widely used knowledge graph in commonsense reasoning tasks, and genericskb @xcite , which expresses knowledge in the form of natural language sentences and has gained increasing usage. we formalize a new quantification of "representational harms," i.e., how social groups (referred to as "targets") are perceived @xcite in the context of cskbs. we consider two types of such harms in the context of cskbs. one is intra-target overgeneralization, indicating that "common sense" in these resources may unfairly attribute a polarized (nega-tive or positive) characteristic to all members of a target class such as "lawyers are dishonest." the other is inter-target disparity, occurring when targets have significantly different coverage in the cskb in terms of both the number of statements about the targets (e.g., "persian" might have much fewer cs statements than "british") and perception toward the targets ("islam" might have more negative cs statements than "christian"). we propose a quantification of overgeneralization and disparity in cskbs using two proxy measures of polarized perceptions: sentiment and regard @xcite . applying the proposed metrics of bias to conceptnet and genericskb, we find harmful overgeneralizations of both negative and positive perceptions over many target groups, indicating that human biases have been conflated with "common sense" in these resources. we find severe disparities across the targets in demographic categories such as professions and genders, both in the number of statements and the polarized perceptions about the targets. we then examine two generative downstream tasks and the corresponding models that use con-ceptnet. specifically, we focus on automatic knowledge graph construction and story generation and quantify biases in comet @xcite and commonsensestorygen (csg) @xcite . we find that these models also contain the harmful overgeneralizations and disparities found in conceptnet. we then design a simple mitigation method that filters unwanted triples according to our measures in conceptnet. we retrain comet using filtered conceptnet and show that our proposed mitigation approach helps in reducing both overgeneralization and disparity issues in the comet model but leads to a performance drop in terms of the quality of triples generated according to human evaluations. we open-source our data and prompts to evaluate biases in commonsense resources and models for future work foot_1 . ## representational harms in cskbs our formalization of representational harms is defined over statements. genericskb uses naturally occurring sentences, so our measures (in sec. 2) directly apply. for conceptnet, we convert each kb triple containing the targets t @xmath0, as detailed in the following sub-sections. ## related work work on fairness in nlp has expanded to different applications and domains including coreference resolution @xcite , named entity recognition @xcite , machine translation (font and costa-jussà, 2019), word embedding @xcite @xcite , as well as surveys @xcite @xcite . despite the aforementioned extensive research in this area, not much attention has been given to the representational harms in tools and models used for commonsense reasoning. injecting commonsense knowledge into nlp tasks is gaining attention @xcite . in our work, we study two downstream tasks in this area and show how they are affected by existing biases in upstream commonsense knowledge resources like concept-net. although @xcite have previously shown that conceptnet word embeddings @xcite are less biased compared to other embeddings, we demonstrate that destructive biases still exist in conceptnet that need to be carefully studied. ## conclusion incorporating commonsense knowledge into models is becoming a popular trend as it is important for our models to mimic humans and the way they utilize commonsense knowledge in performing different tasks. one danger of mimicking humans is adopting their biases. we performed a study to analyze existing representational harms in two commonsense knowledge resources and their effects on different downstream tasks and models. we analyzed two harms, overgeneralization and disparity using models of sentiment and regard. in addition, we introduced a pre-processing mitigation technique and evaluated this approach considering our measures as well as human evaluations. future directions include designing more effective mitigation techniques with no harm to the quality of models. 2020) and made extensions to fill gaps in these groups. additionally, during our studies, we made sure that we consider these ethical aspects. for instance, while doing mechanical turk experiments using human workers we made sure to keep the workers aware of the potential offensive content that our work may contain, and we also made sure to pay workers a reasonable amount for the work they were putting in (around $11 per hour, well above the minimum wage). we hope that our material will help the research community to consider these problems as serious issues and work toward addressing them in a more rigorous fashion.