S tereo KG : Data-Driven Knowledge Graph Construction For Cultural Knowledge and Stereotypes (2022)

Abstract: Analyzing ethnic or religious bias is important for improving fairness, accountability, and transparency of natural language processing models. However, many techniques rely on human-compiled lists of bias terms, which are expensive to create and are limited in coverage. In this study, we present a fully data-driven pipeline for generating a knowledge graph (KG) of cultural knowledge and stereotypes. Our resulting KG covers 5 religious groups and 5 nationalities and can easily be extended to more entities. Our human evaluation shows that the majority (59.2%) of non-singleton entries are coherent and complete stereotypes. We further show that performing intermediate masked language model training on the verbalized KG leads to a higher level of cultural awareness in the model and has the potential to increase classification performance on knowledge-crucial samples on a related task, i.e., hate speech detection.

## introduction fairness, accountability, and transparency to fight model-inherent bias and discrimination have become a major branch of machine learning research in recent years. this includes studying cultural bias and stereotypes in datasets and language models. stereotypes are cognitive schemas that aid in categorizing and perceiving other social groups @xcite , and becoming conscious of this stereotyping can increase cultural knowledge and sensitivity @xcite . however, without mindfulness, stereotypes lead to inferring traits of individuals from their (e.g., socio-economic) status or social group @xcite , which then leads to systemic discrimination. stereotypes as inherent cognitive functions are equally present in human-generated content, e.g., text resources used to train machine learning algorithms, which then further propagate and lead to discrimination @xcite . within the natural language processing community, bias reduction includes work in reducing gender @xcite , ethnic or religious bias @xcite in word embeddings or classification tasks @xcite @xcite . nevertheless, these techniques often rely on predefined lexicons, which are mostly humanwritten and thus expensive in their creation. instead, we present an entirely data-driven pipeline for the creation of a scalable knowledge graph (kg) of cultural knowledge and stereotypes. our resulting knowledge graph, called stereokg, consists of 4,722 entries about 10 different social groups, i.e., 5 religious groups and 5 nationalities. this knowledge graph has several use cases, ranging from analyzing existing stereotypical and cultural knowledge online, to removing ethnic and religious bias or increasing the cultural awareness of classifiers. in our experiments, we focus on the latter: integration of cultural knowledge to improve classification performance. overall, our contributions are threefold: the rest of the paper is structured as follows: after describing the related work (section 2), we present our knowledge graph creation technique (section 3) which is then evaluated in a quantitative and qualitative fashion (section 4). section 5 describes the knowledge integration experiments, which constitute downstream task performance on hate speech detection and masked language modelling predictions of cultural content. we then discuss (section 6) and conclude (section 7) our findings. ethical concerns are addressed in appendix a. ## related work cultural knowledge about different social groups and entities plays an important role in responding to contextual situations. in this work, we target cultural knowledge as a form of commonsense @xcite . incorporating cultural commonsense in reasoning tasks is an understudied practice in nlp. @xcite study the variation of cultural commonsense and how it affects computer applications. while there exist knowledge base resources for general commonsense @xcite @xcite , to the best of our knowledge, @xcite have provided the only work targeting the construction of a cultural knowledge graph, which comprises various rituals and customs for two cultures. however, since it relies on crowdsourcing, it is limited in its coverage and is not easily extendable. cultural knowledge is largely correlated to stereotypes. contrary to exhaustive research avenues analyzing gender and ethnic stereotypes, our work focuses on the lesser-studied nationality and religious stereotypes. @xcite have shown that national stereotypes could be grounded in the collective linguistic behavior of nations, while the harvard pluralism project foot_0 stresses the importance of considering religion as a factor for prejudice. because of the diversity of social groups and their behavioral traits, stereotypes and cultural attributes have unclear boundaries, making it difficult to distinguish between the two. keeping this in mind, we treat cultural knowledge and stereotypes as interchangeable terms. stereotypes have been used to estimate bias in language models using curated datasets @xcite . stereotypical data has also been extracted from search engine auto-complete predictions using query prompts @xcite and then used for analyzing how language models learn these concepts @xcite . @xcite use minimal pairs of male-female terms to debias word embeddings. in our work, we create a unified resource of cultural knowledge and stereotypes. knowledge graphs serve as sources of representing knowledge in a structured format. factual knowledge bases such as dbpedia @xcite , freebase @xcite and wikidata (vrandečić and krötzsch, 2014) contain grounded knowledge about individual entities. knowledge graph construction for commonsense reasoning has also been a common object of research @xcite @xcite . while some kgs comprise an if-then reasoning scheme @xcite , some contain knowledge in the form of triples @xcite or as simple natural language sentences @xcite . crowdsourced kgs, e.g., wikidata, result in good quality knowledge, but require largescale manual annotation and resources. in contrast, kgs constructed in an automated manner have a lower cost in construction, are easily extendable, and have been shown to be useful in several downstream applications @xcite . for example, @xcite use questions as prompts for learning commonsense cues from search engine query logs and question-answering forums and construct a commonsense knowledge base. explicit knowledge integration of knowledge resources into language models can be roughly categorized into fusion based approaches and language modeling based approaches. fusion based approaches @xcite @xcite typically perform knowledge integration by combining language model representations with representations extracted from knowledge bases. compared to language modeling based approaches, as explored by us, they rely on aligned data and are typically applied during the pre-training stage. language modeling based approaches commonly start from a pre-trained language model and perform knowledge integration via intermediate pre-training. for example, @xcite integrate commonsense knowledge by performing language modeling on triples ob-tained from atomic and conceptnet. recently, @xcite analyzed this approach in the fewshot training setting. in contrast to our study, both works consider autoregressive language models and use the resulting models for knowledge base construction, while we study the impact of knowledge integration on downstream task performance. similar to our work, @xcite integrate commonsense knowledge via masked language modeling. they obtain sentences for intermediate pre-training by randomly traversing the conceptnet knowledge graph. unlike our work, they do not update the weights of the pre-trained model and train adapter layers instead. moreover, while we focus on hate-speech classification as our downstream task, they evaluate on glue. ## knowledge graph construction we focus our cultural kg on 5 religious (atheism foot_1 , christianity, hinduism, islam, judaism) and 5 national (american, chinese, french, german, indian) entities. previous work on automatic kg creation depended on external algorithms, i.e., autocompletion of search engine queries @xcite @xcite . this dependency is limiting, as external providers may filter foot_2 outputs of their autocomplete algorithm, especially on sensitive topics such as culture and identity. instead, we keep control over the whole kg creation process. the entire kg construction pipeline is illustrated in figure 1 . using statement and question mining, cultural knowledge and stereotypes regarding our entities of interest are collected from two social media platforms, reddit and twitter. for reddit, we limit our search to subreddits relevant for the respective subjects (e.g. r/germany for germans) together with common question-answering subreddits (e.g., r/askreddit) using the praw foot_3 library. the complete list of queried subreddits is given in appendix b. similar to the commonsense mining approach by @xcite and @xcite , we use fixed question and statement templates ( to reduce redundancies in the kg triples, we cluster the mined sentences with similar content together using the fast clustering method for community detection implemented in the sentencetransformers foot_5 (reimers and gurevych, 2019) library. this step results in 6,993 singletons and 610 clusters with more than one instance. we hypothesize that non-singleton clusters are better representatives of cultural knowledge and stereotypes, as these are based on questions that have been asked by several users, while singletons may be based on unique thoughts which do not represent a popular stereotype or cultural reality. the qualitative difference between singletons and clusters is evaluated in section 4.2. all assertions are then converted into triples using openie @xcite . as openie outputs multiple triples which may be noisy or irrelevant, they are filtered using the following heuristics: ## knowledge graph evaluation the resulting kg consists of 4,722 entries, with americans being the largest represented group (1,071 entries) and jews (43) the smallest. the proposed pipeline can also be utilised to extend the kg with additional entities. in the following section, we describe the qualitative and quantitative evaluation of the generated kg. ## knowledge integration to explore how stereokg can be used to integrate knowledge into an existing language model, we perform intermediate masked language modeling (mlm) on it in its structured (verbalized triple) and unstructured (sentence) form. the unstructured knowledge is more expressive and verbose, while the structured knowledge from triples is concise and less noisy as compared to the unstructured data. we then fine-tune and evaluate the language model performance on hate speech detection, a task for which we esteem stereotype knowledge to be of use. ## discussion we create an automated pipeline to extract cultural and stereotypical knowledge from the internet in the form of queries. while this overcomes the limitations and expenses of crowdsourcing and is easily extendable to a large number of entities, several shortcomings still need to be addressed. automated extraction results in irrelevant and noisy data, which is augmented by erroneous outputs during triple creation. this is also evidenced in the human evaluation that corroborates the existence of many incomplete triples in the resultant kg, which could also be due to the noisy openie outputs. other stages in the analysis, such as statement conversion, fast clustering, and triple verbalization give sufficiently good approximations. our knowledge integration experiments suggest that performing intermediate mlm training on (verbalized) cultural knowledge can improve the classification performance on knowledge-crucial samples. however, the sample of stereotypical examples in the test/dev sets of both hate speech corpora is low (9 for olid and 33 for wsf), indicating that a more extensive dedicated hate speech test set focusing on stereotype entities is required to reduce error margins and verify results. our experiments are limited to intermediate mlm training and we leave the exploration of other knowledge integration techniques for future work. our work serves as a preliminary research for studying stereotypes and cultural knowledge across different entities. extending the kg for other entities than the one proposed in our work is easily done by plugging in new entities into our query templates ( the content used for the construction of stereokg stems from english-speaking twitter and reddit. this comprises a specific demographic which is only a subset of our global society. the stereotypes and cultural knowledge included in stereokg therefore also underlie this sampling model acc@5 (%) example pred (top 3) base 37 muslims are turning away [science]. too, now, again base+uk 48 americans don't have free [healthcare]. healthcare, lunch, tuition base+sk 45 americans are voting for [trump]. freedom, democracy, them dt 37 atheists unilaterally support [abortion]. fascism, abortion, terrorism dt+uk 49 muslims compare apostasy to [treason] treason, sin, genocide dt+sk 41 chinese toilets are [dirty]. disgusting, awful, shit ## conclusion this study presents stereokg, a scalable datadriven knowledge graph of 4,722 cultural knowledge and stereotype entries spanning 5 religions and 5 nationalities. we describe our automated kg creation pipeline and evaluate the resulting kg quality through human annotation, showing that the majority of cluster-derived entries in the kg are of high quality (success rate 59.2%) and more common and credible than their singleton counterparts. the kg can easily be extended to include other nationalities as well as genders, sexual orientations, professions, etc., as the underlying subjects. further, performing intermediate mlm training on verbalized instances of stereokg greatly improves the models' capabilities to predict culture-related content. this improvement of cultural awareness has a positive effect on knowledge-crucial samples, where we observe a slight improvement in classification performance on a related downstream task, i.e., hate speech detection. future work should focus on differentiating between cultural facts that should be represented in language models and stigmatizing stereotypes that should not be present in language models. we make stereokg and the code of our kg creation pipeline available under https:// github.com/uds-lsv/stereokg .