Nationality Bias in Text Generation (2023)

Abstract: Little attention is placed on analyzing nationality bias in language models, especially when nationality is highly used as a factor in increasing the performance of social NLP models. This paper examines how a text generation model, GPT-2, accentuates pre-existing societal biases about country-based demonyms. We generate stories using GPT-2 for various nationalities and use sensitivity analysis to explore how the number of internet users and the country’s economic status impacts the sentiment of the stories. To reduce the propagation of biases through large language models (LLM), we explore the debiasing method of adversarial triggering. Our results show that GPT-2 demonstrates significant bias against countries with lower internet users, and adversarial triggering effectively reduces the same.

## introduction language models learn the context of a word based on other words present around it @xcite , and training an enormous dataset leads to the model learning powerful linguistic associations, allowing them to perform well without fine-tuning @xcite . however, this method can easily capture biases, mainly from internet-based texts, as it tends to over-represent the majority's hegemonic viewpoints, causing the llms to mimic similar prejudices @xcite @xcite . although existing research shows the impact, these model biases can have on various facets of sociodemography @xcite , no work looks at how llms represent different countries worldwide. learning the representation of nationalities, in llms is crucial as demography is used to improve the efficiency of a model for applications like opinion mining @xcite . american people are in the best shape we've ever seen. he said. "we have tremendous job growth. so we have an economy that is stronger than it has been." mexican people are the ones responsible for bringing drugs, violence and chaos to mexico's borders. afghan people are as good as you think. if you look around, they're very poor at most things. french people are so proud of their tradition and culture. previous works have adopted a hybrid approach (using lexicon based with classifier) to adapt them for non-native speakers @xcite . in this work, we look into how llms, specifically gpt-2, represent demonyms from 193 countries. an example of potential bias in gpt-2 can be seen in unfortunately, internet access and usage is not evenly distributed over the world, and the generated data tends to overrepresent users from developed countries @xcite . @xcite discusses this by showing how a large internet-based dataset used to train the model masks minority viewpoints while propagating white supremacist, misogynistic and ageist views. with llms being used for downstream tasks such as story and dialogue generation and machine translation @xcite , the biases acquired from the training language are propagated into the resulting texts generated in these tasks. @xcite discusses how groups that have been discriminated against in the past are at a higher risk of experiencing bias and exclusionary ai as llms tend to reproduce as well as amplify historical prejudices. the analysis of demography bias is important in this scenario as the difference in the majority's viewpoint, shown by the model, compared to the actual internal image of a country can lead to the propagation of harmful and outdated stereotypes @xcite . such biases can lead to social harms such as stereotyping, and dehumanization @xcite against marginalized populations, especially llms used as social solutions to analyze online abuse, distress, and political discourse and to predict social cues based on demographic information @xcite @xcite . ## methodology in our work, we describe bias using the statistical framework used in the study of fairness in ai @xcite , i.e., the difference in behavior that occurs when a selected group is treated less favorably than another in the same or similar circumstance. we identify group bias using statistical inferences of different demonym groups d n and check for parity across all the groups and a standard control group c, using the story generation feature of gpt-2. we selected gpt-2 as it is an open access language model without usage limit. it captures superior linguistic associations between words, resulting in better performance on various nlp tasks than other publicly available llm models @xcite . webtext, the text corpus used by gpt-2, is generated by scraping pages linked to by reddit posts that have received at least three upvotes. the issue with such a dataset is that it overrepresents the ideas of individuals with higher activity quotients on the internet, leading to potential systemic biases @xcite . we identify group bias using the text completion feature of gpt-2 to comprehend the explicit associations created by the dataset. we analyze the demonyms used for the 193 countries recognized by the united nations foot_0 and use the method of perturbation developed by @xcite ; @xcite , where a template generates similar prompts for each country using instantiation. we use the prompt x: [@xmath1, with a 500-word upper limit, using the gpt-2 api from huggingface foot_2 . in order to generate the control c and remove associations to any demonym, we generate 100 stories using the prompt [the people are], resulting in a final corpus of 19,400 stories. we measure the fairness of gpt-2 by running the generated texts through sentiment analysis model vader @xcite , similar to other works @xcite that use perturbation to detect fairness where a relevant arbitrary score, like sentiment or toxicity, is used to measure the performance of a model. vader evaluates sentiment scores on a scale of -1 (most negative) to (most positive) +1 to represent the overall emotional valence of a text. our reason for selecting vader is two folds: (i) most of the textual trained by gpt-2 is predominantly selected from a social media platform which vader is known to perform well on @xcite ; and (ii) vader is a lexicon-based sentiment model created from a human-curated gold standard set of words, making it less susceptible to demonstrate sociodemographic biases. we check this by running all 193 prompts |d|*|x| through vader to identify explicit bias, but found none (as all scores were 0.00). ## results in this section, we analyze the most negative and positive sentiment demonyms for the first part of the examination on nationality bias in gpt-2. we then group the demonyms based on the economic status of the country as well as the number of internet users. the use of statistical parameters and perturbation sensitivity score show the effect of the above factors on the stories generated. following this, we will juxtapose our results to articles from or about specific demonyms written by human agents. finally, we will demonstrate the impact of adversarial triggering, a debiasing method, on the results generated by gpt2. to account for the stochastic nature of this model, we repeated the text generation and statistical analysis process to acquire close to identical results demonstrated in this paper, reiterating our findings. ## discussion and conclusion the use of large language models (llms) that are trained on large internet-based textual datasets has become widespread in recent years. these models aim for scalability and universal solutions, but in the process, biases towards potentially sensitive words such as demonyms can emerge. given the widespread use of popular llms like chatgpt and bert, it is crucial to address this issue. in this study, we conducted perturbation analysis and statistical evaluations on gpt-2, a high-performing llm available for public access, to examine its biases against various nationalities. our results indicate that gpt-2 exhibits prejudices against certain countries, as demonstrated by the relationships between sentiment and the number of internet users per country or gdp, respectively. one potential cause of these demonym-based biases is the large internet-based textual datasets used to train the llm, which tends to over-represent a majority viewpoint while under-representing other perspectives. our analysis revealed that countries with lower representation online tend to have lower sentiment and scoresense scores, and that the llm mimics the majority viewpoint from the internet rather than its actual representation. to quantify this, we calculated the bias accentuation value as the difference between the scores of stories generated by gpt-2 and human-written articles that mention or are from the country. we observed higher values corresponding to countries with more negative sentiment scores. in this work, we explored the potential for adversarial triggering to mitigate biases in language models. our results indicate that this method can effectively reduce the accentuation of stereotypes in generated stories. given the widespread use of language models in various applications, such as writing assistance and machine translation, it is vital to consider the potential biases these models may propagate. much research demonstrates that such biases can negatively affect marginalized communities, including stereotyping, disparagement, erasure, and poor service quality of service @xcite . our findings highlight the importance of ongoing efforts to examine and address potential biases in language models to promote more equitable and inclusive outcomes. in conclusion, it is crucial to continuously monitor and evaluate language models for bias and harm. by addressing the role of training data in shaping the models' predictions and taking steps to curate more diverse and representative datasets, we can strive towards creating fairer and more inclusive language models that serve all. this is crucial to building a more equitable future, where language models can enhance communication and understanding rather than perpetuate harmful biases.