Gender and Representation Bias in GPT -3 Generated Stories (2021)

Abstract: Using topic modeling and lexicon-based word similarity, we find that stories generated by GPT-3 exhibit many known gender stereotypes. Generated stories depict different topics and descriptions depending on GPT-3’s perceived gender of the character in a prompt, with feminine characters more likely to be associated with family and appearance, and described as less powerful than masculine characters, even when associated with high power verbs in a prompt. Our study raises questions on how one can avoid unintended social biases when using large language models for storytelling.

## introduction advances in large language models have allowed new possibilities for their use in storytelling, such as machine-in-the-loop creative writing @xcite @xcite and narrative generation for games @xcite . however, fictional stories can reinforce real stereotypes, and artificially generated stories are no exception. language models mimic patterns in their training data, parroting or even amplifying social biases @xcite ). an ongoing line of research examines the nature and effects of these biases in natural language generation @xcite @xcite . language models generate different occupations and levels of respect for different genders, races, and sexual orientations @xcite . @xcite showed that gpt-3's association of muslims and violence can be difficult to diminish, even when prompts include anti-stereotype content. our work focuses on representational harms in generated narratives, especially the reproduction of gender stereotypes found in film, television, and books. we use gpt-3, a large language model that has been released as a commercial product and thus has potential for wide use in narrative generation tasks @xcite @xcite @xcite . our experiments compare gpt-3's stories with literature as a form of domain control, using generated stories and book excerpts that begin with the same sentence. we examine the topic distributions of books and gpt-3 stories, as well as the amount of attention given to characters' appearances, intellect, and power. we find that gpt-3's stories tend to include more masculine characters than feminine ones (mirroring a similar tendency in books), and identical prompts can lead to topics and descriptions that follow social stereotypes, depending on the prompt character's gender. stereotype-related topics in prompts tend to persist further in a story if the character's gender aligns with the stereotype. finally, using prompts containing different verbs, we are able to steer gpt-3 towards more intellectual, but not more powerful, characters. code and materials to support this work can be found at https://github.com/lucy3/gpt3_gender . ## data our prompts are single sentences containing main characters sampled from 402 english contemporary fiction books, which includes texts from the black book interactive project, global anglophone fiction, pulitzer prize winners, and bestsellers reported by publisher's weekly and the new york times. we use booknlp to find main characters and sentences containing them @xcite . we define a main character as someone who is within their book's top 2% most frequent characters and mentioned at least 50 times. every prompt is longer than 3 tokens, does not contain feminine or masculine pronouns, is from the main narrative and not dialogue, and contains only one singletoken character name. this results in 2154 characters, with 10 randomly selected prompts each. we use the gpt-3 api to obtain 5 text completions per prompt, with the davinci model, a temperature of 0.9, and a limit of 1800 tokens. a high temperature is often recommended to yield more "creative" responses @xcite . we also pull excerpts that begin with each prompt from the original books, where each excerpt length is the average length of stories generated by that prompt. this human-authored text provides a control that contains the same main character names and initial content as gpt-3 data. the collection of generated stories contains over 161 million tokens, and the set of book excerpts contains over 32 million tokens. ## text processing methods we use booknlp's tokenizer and dependency parser on our data @xcite , followed by coreference resolution on named entities using the model annotated and trained on literature by @xcite . pronoun chains containing the same character name within the same story are combined. ## topic differences given this dataset of book excerpts and stories generated by gpt-3, we carry out several analyses to understand the representation of gender within them. we focus on overall content differences between stories containing prompt characters of different genders in this current section, and lexiconbased stereotypes in §5. ## lexicon-based stereotypes now, we measure how much descriptions of characters correspond to a few established gender stereotypes. men are often portrayed as strong, intelli-gent, and natural leaders @xcite @xcite . popular culture has increased its attention towards women in science, politics, academia, and law @xcite @xcite . even so, depictions of women still foreground their physical appearances @xcite , and portray them as weak and less powerful @xcite . thus, our present study measures three dimensions of character descriptions: appearance, intellect, and power. ## conclusion the use of gpt-3 for storytelling requires a balance between creativity and controllability to avoid unintended generations. we show that multiple gender stereotypes occur in generated narratives, and can emerge even when prompts do not contain explicit gender cues or stereotype-related content. our study uses prompt design as a possible mechanism for mitigating bias, but we do not intend to shift the responsibility of preventing social harm from the creators of these systems to their users. future studies can use causal inference and more carefully designed prompts to untangle the factors that influence gpt-3 and other text generation models' narrative outputs.