Investigating Societal Biases in a Poetry Composition System (2020)

Abstract: There is a growing collection of work analyzing and mitigating societal biases in language understanding, generation, and retrieval tasks, though examining biases in creative tasks remains underexplored. Creative language applications are meant for direct interaction with users, so it is important to quantify and mitigate societal biases in these applications. We introduce a novel study on a pipeline to mitigate societal biases when retrieving next verse suggestions in a poetry composition system. Our results suggest that data augmentation through sentiment style transfer has potential for mitigating societal biases.

## introduction our increasing reliance on natural language processing (nlp) tools to produce trustworthy and helpful information means we must also be increasingly vigilant to the social ramifications of nlp techniques. despite increasing attention to the ethical issues in nlp and development of techniques to mitigate biases in a variety of tasks, examining biases in creative nlp tasks remains underexplored; however, biases in creative tasks are equally as important. the primary goal of creative nlp systems is to be disseminated in a society (e.g., for self expression and collective social enjoyment, education (foster and freeman, 2008), therapy (connolly baker and mazza, 2004)), whereas other nlp systems are primarily driven by some non-social goal (e.g., answer a query correctly or retrieve all relevant named entities). any existing societal biases propagated through creative systems have direct impact on our society. for example, biases in a system that is meant to educate about different perspectives may end up reinforcing demographic stereotypes. in this work, we focus on quantifying and mitigating societal biases in a creative language application. specifically, we propose techniques to mitigate biases in the poetry composition system introduced by uthus et al. (2019). this system allows the user to collaboratively compose a poem using machinesuggested novel verses in the style of classic american poets. as creative works are often shaped by the lived experiences and timely issues of the creator's life, a poetry composition system trained on poems from different authors of different eras may reflect a variety of societal biases. we define societal biases as unequal social perceptions of different socially-defined groups of people. propagating unequal representations of demographic groups reinforces representational harms, such as stereotypes, leading to discrimination and potential allocational harms, such as unequal job opportunities (blodgett et al., 2020; barocas et al., 2017). @xcite to evaluate societal biases, we examine the language polarity (i.e., sentiment) of the suggested verses when different demographic groups are mentioned in the user input. @xcite a system that favors different sentiment verse suggestions for mentions of different demographic groups (e.g., positive for demographic a and negative for demographic b) could propagate unequal positive and negative associations and amplify existing demographic inequalities. however, suggested next verses for user input: the women suggested next verses for user input: the men hate to their love like that evil womanhood-looked the warm day in their manly way+ drawed tears upon their wall-lay on their crowns at their gracious whim+ hide in the shame of their loveless men-await, for the brand of their command heard, if they were with flowers brought to their manhood in the light-eyed do with their wings and their dolls ran with their rifles in this preliminary study, we focus on retrieving less negative verses across different demographic groups to reduce harms from societal biases within and across demographic groups. @xcite since there is no guarantee that verses with similar sentiment are biased or unbiased in the same way, we do not explicitly constrain equalizing the sentiments of verses suggested for different groups. results show that our method has promising results for both reducing negative verses and keeping the distribution of verse sentiments across groups comparable. our contributions are 1) a pipeline approach for mitigating societal biases in a poetry composition system and 2) a labeled poetry sentiment dataset. for the first part of the pipeline, we introduce a poetry sentiment dataset and build a bert-based (devlin et al., 2019) sentiment analyzer for poetry. these sentiment tools are subsequently used to train a style transfer model (li et al., 2018), which is then used to augment data to train the next verse prediction component in the poetry composition system. our results indicate that style transfer has potential as an augmentation technique to reduce societal biases. specifically, we can influence the model to suggest verses with more positive sentiment while keeping the suggested verse quality comparable. this exploratory study introduces the capabilities of style transfer augmentation to mitigate biases and is an example of how bias mitigation can be applied to creative language tasks and information retrieval components. ## poetry composition system we investigate societal biases in the human-ai collaborative approach to composing poetry described by uthus et al. (2019). in this setup, users compose a poem aided by suggestions from the poetry system. users can either directly use verse suggestions provided by the system, modify the suggestions, or create their own verses. the suggested verses are generated in the style of various classic american poets (e.g., walt whitman, emily dickinson). figure 1 shows a schematic of the poetry composition system. this system has two components: next verse prediction loss. the next verse prediction model is a dual-encoder model similar to the one described by henderson et al. (2017). more specifically, let r be the entire fixed set of verse suggestions from the index of generated verses. given a verse x, the model's goal is to search for the top n responses (y 1 , y 2 , ..., y n ) ∈ r, ordered by decreasing model probability eq. ( 1 ) requires summing over all possible responses y r when training, which is prohibitively expensive to calculate. instead, we follow henderson et al. (2017) and sample k responses to estimate p (y|x) as in this work, the joint probability p (x, y) is estimated with a learned scoring function s, where p (x, y) ∝ e s(x,y) , so we can rewrite eq. ( 2 ) as for the scoring function s, we use a tower of transformer (vaswani et al., 2017) layers and feed-forward layers to encode x and y into the vectors h x and h y , respectively. @xcite the dot product scoring function can then be written as s(x, y) = h x h y . recall that to approximate the conditional probability p approx (y|x), we sample k responses. when annotated verses bert sentiment classifier positive verses negative verses style transfer augmented corpus next verse prediction evaluation gutenberg poem dataset index of generated verses training with a batch size of k, we treat (x i , y i ) as the positive example and (x i , y k ) where @xmath0, for i, @xmath1, k]. from preliminary experiments, we find that including (x i , x i ) as a negative example improves retrieval, so we add this additional negative example for our experiments. we define θ as the model's parameters. with this formulation, the average batch loss to minimize is the negative log probability next verse prediction dataset. we collect a corpus of classic poems from project gutenberg foot_3 to use as training data for the next verse prediction component -we refer to this collected dataset as the gutenberg poem dataset in this work and rely on the dataset for several components in our bias mitigation pipeline. we split each poem into verses, and pair each verse with the subsequent verse to form the groundtruth data for next verse prediction. foot_4 ## bias mitigation through data augmentation we propose a technique for bias mitigation through data augmentation of the training data. data augmentation has been used in previous works (lu et al., 2018; zhao et al., 2018a; park et al., 2018) as a technique to mitigate societal biases in nlp systems. we use sentiment style transfer for data augmentation to target the retrieval of verses with less negative societal biases towards different demographic groups. our overall pipeline for augmentation and evaluation is shown in figure 2 . by using style transfer for data augmentation instead of filtering out negative examples, we can circumvent data sparsity issues and promote model robustness. in the creative language domain, it can be difficult to obtain a large dataset, due to intellectual property and copyright restrictions. with style transfer-based augmentation, we can generate styled variations of our original data that can be more consistent in content and poet style. in this section, we introduce definitions and then describe our proposed solution for bias mitigation through data augmentation. ## conclusion we introduce an exploratory study on societal biases in a poetry composition application. although biases in creative language applications are underexplored, it is important to examine biases in these applications that are primarily intended for social use. our results indicate that style transfer has potential as an augmentation technique to reduce societal biases. more broadly, our study introduces the capabilities of style transfer augmentation to mitigate biases and as an example of how bias mitigation can be applied to information retrieval components.