Born Differently Makes a Difference: Counterfactual Study of Bias in Biography Generation from a Data-to-Text Perspective (2024)

Abstract: How do personal attributes affect biography generation? Addressing this question requires an identical pair of biographies where only the personal attributes of interest are different. However, it is rare in the real world. To address this, we propose a counterfactual methodology from a data-to-text perspective, manipulating the personal attributes of interest while keeping the co-occurring attributes unchanged. We first validate that the fine-tuned Flan-T5 model generates the biographies based on the given attributes. This work expands the analysis of gender-centered bias in text generation. Our results confirm the well-known bias in gender and also show the bias in regions, in both individual and its related co-occurring attributes in semantic machining and sentiment.

## introduction to what extent do personal attributes affect biography content? biography consists of the facts of personal attributes @xcite . current research has shown that biographies from wikipedia reflect bias from society @xcite , such as well-known bias in gender @xcite @xcite @xcite and culture @xcite @xcite . however, personal attributes are compounded. for instance, religions could be prevalent based on geography @xcite . this results in the challenge of isolating co-occurring attributes and evaluating the effect of personal attributes alone. answering this question directly would require paired-wise comparisons of biographies that are identical except for the particular personal attribute of interest @xcite . it would allow us to measure the causal effect of the attribute value (treatment) on biography text (outcome) @xcite . however, having such identical biographies is rare and nearly impossible. additionally, wikipedia biographies mostly consist of notable people. 1 large language models (llms) have shown the capability of remembering training data @xcite and generating factual biographies based on only names of celebrities @xcite . in light of these observations, we propose a counterfactual methodology based on a data-to-text framework. we formulate the task as generating biographies by given attributes (figure 1 , top-@xmath0. by doing so, we maintain a controllable setting, enforcing biography generation focusing on the given attributes, thus allowing us to study the effect of individual personal attributes. to mitigate the effect of celebrities, we do our analysis on carefully designed fictional biographies, the syn-thbio dataset @xcite , where fictional names and related personal attributes are controlled by human-llms collaboration. since personal attributes are compounded and diverse, we consider two universal types of personal attributes, i.e., gender and region. we evaluate the generated biographies from two dimensions: se-mantic matching @xcite , evaluating how the biography correctly represents the meaning in the attributes; and, sentiment @xcite , measuring how positive or negative the tone of the text is. we first show a significant difference among generated biographies from different gender and region groups in both semantic matching and sentiment (section 3). we further perform counterfactual analysis by explicitly manipulating the personal attributes of interest (section 4). we compare the generated biographies (figure 1 , top-right vs., bottom-right, respectively) from true attributes (male, top-left) vs. manipulated attributes (female, bottom-left). we ask how would the generated biographies change if the given personal attributes were changed? we show that disentangling individual and related co-occurring personal attributes, llms finetuned on the wikibio dataset @xcite encode gender and region bias in semantic matching and sentiment, prompting further research in biography generation going beyond gendercentered @xcite , and general quality evaluations, e.g., rouge @xcite . ## methodology data we use the wikibio dataset @xcite for training, consisting of 728,321 biographies from real english wikipedia pages where the infobox and first paragraph from the articles are provided. on average, each infobox contains 12.5 personal attributes. we explicitly add the gender label (male, female or non-binary/identifiable), inferring from the pronouns in the paragraph @xcite , to the infobox. we remove the biographies where the nationality is not available. to mitigate the cross-contamination of training and evaluation sets @xcite , we use the synthbio dataset @xcite for evaluation, which is a synthetic dataset consisting of structured attributes-which we refer as true attributes-describing fictional individuals. it consists of 2,237 infoboxes and each infobox has on average 19 personal attributes and multiple fictional biographies. the comparison of the wikibio and synthbio datasets is shown in ## true attributed biography generation first, we validate that the fine-tuned flan-t5 model generates biographies based on the given personal attributes. to explore the effect of personal attributes, we compare the semantic matching and sentiment on the generated biographies with true attributes (equation ( 1 )) against those without given the particular attribute (masked), i.e., model validation our fine-tuned flan-t5 model outperforms the t5 model @xcite reported in the synthbio dataset @xcite , with a rougel score of 26.4 (vs., 22.6) and a parent-f score @xcite of 0.114 (vs., 0.049). we first validate whether the personal attribute of interest can be inferred from the biographies. specifically, for gender, we use the pronouns as the proxy of gender @xcite and compare it against the given gender attribute. for the region, since there is no direct method to predict the nationality from the biography, we consider whether the nationality or related country name is mentioned in the biography as the proxy of the nationality encoded in the biography. we do not train a classifier for nationality as the biography contains rich personal information-the classifier may remember the training instances instead of the nationality signals. we then group the results for nationality based on the region. as shown in true attributed biography do llms generate different biographies for different gender and nationality groups? figure 2 shows that generated biographies are significantly different among different gender groups (purple bars, gender) in semantic matching and sentiment. 6 for region, we observe significant differences in some region groups, e.g., af vs., ap in both measurements, indicating the potential bias among region groups. however, we do not observe constant significant differences for any particular region. ## counterfactual attributed generation we apply our counterfactual methodology based on our fine-tuned flan-t5 model. we manipulate only the personal attributes of interest and keep the cooccurring attribute unchanged to study the effect of individual attributes. specifically, we change the personal attribute (figure 1 , male, top-left) to a different attribute (figure 1 , female, bottom-left) and compare the true (equation (1)) and counterfactual attributed biographies (figure 1 , top-right vs., bottom-right, respectively), formulating as: we first investigate whether the counterfactual biographies encode the desired attributes via the same validation described in section 3. 7 and obtain a sufficient amount of data for the analysis, we select nationalities that have a score larger than 0.75 for the analysis based on our empirical experience where similar patterns are observed with different thresholds among different region groups (details in appendix d), resulting in a score of 80.9% ( the semantic matching and sentiment on counterfactual results are shown in figure 3 . we observe similar patterns among the personal attributes of interest. for the sake of brevity, we only show the t-test results about two groups: male and europe. a full pair-wise comparison is listed in appendix h. we first ask to what extent the individual personal attributes affect the generated biographies in semantic matching and sentiment. we compare the results where co-occurring attributes are the same but with different individual personal attributes (figure 3 , bars with different colours but the same shapes). for gender, semantic matching is significantly different when given the same co-occurring attributes but different genders, e.g., given male attribute achieve lower semantic matching scores compared to female attribute, m, co(m) (blue, slash) vs., f, co(m) (red, slash). but we do not observe such in sentiment. we find a significant difference in some region groups in both measurements, e.g., af, co(eu) (red, slash) vs., ap, co(eu) (brown, slash). however, the difference is not consistent among all region attributes. we further investigate the effect of the cooccurring attributes in biography generation. we do so by comparing the biographies given the same individual personal attributes but different cooccurring attributes (figure 3 , bars with different shapes but the same colour). we find a significant difference towards different co-occurring attributes of the gender groups in both semantic matching and sentiment, e.g., m, co(m) (blue, slash) vs., m, co(f) (blue, dot), echoing the finding in section 3. a significant difference is also observed for some regions in sentiment. however, we do not find such a pattern in semantic matching. ## discussion to what extent do personal attributes affect biography content? we answer with a counterfactual methodology, comparing the generated biographies based on manipulating the personal attribute of interest while keeping the co-occurring attributes unchanged. using llms, we disentangle the effect of individual and related co-occurring attributes in biography generation. we utilize a syntheticconstructed biography dataset to mitigate the effect of names and balance the attribute distribution. we find that (1) gender and its co-occurring attributes significantly impact semantic matching and sentiments. generated biographies from male and male-related co-occurring attributes have a higher sentiment score but are less aligned with the given attributes; (2) there is a significant difference in some region groups and their co-occurring attributes in both measurements. yet the pattern is not consistent among the region groups; and, (3) manipulating personal attributes of interest only does not resolve the bias in biography generation as the related co-occurring also significantly impacts results. our study extends bias in text generation (e.g., @xcite al. (2020) 2022 )) research on a new perspective, i.e., data-to-text, and go beyond heavily gendercentered studies. with the controllable setting formulated in a data-to-text framework, we go further from group disparity on the observant text data and explore the causal effect of the individual and its co-occurring attributes. our counterfactual methodology can be extended to other personal attributes, e.g., regard @xcite (appendix f) and religion @xcite , and other evaluation dimensions, e.g., readability @xcite and diversity @xcite . ## ethical discussion our study is based on a synthetic-constructed biography dataset and we analyzed the bias at the group level. our proposed method aims to uncover the bias in biography generation and can be applied to real biographies such as wikipedia biography. however, we do not target nor encourage to target specific individuals or names. we categorize the gender based on the given category from the synthbio dataset. we acknowledge that the category of gender does not represent all identified gender types. particularly, non-binary does not reflect the actual gender identification of the biography. additionally, although our experiment shows evidence of bias in the region, we only consider a selected set of nationalities for each region, i.e., it only partially represents the region. the advanced development of llms allows us to study the counterfactual scenarios of the case. however, llms have been shown to be biased @xcite @xcite . apart from the inherited bias from the wikibio dataset, the usage of the counterfactual method could potentially introduce undetected biases and risks, such as reinforcing stereotypes or perpetuating harmful biases. data generated from such methods should be used with care. for instance, the generated biographies should only be used for bias analysis at the group level. similarly, the data should be only used for augmenting the training data, instead of replacing it, and only to mitigate the bias. we do not encourage the other usages. for copyright, the wikibio dataset is under license cc by-sa 4.0 deed foot_6 and the synthbio dataset in under license apache 2.0. 9 the usage of the flan-t5 model is also under license apache 2.0. ## limitations we use flan-t5 for our experiments. there is room for exploring more advanced llms for biography generations, e.g., llama models @xcite , phi models @xcite , or models curated for the data-to-text task @xcite @xcite for studying whether generated biographies encode provided nationality information, we use a rule-based method, explicitly matching the nationality keywords with the biographies. it could measure the generation quality to some extent (e.g., in appendix e). however, employing a better nationality classifier could further enhance our data filtering process and generation quality. our study requires reference-free evaluators as the counterfactual results do not contain corresponding ground-true text. although data-questeval @xcite has shown to be effective in evaluating semantic matching in the wikibio dataset and our analysis data, synthbio, follow the same structure as wikibio, this evaluator might still introduce undesired harms in comparing the counterfactual performances. similarly, we use a rule-based method to measure the sentiment of the biography, i.e., sentiwords @xcite , which has also shown to be suitable for general use. subtle or contextual changes in sentiment can not be captured by our sentiment evaluator. having human annotation would further enhance the analysis of the bias and the alignment study between automatic evaluations and human annotation would be an interesting further direction in the context of fairness in biography generation. additionally, although we conducted a primarily qualitative analysis on the correlation between the length of generated biographies and evaluation scores (appendix g), further in-depth analysis is needed to understand how the choice of words affects semantic matching and sentiment. in counterfactual data-to-text biography generation, one key factor is to maintain the coherence of the personal attributes. our experiment considers two universal personal attributes and flipping these two attributes generally would not conflict with other attributes. however, to expand our framework to other personal attributes, a careful design of attribute manipulation is needed. one possible solution is to follow the attribute construction process described in the synthbio dataset @xcite , only making a minimal change in the related co-occurring attributes. we use the synthbio dataset for our bias analysis. the synthetic-constructed infobox is carefully created via human-ai collaboration, which provides a balanced distribution covering a limited set of attributes. although it is beneficial as a starting point for analysis bias in data-to-text biography generation, this dataset does not fully capture the complexity and diversity of real-world biographies. the relationship between personal attributes of interest and cooccurring attributes could be expanded. for example, names could strongly influence biography generation in the real world. deepening the understanding of the correlation of attributes is one of the directions to further this work. hugo touvron, louis martin, kevin stone, peter albert, amjad almahairi, yasmine babaei, nikolay bashlykov, soumya batra, prajjwal bhargava, shruti bhosale, et al. 2023. llama 2: open foundation and fine-tuned chat models. arxiv preprint arxiv:2307.09288. francesca tripodi. 2023. ms. categorized: gender, notability, and inequality on wikipedia. new media & society, 25(7):1687-1707. claudia wagner, david garcia, mohsen jadidi, and markus strohmaier. 2015. it's a man's wikipedia? assessing gender inequality in an online encyclopedia. in proceedings of the international aaai conference on web and social media, volume 9, pages 454-463. julia watson, barend beekhuizen, and suzanne stevenson. 2023. what social attitudes about gender does bert encode? leveraging insights from psycholinguistics. in proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pages 6790-6809, toronto, canada. association for computational linguistics. ann yuan, daphne ippolito, vitaly nikolaev, chris callison-burch, andy coenen, and sebastian gehrmann. 2021. synthbio: a case study in humanai collaborative curation of text datasets. in 35th conference on neural information processing systems (neurips 2021) track on datasets and benchmarks.