On the Influence of Gender and Race in Romantic Relationship Prediction from Large Language Models (2024)

Abstract: We study the presence of heteronormative biases and prejudice against interracial romantic relationships in large language models by performing controlled name-replacement experiments for the task of relationship prediction. We show that models are less likely to predict romantic relationships for (a) same-gender character pairs than different-gender pairs; and (b) intra/inter-racial character pairs involving Asian names as compared to Black, Hispanic, or White names. We examine the contextualized embeddings of first names and find that gender for Asian names is less discernible than non-Asian names. We discuss the social implications of our findings, underlining the need to prioritize the development of inclusive and equitable technology.

## introduction identifying romantic relationships from a given dialogue presents a challenging task in natural language understanding @xcite . the perceived gender, race, or ethnicity of the speakers, often inferred from their names, may inadvertently lead a model to predict a relationship type that conforms to conventional societal views. we hypothesize that, when predicting romantic relationships, models may mirror heteronormative biases @xcite and prejudice against interracial romantic relationships @xcite present in humans and society. heteronormative biases assume and favor traditional gender roles, heterosexual relationships, and nuclear families, often marginalizing other gender expressions, sexuality, and family dynamics. in the us, legal protections for interracial and gay marriages were not achieved nationwide until 1967 and 2015, respectively. these relationships continue to face prejudice and discrimination in the present days @xcite @xcite @xcite . in this paper, we consider the task of predicting romantic relationships from dialogues in movie scripts to study whether llms make such predictions based on the demographic attributes associated with a pair of character names, in ways that reflect heteronormative biases and prejudice against interracial romantic relationships. for instance, figure 1 shows a conversation between a female and a male spouse pair, for which llama2-7b predicts a romantic relationship when the names in the conversation are replaced with a pair of differentgender names, but predicts a non-romantic relationship when replaced by same-gender names. ideally, name-replacement should not significantly alter the predictions of a fair and robust model, as the utterance content plays a more substantial role in language understanding, despite the potential interdependence between utterances and original names. different predictions suggest that a model may be prone to overlooking romantic relationships that diverge from societal norms, thus raising ethical concerns. such behavior would indicate that language models inadequately represent certain societal groups @xcite , potentially exacerbating stigma surrounding relation-ships @xcite and sidelining underrepresented groups @xcite . through controlled character name-replacement experiments, we find that relationships between (a) same-gender character pairs; and (b) intra/interracial character pairs involving asian names are less likely to be predicted as romantic. these findings reveal how some llms may stereotypically interpret interactions between people, potentially reducing the recognition of non-mainstream relationship types. while prior work studies gender and racial biases by identifying stereotypical attributes of individuals @xcite @xcite , this paper investigates the role of gender and race in llms' inferences about relationships between two individuals using a relationship prediction dataset @xcite . ## experimental setup we define the following task. given a conversation c which consists of a sequence of turns ((s 1 , u 1 ), (s 2 , u 2 ), . . . , (s n , u n )) between characters a and b, where s @xmath0, s b } indicates that the speaker of an utterance (u i , @xmath1: n}) is either a or b, the task is to identify the relationship represented as a categorical label from a pre-defined set. we carry out controlled namereplacement experiments by prompting llms (zero-shot) to predict the relationship type between a and b given c. ## analysis and discussion we perform additional experiments to understand the observed model behavior. why does a model tend to predict fewer romantic relationships for racial pairings that involve asian names? although we select names for each race that have strong real-world statistical associations with one gender, we hypothesize that low recall on pairs with one or more asian names may be due to model's inability to discern gender from asian names. to test this hypothesis, we retrieve the contextualized embeddings from llama2-7b for each first name (collected in §2.2) occurrence in 15 romantic and 15 non-romantic random dialogues. we obtain 209, 800 embeddings, which are used to train logistic regression models that classify the gender or race associated with a name (details in §a). as we compare the average classification accuracy (across 5 different train-test splits) against a majority baseline, we observe, in ## social implications it has been a prolonged and arduous struggle to recognize and accept gay marriages in the us @xcite . legal recognition of these relationships remains a challenge in many other countries @xcite @xcite . even within the us, lgbtqia+ people still encounter discrimination (buist, 2019; knauer, 2020; naylor, 2020). we believe heteronormative biases we have observed could impact various downstream llm use cases, potentially causing both representational and allocational harms @xcite . for example, when llms are used for story generation based on social media posts as the premise @xcite , the life events of members of the lgbtqia+ community may be overlooked or misrepresented. if llms struggle to recognize same-gender romantic relationships, they may further marginalize the lgbtqia+ community by diminishing their social visibility and representation. in addition, such model behavior may result in uneven allocation of resources or opportunities. consider an online advertising system that promotes low-interest home loans for married couples based on social media interactions. a model unable to identify same-gender marriages would exclude these couples from the promotion. therefore, building inclusive technology that respects minority rights is essential. ## related work prior works @xcite @xcite @xcite @xcite show that language models often treat first names differently, even with controlled input contexts, due to factors like frequency and demographic attributes associated with names @xcite @xcite @xcite . our work uses models' interpretations of gender associated with first names to reveal heteronormative biases in some llms. further, nlp systems often fail in interpreting various social factors (e.g., social norms, cultures, and relations) of language (hovy and yang, 2021). one such factor of interest is the representation of social relationships in these systems, including power dynamics @xcite , friendship (krishnan and eisenstein, 2015), and romantic relationships @xcite . recently, stewart and mihalcea (2024) show failure of popular machine translation systems in translating sentences concerning relationships between nouns of samegender. leveraging the task of relationship prediction and using an existing dataset @xcite , our work contributes to the assessment of social relationship-related biases in llms arising from gender and race associations with first names. ## conclusion through controlled name-replacement experiments, we find that llms predict romantic relationships between characters based on the demographic identities associated with their first names. specifically, relationship predictions between samegender and intra/inter-racial character pairs involving asian names are less likely to be romantic. our analysis of contextualized name embeddings sheds light on the cause of our findings. we also highlight the social implications of this potentially harmful model behavior for the lgbtqia+ community. we urge advocates to build technology that respects the rights of marginalized social groups.