Anthropomorphization of AI : Opportunities and Risks (2023)

Abstract: Anthropomorphization is the tendency to attribute human-like traits to non-human entities. It is prevalent in many social contexts – children anthropomorphize toys, adults do so with brands, and it is a literary device. It is also a versatile tool in science, with behavioral psychology and evolutionary biology meticulously documenting its consequences. With widespread adoption of AI systems, and the push from stakeholders to make it human-like through alignment techniques, human voice, and pictorial avatars, the tendency for users to anthropomorphize it increases significantly. We take a dyadic approach to understanding this phenomenon with large language models (LLMs) by studying (1) the objective legal implications, as analyzed through the lens of the recent blueprint of AI bill of rights and the (2) subtle psychological aspects customization and anthropomorphization. We find that anthropomorphized LLMs customized for different user bases violate multiple provisions in the legislative blueprint. In addition, we point out that anthropomorphization of LLMs affects the influence they can have on their users, thus having the potential to fundamentally change the nature of human-AI interaction, with potential for manipulation and negative influence. With LLMs being hyper-personalized for vulnerable groups like children and patients among others, our work is a timely and important contribution. We propose a conservative strategy for the cautious use of anthropomorphization to improve trustworthiness of AI systems.

## introduction anthropomorphization refers to ascribing humanlike traits to non-human entities, and has been used in diverse areas encompassing literature, science, art, and marketing @xcite @xcite . it occurs when humans assign emotional or behavioral traits to entities. several behavioral psychology studies have posited and argued that anthropomorphization is a 1 whitehouse.gov/ostp/ai-bill-of-rights/ figure 1 : conversational ai systems are increasingly being integrated into the daily lives of many. while their improved quality is a welcome change, their personalization increases the tendency to anthropomorphize them, which has legal and psychological risks. recently, generative large language models (llms) @xcite have been deployed in a variety of applications. conversational systems like chatgpt (ope-nai, 2023) and bard @xcite have modified llms with a purposeful push towards making them more human-like @xcite . the quality of these systems has enabled human-ai interactions at unprecedented scales, thus increasing the chances of these systems being anthropomorphized. in this work, we analyze anthropomorphization in llms and discuss its: (1) legal implications and (2) psychological effects. customization of systems and brands has long been seen as an effective way to increase anthropomorphization and establish an emotional connection with humans @xcite . thus, although not strictly interchangeable, we refer to customized and personalized llms as anthropomorphized llms. we analyze results from prior work @xcite and find that anthropomorphized llms violate at least two legislative principles penned in "blueprint for an ai bill of rights" (ostp, 2022) released by the white house: (1) algorithmic discrimination protections and (2) safe and effective systems. for example, @xcite 's results shows that customized chatgpt targets certain demographics more than others. furthermore, the safety of the system depends on the kind of persona used to customize llms, leading to second-order discriminatory patterns. we also analyze the concept of corporate personhood for powerful ai systems, since they have potential to be large-scale decision making agents. given that different personas assigned to the same ai system lead to varied behavior, we urge legal experts to consider if personhood should be applied at a persona-level, a model-level, or a firm-level. we also discuss the psychological effects by understanding how important factors like trustworthiness, explainability, and transparency are affected by anthropomorphization. several marketing and consumer behavior studies have found that selfcongruence, which is the degree to which a system matches a consumer's self-image, can influence a user's behavior significantly @xcite . given the ease with which the fine-grained personality of conversational systems can be manipulated (figure 2 ), malicious actors can use it to exploit users by creating a false sense of attachment. an example of this is a chatbot built for school children or teenagers which influences them to buy certain products. despite these vulnerabilities anthropomorphization has advantages if used responsibly. studies have shown that it can be used to improve trust in systems @xcite . given the increasing adoption of ai systems in the real world, anthropomorphization is a powerful tool to improve accessibility of these systems, but both creators and users should be educated about its consequences. in this paper we argue for conservative and responsible use of this subtle and powerful tool while being cautious about outright anthropodenial. ## anthropomorphization in llms large language models (llms) are a class of neural networks trained on large amounts of text data to learn the probabilistic structure of language. historically, llms have been deployed in task-specific contexts like text classification @xcite . recently however, the performance of conversational llms (e.g., chatgpt and bard) have rendered them useful to interact with humans in a variety of contexts. given their conversation ability, several companies and products have started to use llms for personalization. this is easy and enables model behavior modification by simply changing the system parameters of the model's api, as shown in figure 2 . for example, snapchat's my ai uses ope-nai's apis underneath. customization has been long seen as a way to establish self-congruence with users and increases the chances of users anthropomorphizing the systems @xcite @xcite . for example, telling a chatbot to "talk like a doctor" allows it to impersonate a doctor, which anthropomorphizes the model to a larger degree than the original general system. in this work, we refer to customized llms as anthropomorphized llms and highlight such publicly available systems in ## legal aspects of anthropomorphization we discuss the legal aspects in the context of the "blueprint for an ai bill of rights" (ostp, 2022) which was released by the white house in october 2022. the blueprint lays down a set of five principles that should be followed by ai systems and we focus specifically on algorithmic discrimination protections, which says that "you should not face discrimination by algorithms and systems should be used and designed in an equitable way." ## psychological implications another principle mentioned in the ai bill of rights is safe and effective systems: "you should be protected from unsafe or ineffective systems. automated systems should be developed with consultation from diverse communities, stakeholders, and domain experts to identify concerns, risks, and potential impacts of the system." we believe that anthropomorphization can have subtle psychological effects on the users, and we discuss them by comparing it with other technologies and spheres. several behavioral psychological studies have posited and argued that anthropomorphization is a natural tendency in humans @xcite , with others further suggesting that anthropomorphization is grounded in interaction @xcite . with conversational systems getting more useful, interaction between humans and these systems, and thus the tendency to anthropomorphize is only going to increase, which has the potential to influence and manipulate. analyses have shown that anthropomorphization of ai systems has changed the behavior of users significantly @xcite @xcite @xcite . most interestingly, @xcite establish a conceptual link between anthropomorphization and self-congruence, which is the fit between the user's self-concept and the system's personality. this is of extreme importance because self-congruence increases the trust that a user has on the system @xcite @xcite . this is a concept that is extensively studied in marketing and consumer behavior and studies have shown that it can influence behaviors such as willingness to pay @xcite , customer satisfaction @xcite , and trustworthiness @xcite . the exact demographics of the personality associated with systems or brands plays a key role in self-congruence as well, with studies finding that the demographics of the logo or mascot associated with the brand like the gender @xcite or race @xcite have significant impact on self-congruence. given that current llms are powerful enough to be bestowed specific demographic traits, malicious actors can easily use this to their advantage by manipulating users into trusting the system. frequent interaction with personalized llms also has the potential of creating an echochamber @xcite , with seemingly benign "personalized" generations about sensitive topics like health, one's looks, their mental health leading people to have wrong assumptions about themselves @xcite . ## conclusion while anthropomorphization has issues, it also poses a suite of opportunities to improve accessibility of ai systems. in the past, it has been used as a tool to improve trustworthiness and acceptability of ai systems @xcite @xcite . with more ai systems being deployed in the real world, accessibility can be improved with constructive anthropomorphization encompassing features like using the native language of the user, assigning virtual personas which are familiar to the user, and making them more relatable and empathetic. however, it is our responsibility to find the balance between anthropodenial and unfettered anthropomorphization.