Investigating Ableism in LLM s through Multi-turn Conversation (2024)

Abstract: To reveal ableism (i.e., bias against persons with disabilities) in large language models (LLMs), we introduce a novel approach involving multi-turn conversations, enabling a comparative assessment. Initially, we prompt the LLM to elaborate short biographies, followed by a request to incorporate information about a disability. Finally, we employ several methods to identify the top words that distinguish the disability-integrated biographies from those without. This comparative setting helps us uncover how LLMs handle disability-related information and reveal underlying biases. We observe that LLMs tend to highlight disabilities in a manner that can be perceived as patronizing or as implying that overcoming challenges is unexpected due to the disability.

## introduction social bias persists in large language models (llms), as highlighted by previous research @xcite , and the implications of these biased representations grow more concerning as llms become increasingly prevalent. recent studies @xcite have assessed bias in llms by prompting them to generate personas, stories, or reference letters. however, most existing research has primarily focused on biases related to gender and race, leaving biases against different disability groups relatively underexplored. additionally, these studies often rely on single prompts, overlooking the potential of multi-turn conversations that could reveal implicit biases through comparative assessment. in our study, we propose investigating bias against persons with disabilities (pwd) using a multi-turn conversation approach with llms. as illustrated in figure 1 , our method begins by prompting an llm to elaborate a short biography. we then prompt the llm to incorporate information about the person's disability into this biography. this figure 1 : an instance of a multi-turn conversation with an llm involves initially requesting it to expand on a brief biography, followed by instructing it to incorporate disability-related information into the expanded biography. highlighted words signify the integrated information. upon comparing the marked and unmarked biographies, we observe that terms related to pwd are incorporated into the marked versions. we notice a potential bias where the disability is emphasized in a manner suggesting that overcoming challenges is exceptional due to the disability. this inadvertently reinforces negative stereotypes and may diminish the individual's accomplishments by primarily framing them in terms of overcoming obstacles. additionally, during manual examination, we frequently encounter instances where pwd are portrayed as sources of inspiration. ## related work in this section, we discuss ableism identified in prior studies from various perspectives. ## experiments in this section, we explain the process of generating biographies and the method used to identify the top words. ## results in this section, we analyze the top words identified by marked words, svm, and jsd. additionally, we perform sentiment analysis on the generated biographies to approximate whether the biographies for pwd are perceived as more motivational or inspirational compared to those for the unmarked group. ## discussion the investigation of ableism in large language models (llms) reveals the pervasive and often unnoticed biases ingrained in ai systems. as llms become increasingly integrated into various aspects of society-from content moderation to virtual assistants-the presence of biases against pwd can reinforce harmful stereotypes. we aim to raise awareness of often overlooked biases, such as society's tendency to view pwd as sources of inspiration. this perspective, while seemingly positive, can be patronizing and reduce individuals to their disabilities. by addressing these subtle biases, we can contribute to a more equitable and just representation of pwd in ai systems, ultimately fostering a broader societal shift towards inclusivity and respect for all individuals. ## conclusion in this paper, we have investigated ableism in llms through multi-turn conversations, allowing for the identification of bias via direct comparative evaluations. by identifying significant words that differentiate marked biographies from unmarked ones, we have detected potential biases against pwd. these biases often emphasize disabilities in a way that implies overcoming challenges is exceptional due to the disability, inadvertently reinforcing negative stereotypes.