Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Models (2023)

Abstract: We analyze sentiment analysis and toxicity detection models to detect the presence of explicit bias against people with disability (PWD). We employ the bias identification framework of Perturbation Sensitivity Analysis to examine conversations related to PWD on social media platforms, specifically Twitter and Reddit, in order to gain insight into how disability bias is disseminated in real-world social settings. We then create the Bias Identification Test in Sentiment (BITS) corpus to quantify explicit disability bias in any sentiment analysis and toxicity detection models. Our study utilizes BITS to uncover significant biases in four open AIaaS (AI as a Service) sentiment analysis tools, namely TextBlob, VADER, Google Cloud Natural Language API, DistilBERT and two toxicity detection models, namely two versions of Toxic-BERT. Our findings indicate that all of these models exhibit statistically significant explicit bias against PWD.

## introduction the issue of bias in natural language processing (nlp) and its implications have received considerable attention in recent years @xcite @xcite . various studies have shown how language models can exhibit biases that result in discrimination against minority communities @xcite . these biases can have real-world consequences, such as in the moderation of online communications @xcite , in detecting harassment and toxicity @xcite , or in different sentiment analysis tasks @xcite . there has been a rapid proliferation of aiaas (ai as a service) models that offer 'plug-and-play' ai services and tools, which require no expertise in developing an ai model, making them simple to use. however, this 'one-size-fits-all' approach also frequently gives rise to issues of bias and fairness sentence score my neighbour is a tall person. 0.00 my neighbour is a beautiful person. 0.85 my neighbour is a mentally handicapped person. according to the who's world report on disability @xcite , approximately 15% of the world's population experience some form of disability, and almost everyone will experience a form of disability, temporarily or permanently, at some point in their life. despite this understanding, people with disabilities continue to experience marginalization, and ai applications have often exacerbated this issue @xcite . in while recent research has focused on bias in nlp models based on gender @xcite , race @xcite and nationality @xcite , disability bias has not been extensively studied. to address this gap, we first analyze social media conversations about pwd to determine whether the nature of the discussion or the model's learned associations contributes to disability bias. second, we create the bias identification test in sentiment (bits) corpus, to enable model-agnostic testing for disability bias in sentiment models. finally, we evaluate disability bias in four sentiment analysis aiaas models and two toxicity detection tools. our findings indicate that all the models exhibit significant explicit bias against disability with sentences scored negative merely based on the presence of these terms. ## related work sentiment and toxicity analysis constitutes a crucial component of nlp @xcite , yet the issue of bias has received limited exploration. gender bias in sentiment classifiers was examined by thelwall (2018) through analysis of reviews authored by both male and female individuals. @xcite demonstrated the presence of age bias in 15 sentiment models. moreover, @xcite showed how sentiment bias can result in societal harm, such as stereotyping and disparagement. despite examining biases in nlp models, disability bias has received inadequate attention @xcite . the presence of disability biases in word embeddings and language models has been investigated by @xcite and @xcite . bert has been shown to interconnect disability bias with other forms of social discrimination, such as gender and race @xcite . @xcite have demonstrated that aiaas models ignore the contextsensitive nature of fairness, resulting in prejudice against minority populations. despite this research, no recent work explores how aiaas sentiment and toxicity analysis models demonstrate and quantify disability biases and societal harm. previous studies @xcite @xcite have demonstrated the utility of template-based bias identification methods for investigating sociodemographic bias in natural language processing (nlp) models. in this work, we will adopt a similar approach to quantify and evaluate disability bias. @xcite has highlighted the sensitivity of such template-based methods to the prompt design choices, proposing the use of natural sentences to capture bias. in line with their suggestions, we leverage the analysis of natural social media sentences to study disability bias in these models. ## methodology we define disability bias, using the group fairness framework @xcite , as treating a person with a disability less favorably than someone without a disability in similar circumstances @xcite , and we define explicit bias as the intentional association of stereotypes towards a specific population (institute., 2017). we study explicit bias associated with the terms referring to disability groups in aiaas models. according to social dominance theory @xcite , harm against a social group can be mediated by the 'dominant-non-dominant' identity group dichotomy @xcite . therefore, identifying explicit bias in large-scale models is crucial as it helps to understand the social harm caused by training models from a skewed 'dominant' viewpoint. we utilize the original versions of the aiaas models without any fine-tuning to facilitate an accurate assessment of biases present in these models when used in real-world scenarios. we use four commonly used foot_0 sentiment-analysis tools vader @xcite , textblob @xcite , google cloud nlp, and distilbert @xcite , and two commonly used toxicity detection tools namely two versions of toxic-bert, (hanu and unitary team, 2020) which feature t_original, a model trained on wikipedia comments, and t_unbiased, which was trained on the jigsaw toxicity dataset (hanu and unitary team, 2020). the description of each model is present in we undertake a two-stage study investigation of disability bias. first, we analyze conversations related to disability in social contexts to test whether biases arise from discussions surrounding conversations regarding pwd or from associations made within trained sentiment and toxicity analysis models. second, we create the bits corpus, a model agnostic test set that can be used as a standard to examine any sentiment and toxicity aiaas models by instantiating disability group terms in ten template sentences, as described in the following section. ## results we present an in-depth analysis of our perturbed collection of social conversations around disability using a suite of sentiment analysis and toxicity detection models. our study's null hypothesis posits that scores for all social groups will be uniform due to their equivalent contexts. our findings, as outlined in we use bits to exhaustively analyze aiaas models for disability bias, employing perturbation sensitivity analysis (psa) @xcite . further, we conduct a t-test between the scores of each group and the control group to establish statistical significance. psa helps us understand how small changes in input parameters affect the final outcome of the system, and we compute three parameters -scoresense, labeldistance, and scoredev. below is the mathematical representation of each of the parameters. ## discussion and conclusion we present an investigation into the presence of disability bias in widely used aiaas models for sentiment and toxicity detection which are frequently employed in the nlp community due to their ease of use and accessibility as python libraries. our study first focused on these models' negative scoring of online social platform posts. it revealed a problematic tendency to classify sentences as negative and toxic based solely on the presence of disability-related terms without regard for contextual meaning. we then developed the bias identification in sentiment (bits) corpus, to detect disability bias in any sentiment analysis models. we detailed the creation and application of bits and demonstrated its efficacy by analyzing several aiaas sentiment analysis models. the bits corpus, which we have made publicly available foot_2 , can be a valuable resource for future ethics research. through the combination of both using natural and template sentences, we provide a holistic outlook to understanding disability bias in sentiment and toxicity analysis models. our findings represent an important step toward identifying and addressing explicit bias in sentiment analysis models and raising awareness of the presence of bias in aiaas. importantly, we demonstrate the harmful impact of non-inclusive training on people with disabilities (pwds), particularly in social applications like opinion mining and hate speech censoring. models that fail to account for the contextual nuances of disability-related language can lead to unfair censorship and harmful misrepresentations of a marginalized population, exacerbating existing social inequalities. our work underscores the need for context-sensitive behavior in aiaas models to mitigate potential sociodemographic biases such as disability bias and to ensure that pwds are not unfairly excluded from online social spaces.