Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Models
Year: 2023

Abstract:
We analyze sentiment analysis and toxicity detection models to detect the presence of explicit bias against people with disability (PWD). We employ the bias identification framework of Perturbation Sensitivity Analysis to examine conversations related to PWD on social media platforms, specifically Twitter and Reddit, in order to gain insight into how disability bias is disseminated in real-world social settings. We then create the Bias Identification Test in Sentiment (BITS) corpus to quantify explicit disability bias in any sentiment analysis and toxicity detection models. Our study utilizes BITS to uncover significant biases in four open AIaaS (AI as a Service) sentiment analysis tools, namely TextBlob, VADER, Google Cloud Natural Language API, DistilBERT and two toxicity detection models, namely two versions of Toxic-BERT. Our findings indicate that all of these models exhibit statistically significant explicit bias against PWD.

heading: 1 Introduction
text: The issue of bias in natural language processing (NLP) and its implications have received considerable attention in recent years (Bolukbasi et al., 2016; Kiritchenko and Mohammad, 2018; Caliskan et al., 2017). Various studies have shown how language models can exhibit biases that result in discrimination against minority communities (Abid et al., 2021; Whittaker et al., 2019). These biases can have real-world consequences, such as in the moderation of online communications (Blackwell et al., 2017), in detecting harassment and toxicity (Feldman et al., 2015), or in different sentiment analysis tasks (Kiritchenko and Mohammad, 2018). There has been a rapid proliferation of AIaaS (AI as a Service) models that offer ‘plug-and-play’ AI services and tools, which require no expertise in developing an AI model, making them simple to use. However, this ‘one-size-fits-all’ approach also frequently gives rise to issues of bias and fairness
(Lewicki et al., 2023). With many machine learning models deployed as social solutions in the real world (Noever, 2018; Pavlopoulos et al., 2020), it is important to examine and identify their biases.
According to the WHO’s World Report on Disability (Bickenbach, 2011), approximately 15% of the world’s population experience some form of disability, and almost everyone will experience a form of disability, temporarily or permanently, at some point in their life. Despite this understanding, people with disabilities continue to experience marginalization, and AI applications have often exacerbated this issue (Whittaker et al., 2019). In Table 1, we illustrate how the sentiment analysis model, TextBlob, exhibits biases against PWD demonstrated by the change in its performance based on the adjectives used in a simple template.
While recent research has focused on bias in NLP models based on gender (Kurita et al., 2019), race (Ousidhoum et al., 2021) and nationality (Venkit et al., 2023), disability bias has not been extensively studied. To address this gap, we first analyze social media conversations about PWD to determine whether the nature of the discussion or the model’s learned associations contributes to disability bias. Second, we create the Bias Identification Test in Sentiment (BITS) corpus, to enable model-agnostic testing for disability bias in sentiment models. Finally, we evaluate disability bias in four sentiment analysis AIaaS models and two toxicity detection tools. Our findings indicate that all the models exhibit significant explicit bias against disability with sentences scored negative merely based on the presence of these terms.
26

heading: 2 Related Work
text: Sentiment and toxicity analysis constitutes a crucial component of NLP (Medhat et al., 2014), yet the issue of bias has received limited exploration. Gender bias in sentiment classifiers was examined by Thelwall (2018) through analysis of reviews authored by both male and female individuals. Díaz et al. (2018) demonstrated the presence of age bias in 15 sentiment models. Moreover, Dev et al. (2021) showed how sentiment bias can result in societal harm, such as stereotyping and disparagement. Despite examining biases in NLP models, disability bias has received inadequate attention (Whittaker et al., 2019). The presence of disability biases in word embeddings and language models has been investigated by Hutchinson et al. (2020) and Venkit et al. (2022). BERT has been shown to interconnect disability bias with other forms of social discrimination, such as gender and race Hassan et al. (2021). Lewicki et al. (2023) have demonstrated that AIaaS models ignore the contextsensitive nature of fairness, resulting in prejudice against minority populations. Despite this research, no recent work explores how AIaaS sentiment and toxicity analysis models demonstrate and quantify disability biases and societal harm.
Previous studies (Kiritchenko and Mohammad, 2018; Nangia et al., 2020; Nadeem et al., 2020; Prabhakaran et al., 2021) have demonstrated the utility of template-based bias identification methods for investigating sociodemographic bias in natural language processing (NLP) models. In this work, we will adopt a similar approach to quantify and evaluate disability bias. Alnegheimish et al. (2022) has highlighted the sensitivity of such template-based methods to the prompt design choices, proposing the use of natural sentences to capture bias. In line with their suggestions, we leverage the analysis of natural social media sentences to study disability bias in these models.

heading: 3 Methodology
text: We define disability bias, using the group fairness framework (Czarnowska et al., 2021), as treating a person with a disability less favorably than someone without a disability in similar circumstances (Commission, 2012), and we define explicit bias as the intentional association of stereotypes towards a specific population (Institute., 2017). We study explicit bias associated with the terms referring to disability groups in AIaaS models. According to
Social Dominance Theory (Sidanius and Pratto, 2001), harm against a social group can be mediated by the ‘dominant-non-dominant’ identity group dichotomy (Dev et al., 2021). Therefore, identifying explicit bias in large-scale models is crucial as it helps to understand the social harm caused by training models from a skewed ‘dominant’ viewpoint. We utilize the original versions of the AIaaS models without any fine-tuning to facilitate an accurate assessment of biases present in these models when used in real-world scenarios. We use four commonly used3 sentiment-analysis tools VADER (Gilbert and Hutto, 2014), TextBlob (Loria, 2018), Google Cloud NLP, and DistilBERT (Sanh et al., 2019), and two commonly used toxicity detection tools namely two versions of ToxicBERT, (Hanu and Unitary team, 2020) which feature T_Original, a model trained on Wikipedia comments, and T_Unbiased, which was trained on the Jigsaw Toxicity dataset (Hanu and Unitary team, 2020). The description of each model is present in Table 2.
We undertake a two-stage study investigation of disability bias. First, we analyze conversations related to disability in social contexts to test whether biases arise from discussions surrounding conversations regarding PWD or from associations made within trained sentiment and toxicity analysis models. Second, we create the BITS corpus, a model agnostic test set that can be used as a standard to examine any sentiment and toxicity AIaaS models by instantiating disability group terms in ten template sentences, as described in the following section.

heading: 3.1 Social Conversations Around Disability
text: We examine the potential presence of bias in realtime social conversations related to PWD on two major social media platforms, Reddit and Twitter. Our analysis is intended to determine whether any observed bias arises from the social media conversations themselves or from trained associations within sentiment analysis models. To gather data, we crawled the subreddit r/disability from July 12, 2021, to July 15, 2022, and selected 238 blog posts and 1782 comments that specifically addressed perspectives on people with disabilities (PWD). Similarly, we used the Twitter API to collect 13,454 tweets between July 9, 2021, and July 16, 2022, containing the terms or hashtags ‘disability’ or ‘disabled’. We then manually filtered out any discus-
3based on high citation and download counts
sions that only tangentially addressed disability, following selection criteria similar to those of Díaz et al. (2018).
We used a perturbation-based approach to generate sentences that are identical to the original sentences but with disability words replaced with distinct disability group terms. Specifically, we perturbed the words ‘disability’ and ‘disabled’ by replacing them with words from four distinct groups (Table 4). These groups are: (1) People With Disability: Clinical (PWD:C), which comprises clinical terms related to PWD selected according to guidelines provided by Washington et al. (2008); NCDJ. (2021), and CDC’s National Center on Birth Defects and Developmental Disabilities4; (2) PWD: Social Discourse (PWD:SD), which consists of the most common terms used in social discourse re-
4https://www.cdc.gov/ncbddd/sitemap.html
lated to disability, identified through discourse analysis of 2,000 randomly sampled tweets; (3) People without Disability (PWoD), which contains politically correct terms related to the population without disabilities (NCDJ., 2021; Washington et al., 2008); and (4) Normalized Adjectives (NRMA), which comprises the top 5 adjectives commonly used in English (Davies, 2010). Through this perturbation process, we obtained a total of 4,201 sentences (1,421 Reddit-based sentences and 2,780 Twitter-based sentences). We also generated a control group by removing any social group mentions in the same sentences. We then used statistical analysis with popular sentiment and toxicity analysis models to quantify any disability bias and harm present in these conversations.

heading: 3.2 Sentiment and Toxicity Analysis Models
text: We create the Bias Identification Test in Sentiment (BITS) corpus as a general purpose model agnostic approach to check for explicit disability bias in any sentiment and toxicity analysis model. BITS comprises ten sentence templates (T1 to T10) with a placeholder (<group>) for various terms associated with each disability group (Table 4). We divide the templates into two groups, namely neutral and sentiment-holding, motivated by the work of Kiritchenko and Mohammad (2018). The sentiment-
holding templates contain an emotion or an event word, which we instantiate based on eight primary emotions (Ekman, 1993) (Table 3), to convey varying degrees of the same sentiment.
We also generate a control group of 420 sentences without any <group> words. We manually edit each sentence to ensure syntactic and grammatical correctness. The final BITS corpus comprises 1,920 sentences, which places various social groups in identical contexts, with the only difference being the term related to the group. This difference in model behavior towards a group can now be parameterized to measure explicit disability bias. We use perturbation sensitivity analysis (Prabhakaran et al., 2019) on popular sentiment and toxicity analysis AIaaS models to compare and quantify the biases between social groups.

heading: 4 Results
text: We present an in-depth analysis of our perturbed collection of social conversations around disability using a suite of sentiment analysis and toxicity detection models. Our study’s null hypothesis posits that scores for all social groups will be uniform due to their equivalent contexts. Our findings, as outlined in Table 6, demonstrate that PWoD and NRM groups generate neutral scores. Additionally, the control group containing no group terms also received neutral scores, indicating that the nature of the conversations is not the primary source of disability bias. Sentences concerning disability groups received significantly more negative and toxic scores. Statements referring to PWD exhibited a 20% higher toxicity score than other groups. By performing a t-test between the control group and individual social groups (Table 6),
we can reject our null hypothesis. Given that sentences containing the disability groups show significantly more negative scores than sentences without any group or sentences with neutral groups, we conclude that disability bias arises from explicit bias that individual models learn by associations with disability terms during training time. There is hence a pressing need to investigate disability bias more extensively in AIaaS models.
We use BITS to exhaustively analyze AIaaS models for disability bias, employing Perturbation Sensitivity Analysis (PSA) (Prabhakaran et al., 2019). Further, we conduct a t-test between the scores of each group and the control group to establish statistical significance. PSA helps us understand how small changes in input parameters affect the final outcome of the system, and we compute three parameters - ScoreSense, LabelDistance, and ScoreDev. Below is the mathematical representation of each of the parameters.
Perturbation Score Sensitivity (ScoreSense): The average difference between the results generated by the corpus X through a selected social group f(xn) and the results generated by the corpus without any mention of the social group f(x) is defined as ScoreSense of model f. ScoreSense =∑ x∈X
[f(xn)− f(x)] Perturbation Score Deviation (ScoreDev): The standard deviation of scores of a given model f with a corpus X is the mean standard deviation of the scores acquired my passing all sentences xn, of all every group N in consideration. ScoreDev =∑ x∈X [σn∈N (f(xn))]
Perturbation Label Distance (LabelDist): The Jaccard Distance for a set of sentence where f(x) = 1 and f(xn) = 1, averaged for all terms n in a social group N is the LabelDist of the model. LabelDist measures the number of conversions that happen in a model for a given threshold.
LabelDist = ∑ n∈N [Jaccard(x|y(x) = 1, x|y(xn) = 1)]], where Jaccard(A|B) = 1− |A ∩B|/|A ∪B|
Table 7 shows the ScoreSense values for all the selected models and identified groups. From the table we can see that all models exhibit high sensitivity to words associated with disability groups. Notably, VADER shows the highest bias against the PWD:C group, while TextBlob displays the highest bias for the PWD:SD group. The mere addition of PWD:C and PWD:SD terms results in a dip of -0.25 and -0.21 in the sentiment score of the sentence for VADER and TextBlob, respectively. Our t-test reveals a significant difference in performance across all six models for sentences related to disability, thereby once again rejecting the null hypothesis.
Table 8 shows the LabelDistance and ScoreDev values for all the models and PWD:SD and PWD:C groups. LabelDistance measures the Jaccard distance between the sentiments of the set of sentences before and after perturbation. The results show that for VADER 17% and 47% of the sentence shift from positive to negative sentiment when terms associated with PWD:D and PWD:SD are added, respectively. The high LabelDistance values reveals that there is a significantly decrease in sentiment when disability-related terms are added, demonstrating explicit bias against PWD in all models. Finally, ScoreDev measures the standard deviation of scores due to perturbation, averaged across all groups, further showcasing the degree of polarity in the scores generated for each model. Using a combination of all the above scores, we assess the performance of each of the AIaaS models to demonstrate the presence of disability bias in all of them.

heading: 5 Discussion and Conclusion
text: We present an investigation into the presence of disability bias in widely used AIaaS models for sentiment and toxicity detection which are frequently employed in the NLP community due to their ease of use and accessibility as Python libraries. Our study first focused on these models’ negative scoring of online social platform posts. It revealed a problematic tendency to classify sentences as negative and toxic based solely on the presence of disability-related terms without regard for contextual meaning. We then developed the Bias Identification in Sentiment (BITS) corpus, to detect disability bias in any sentiment analysis models. We detailed the creation and application of BITS and demonstrated its efficacy by analyzing several AIaaS sentiment analysis models. The BITS Corpus, which we have made publicly available5, can be a valuable resource for future ethics research. Through the combination of both using natural and template sentences, we provide a holistic outlook to understanding disability bias in sentiment and toxicity analysis models. Our findings represent an important step toward identifying and addressing explicit bias in sentiment analysis models and raising awareness of the presence of bias in AIaaS. Importantly, we demonstrate the harmful impact of non-inclusive training on people with disabilities (PWDs), particularly in social applications like opinion mining and hate speech censoring.
Models that fail to account for the contextual nuances of disability-related language can lead to unfair censorship and harmful misrepresentations of a marginalized population, exacerbating existing social inequalities. Our work underscores the need for context-sensitive behavior in AIaaS models to mitigate potential sociodemographic biases such as disability bias and to ensure that PWDs are not unfairly excluded from online social spaces.
5https://github.com/PranavNV/BITS
Limitations
Through our work, we analyze various sentiment and toxicity analysis models to determine if they show an ableist viewpoint. The results depict a statically significant presence of disability bias, and we publish our method for any individual to access and use. This step is crucial in the field of NLP to mention the ramifications a given model can have on society. One limitation of this work is that we analyze models that are trained in the English language. We understand that the social concept of disability can change for various cultures and languages. The scope of this paper for now only looks into one language.
Ethical Statement
The paper provides a method to parameterize ableist bias in NLP models, but we acknowledge that this is not the sole method that can be used for identification. The work is limited only to identification in sentiment analysis and toxicity detection models. There can be other methods of identification that are rapidly being worked on which may not have been included in this process. We also understand the effects various other forms of social biases can have when viewed alongside disability bias. We, therefore, will be working on measuring the combination of social biases through a cultural lens for the future.

heading: A Appendix
text: In this section, we have included supplementary exploration to the selected models to provide more insight on their behaviour in exhibiting potential disability bias.

heading: Future Work