A Study of Implicit Bias in Pretrained Language Models against People with Disabilities (2022)

Abstract: Pretrained language models (PLMs) have been shown to exhibit sociodemographic biases, such as against gender and race, raising concerns of downstream biases in language technologies. However, PLMs’ biases against people with disabilities (PWDs) have received little attention, in spite of their potential to cause similar harms. Using perturbation sensitivity analysis, we test an assortment of popular word embedding-based and transformer-based PLMs and show significant biases against PWDs in all of them. The results demonstrate how models trained on large corpora widely favor ableist language.

## introduction recent work on language models show substantial evidence of the presence of sociodemographic biases associated with race and gender @xcite @xcite . such biases result in wrongful associations of text related to minority groups as being negative and toxic @xcite . however, little prior work has focused on the identification and impact of disability bias @xcite . according to a report on disability by the world health organisation (who), approximately one billion people, or 15% of the world's population, experience some form of disability @xcite . research shows that people with disability (pwd) are the largest population group that faces discrimination regularly @xcite . we find various forms of disability biases in ai systems as well, where language involving pwd can often be classified as toxic @xcite or even violent @xcite . corpora used to train large language models often only reflect the 'loudest voices' or the the most dominant viewpoints even if they are not representative of the population @xcite thereby enabling harmful semantic biases @xcite . in this work, we test for the the presence of implicit bias in 11 popularly used, publicly available pretrained word embeddings, namely those in @xcite and @xcite and 2 pretrained language models (plms), namely @xcite and @xcite . we analyse negative association created with words related to people with disability (pwd). understanding this bias is essential as we deploy more models as real-world social solutions @xcite @xcite , such as fighting online abuse @xcite , identifying health indicators from texts @xcite , and understanding group opinions in social platforms @xcite . we use perturbation sensitivity analysis @xcite to quantify bias in sentiment around language discussing pwd. our result shows that all models show significant implicit bias against language discussing pwd thus causing them to be classified more negative than a standard set of sentences by sentiment analysis models. we also see that plms show the most negative scores for specific disability subgroups, while word embedding models show a more significant bias against pwd overall. ## related work prior work shows that large corpora used to train language models primarily represent hegemonic viewpoints and propagate harmful biases against marginalized populations @xcite . prior work identifying bias in nlp models has shown how they can be discriminatory against specific races @xcite and genders @xcite . work analyzing gender bias in word embedding and plms @xcite @xcite shows how vector representations encode misogynistic, outdated, and harmful stereotypes. we see similar results @xcite concerning race and religion, where minority terms such as black, and muslim are associated with hateful phrases. however, disability bias exacerbated by plms has been relatively unexplored @xcite . offences against pwd are one of the most underreported and concealed hate crimes in the present day @xcite . hidden prejudice related to disability have 'hardly changed over a 14-year period and could take more than 200 years to reach zero bias' (rojas, 2022), making it the hardest sociodemography bias to reduce. every experience of pwd is unique and complex @xcite . we cannot simplify the nuanced nature of pwd as it undermines their experience @xcite making it easy to classify them to be out of the expected 'norm'. @xcite show how even 'civil' conversations related to pwd show strong associations with terms such as firearms, homelessness and alcoholism. @xcite demonstrates how bert perpetuates explicit bias against pwd. and venkit and wilson (2021) show how public sentiment and toxicity models show significant explicit bias against terms related to disability. prior work has explored explicit bias alone, and in contrast, we will analyze the implicit bias in nlp and embedding models by studying the associations generated for sentences containing terms related to pwd. ## methodology we define disability bias as when a person with a disability is treated less favourably than a person without the disability in the same or similar circumstances @xcite . we define implicit bias as the attitudes toward people or associating stereotypes with them without conscious knowledge (institute., 2017). in contrast, explicit bias is the intentional association of stereotypes towards a specific population @xcite . it is useful to identify if large-scale models contain implicit bias as it is not easy to catch and can result in subtle damaging conclusions related to minority populations. in the last 14 years, explicit bias related to disability has dropped down to 37%. still, its implicit bias has only been reduced by 3%, making name dataset source embedding gn_3b_300d google news word2vec cc_42b_300d cc_840b_300d common crawl wp_6b_100d wp_6b_200d wp_6b_300d wikipedia wp_6b_50d glove tw_27b_100d tw_27b_200d tw_27b_25d twitter tw_27b_50d ## results and discussion ## conclusion we identify the presence of a challenging form of bias in language associated with people with disability (pwd): implicit bias in language models. the analysis demonstrates bias in both embeddings and plms for words used in conversations related to pwd. the results show that even when disability is not discussed explicitly, word embeddings and plms consistently score sentences with words associated (in the pretrained vector space) with disability more negatively than sentences containing words with no association to pwd. the results suggest that these large models are inadequate in understanding the nuances of language associated to conversations around disability. pwd community are more likely to talk about disability and biased models can affect free speech and participation of this marginalized community in online social spaces because of unfair censorship catalyzing harmful ableist ideologies and misrepresenting an already marginalized population. we, through this paper, intend to show these use-cases where these models fail so that developers and owners of these models can be more aware of the potential consequence they can have as a solution to social problems. ben hutchinson, vinodkumar prabhakaran, emily denton, kellie webster, yu zhong, and stephen denuyl. 2020. social biases in nlp models as barriers for persons with disabilities. in proceedings of the 58th annual meeting of the association for computational linguistics, pages 5491-5501. clayton hutto and eric gilbert. 2014. vader: a parsimonious rule-based model for sentiment analysis of social media text. in proceedings of the international aaai conference on web and social media, volume 8, pages 216-225. perception institute. 2017. implicit bias explained. christian karmen, robert c hsiung, and thomas wetter. 2015. screening internet forum participants for depression symptoms by assembling and enhancing multiple nlp methods. computer methods and programs in biomedicine, 120(1):27-36. brendan kennedy, xisen jin, aida mostafazadeh davani, morteza dehghani, and xiang ren. 2020. contextualizing hate speech classifiers with post-hoc explanation. in proceedings of the 58th annual meeting of the association for computational linguistics, pages 5435-5442. jacob devlin ming-wei chang kenton and lee kristina toutanova. 2019. bert: pre-training of deep bidirectional transformers for language understanding. in proceedings of naacl-hlt, pages 4171-4186. alfred lee; benjamin kinsella, alfred lee, and benjamin kinsella. 2020. how the social sector can use natural language processing (ssir). keita kurita, nidhi vyas, ayush pareek, alan w black, and yulia tsvetkov. 2019. measuring bias in contextualized word representations. in proceedings of the first workshop on gender bias in natural language processing, pages 166-172. stephen j macdonald, catherine donovan, and john clayton. 2017. the disability bias: understanding the context of hate in comparison with other minority populations. disability & society, 32(4):483-499. tomáš mikolov, édouard grave, piotr bojanowski, christian puhrsch, and armand joulin. 2018. advances in pre-training distributed word representations. in proceedings of the eleventh international conference on language resources and evaluation (lrec 2018). marzieh mozafari, reza farahbakhsh, and noël crespi. 2020. hate speech detection and racial bias mitigation in social media based on bert model. plos one, 15(8):e0237861. ncdj. 2021. national center on disability and journalism. cathy o'neil. 2016. weapons of math destruction: how big data increases inequality and threatens democracy. broadway books. nedjma djouhra ousidhoum, xinran zhao, tianqing fang, yangqiu song, and dit yan yeung. 2021. probing toxic content in large pre-trained language models. in proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing. alexander pak and patrick paroubek. 2010. twitter as a corpus for sentiment analysis and opinion mining. in proceedings of the seventh international conference on language resources and evaluation (lrec'10). ji ho park, jamin shin, and pascale fung. 2018. reducing gender bias in abusive language detection. in proceedings of the 2018 conference on empirical methods in natural language processing, pages 2799-2804. jeffrey pennington, richard socher, and christopher d manning. 2014. glove: global vectors for word representation. in proceedings of the 2014 conference on empirical methods in natural language processing (emnlp), pages 1532-1543. vinodkumar prabhakaran, ben hutchinson, and margaret mitchell. 2019. perturbation sensitivity analysis to detect unintended model biases. in proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp), pages 5740-5745. alec radford, jeffrey wu, rewon child, david luan, dario amodei, ilya sutskever, et al. 2019. language models are unsupervised multitask learners. openai blog, 1(8):9. nikki rojas. 2022. why disability bias is a particularly stubborn problem. yi chern tan and l elisa celis. 2019. assessing social and intersectional biases in contextualized word representations. advances in neural information processing systems, 32. shari trewin. 2018. ai fairness for people with disabilities: point of view. arxiv preprint arxiv:1811.10670. pranav narayanan venkit and shomir wilson. 2021. identification of bias against people with disabilities in sentiment analysis and toxicity detection models. arxiv preprint arxiv:2111.13259. anna cavender university of washington, anna cavender, university of washington, university of wash-ingtonview profile, shari trewin ibm t. j. watson research center, shari trewin, ibm t. j. watson research center, ibm t. j. watson research cen-terview profile, vicki hanson ibm t. j. watson research center, vicki hanson, and et al. 2008. general writing guidelines for technology and people with disabilities. meredith whittaker, meryl alper, cynthia l bennett, sara hendren, liz kaziunas, mara mills, meredith ringel morris, joy rankin, emily rogers, marcel salas, et al. 2019. disability, bias, and ai. ai now institute. worldbank. 2015. individuals using the internet (% of population) -united states.