“# D isabled O n I ndian T witter” : A Dataset towards Understanding the Expression of People with Disabilities on I ndian T witter (2022)

Abstract: Twitter serves as a powerful tool for self-expression among the disabled people. To understand how disabled people in India use Twitter, we introduce a manually annotated corpus #DisabledOnIndianTwitter comprising of 2,384 tweets posted by 27 female and 15 male users. These users practice diverse professions and engage in varied online discourses on disability in India. To examine patterns in their Twitter use, we propose a novel hierarchical annotation taxonomy to classify the tweets into various themes including discrimination, advocacy, and self-identification. Using these annotations, we benchmark the corpus leveraging state-of-the-art classifiers. Finally through a mixed-methods analysis on our annotated corpus, we reveal stark differences in self-expression between male and female disabled users on Indian Twitter.

## introduction a majority of disabled indians exist at the margins of society with little to no access to social media @xcite . structural embeddings of ableism and patriarchy further intersect to produce multiply oppressive conditions for indian women with disabilities @xcite . however, as access to icts and high-speed internet grows, indian twitter's user base is expanding to include disability influencers, activists, and everyday disabled users. recent work from the west examines elements of 'disability @xcite @xcite but little is known of such twitter use among the indian user base. to fill this gap, we study self-expression among disabled users on indian twitter. as researchers with overlapping interests in disability and gender, we orient our analysis towards gendered self-expression. we introduce a novel human-annotated corpus #disabledonindiantwitter comprising 2,384 tweets published by a preliminary set of 15 male and 27 female disabled people who are active and * this work was done when the author was a research fellow at microsoft research india. in sum, our work makes three major contributions to nlp and accessibility research. with the rising influence of social media on everyday lives, several scholars have started studying the use and non-use of social media platforms by people with disabilities @xcite . for example, scholars have examined the impact of social media on agency and representation of people with vision impairments @xcite as well as its use during public health crises @xcite . however, there is a scarcity of research on gender differences in how disabled people engage with others on social media. nlp methods and datasets on disability: nlp researchers have recently been focusing on studying unintended biases in nlp models against several historically marginalized groups such as those based on differences in race, culture,and gender @xcite @xcite @xcite . several datasets have been created with the goal of fostering research in quantifying societal bias, i.e., the under-representation of these demographics in nlp models that can be detrimental for downstream nlp tasks @xcite @xcite @xcite . although some recent work @xcite has focused on quantifying the representation of the people with disabilities in pre-trained language models, there has been a general lack of attention towards building datasets to understand how disabled people engage and ex-press themselves on social media. @xcite . our work fills this gap by: (1) creating a new dataset containing tweets from people with disabilities in india and making it publicly available foot_0 , and (2) analyzing the dataset to identify differential patterns of twitter usage based on gender, and other attributes. ## dataset we used twitter to collect public data since it allows such analysis through apis available for researchers. we manually selected twitter accounts where users disclosed their disability identity, for example, in their twitter bio, profile picture, username, display name, or within the content of their tweets. we note that disabled representation on indian twitter is marginal due to a lack of access to icts and high-speed internet among the disabled population. further, due to the stigma associated with disability, a limited number of users disclose their disability identity on indian twitter. so while the manual process ensures that our dataset is accurate, it also means that we have a limited number of twitter accounts to analyze. ## annotation taxonomy we propose a linguistic annotation schema (figure 1 ) to study the patterns of self-expression of tweets. the purpose is to categorize each tweet into different classes with each category indicating one of the aspects mentioned above. in this section, we define the broader and fine-grained sub-categories under each category. ## annotation and quality control we manually annotated the tweets to provide a solid benchmark and foster future research. the first two authors of the paper went through a pilot annotation exercise to verify the quality of their annotation schema and guidelines along with two other annotators. for the pilot study, we sampled 250 tweets from our collection following the criteria: 1) the sample contains a considerable percentage of tweets containing disability related keywords and 2) some of the tweets are related to employment, education and health, and 3) the rest of the sample consists of random tweets not related to the above topics. the annotation is based only on the actual text of the tweet without considering additional modalities (e.g. images). this is similar to the information available to the predictive models at the time of training. after the first round of annotations, the inter-annotator agreement was calculated with a pairwise comparison between the annotators using fleiss's kappa (κ) for all the categories. figure 2 lists the agreement values for each annotation category. overall, high inter-rater reliability scores were achieved over all categories. adjudication: the last step of the pilot annotation was to reconcile disagreements among the annotators to produce the final canonical annotation. this step also allowed us to further refine the annotation guidelines. for example, whether a tweet is a fact or an opinion could sometimes be ambiguous and the annotators had to carefully consider and decide whether or not a user was stating opinions as facts. as a result, we refined the definition of "facts" to clearly include a condition that it belong to a set which is universally true. take the following tweet as an example: "people who live in places which have free healthcare are privileged. just saying." this tweet is a classic example of the user's opinion being stated as a fact. but since this statement is not universally true, we classified it as an opinion. ## benchmarking experiments text pre-processing: we pre-processed the tweets using tweetpreprocessor api foot_1 which helps in cleaning the tweet by parsing urls, hashtags, mentions and emojis. ## n-gram analysis to understand the most prominent and distinguishing patterns in each category, we used unigram and bigram tags associated with the annotated categories of the tweets in our data set. each tweet was represented as a tf-idf distribution over the unigrams and bigrams to reveal distinctive syntactic patterns of different categorical themes. for each feature, we computed the strength of correlation between its distribution across posts and the label of the post using pearson correlation (r) @xcite -a standard approach used by other researchers @xcite . finally, we sorted these values and obtained the most important n-grams for each category. on the other hand, the most important features in advocacy/inclusion categories can be classified into positive nouns and supportive or encouraging keywords (e.g. champion, accepted, high support, accessible india); and (b) some suggestions on improving access to vaccination, transportation (e.g. application accessible, education accessible). in the identity category, we observe that most n-grams are related to people disclosing their status as a disabled person. similarly, in the themes, there is a high degreee of association in education and employment related keywords. one interesting highly frequent n-gram is "reservation", and it appears that disabled people are vocal about affirmative action in education and employment. ## analysis on disability and gender since gender and professions play a crucial role in shaping up the ways in which people express themselves on social media, we conducted a preliminary quantitative and content analysis on our corpus to determine the gendered differences in patterns of self-expression of people with disabilities on twitter. we illustrate three preliminary observations emerging from our analysis: figure 4 shows that 15% of the tweets from the male handles were on discrimination, 12% were on advocacy of rights and 10% were personal accounts of harassment. in contrast, 22% of the tweets from the female handles described discrimination related issues, 35% advocated for rights for the disabled and 26% described personal accounts of harassment. we perform a statistical significance test using chi-square @xcite to determine the gendered differences between the patterns of expressing advocacy, harassment and discrimination. the p-values obtained were 0.019, 0.599 and 0.011 for advocacy, harassment and discrimination patterns, respectively. except harassment, the other values were statistically significant. the content analysis revealed that male users are more likely to comment on broader, structural issues underlying discrimination, such as exclusionary government policies. female users, on the other hand, publish a larger number of tweets centering personal experiences of exclusion and disability-related discrimination. disability studies work often cites that disability -associated with being 'dependent and helpless' -is in conflict with masculinity, which is associated with being powerful and autonomous @xcite . within india's deeply patriarchal society in which ableist norms stigmatize male expression of need, this 'masculinity dilemma' may disincentivize male users from sharing personal experiences on public profiles. given a disproportionate burden of discrimination, disabled women may have simply have a larger bank of discrimination-related personal experiences to draw upon. further, within economies of visibility, highly visible women are more likely to perform the labour of authenticity (duffy, 2015; toffoletti and thorpe, 2018; banet-weiser, 2021). since there is a link between personal vulnerability and online harassment (duffy and hund, 2019), this opens up an avenue for further research on experiences of disabled women in india with online harassment. figure 5 shows the quantitative distribution pattern which indicates that disabled sportswomen play a much larger role in tweeting about inclusion (39% of the tweets) compared to disabled women in other professions (33% of the tweets). the difference in advocacy patterns between males with disabilities in sports (22%) and those in other professions (21%) is marginal. our content analysis shows that while paralympians tweet about inclusion, they often use positive tonality, praising the government for new policies, schemes, and initiatives. they also receive significant media engagement from political influencers and government bodies @xcite @xcite . previous work shows that indian sportspeople tend to use twitter to support the government @xcite )a phenomenon rooted in the state's attempts to garner political support from influential figures. this celebration of disabled people in sports is part of the creation of a national identity centered around empowerment and unity. however, the disabled body is positioned as a form of 'apolitical diversity' -a condition produced by the conflation of nationalism and neoliberalism @xcite . in such cases, paralympians may come to be constructed as inspirational 'feel-good' figures who are disincentivized from appearing to be critical online. this finding also points to the fact that online performances of positivity themselves may be gendered among influential disabled users. we note that a marginal percentage of paralympians in our set acted against this norm, tweeting about non-reception of promised rewards, such as jobs and monetary payouts for achievements in paralympic sports. this is a valuable insight showing that disability-related discrimination in india is the norm for even the most influential figures. from the distribution of tweets generated by disabled men, we found that 18% of male users raised their voices about harassment either on a broad societal or personal level, whereas the percentage of disabled women doing the same was only 6%. there is overwhelming evidence that disabled indian women face disproportionately more harassment in contrast to disabled indian men. that online self-expression is not reflective of this points not only to the perceived stigma of mentioning harassment on twitter, but that for women, many such discussions may occur in private online communities rather than the public sphere of participatory social media. previous work has also shown that indian women often limit self-expressions on topics intersecting with patriarchy @xcite . ## conclusion this paper introduced a novel human-annotated corpus "#disabledonindiantwitter" comprising of tweets posted by disabled people in india from a diverse set of professions. we manually tagged the corpus to categorize different patterns of self-expression based on a hierarchical annotation taxonomy. using our corpus, we next conducted quantitative and content analysis to identify gendered differences in expressions of disabled people on indian twitter. we believe that the annotation schema as well as the dataset can be valuable in understanding social media use by disabled people. we aim to make our dataset publicly available to foster research at the nexus of nlp and accessibility.