Don’t Patronize Me! An Annotated Dataset with Patronizing and Condescending Language towards Vulnerable Communities (2020)

Abstract: In this paper, we introduce a new annotated dataset which is aimed at supporting the development of NLP models to identify and categorize language that is patronizing or condescending towards vulnerable communities (e.g. refugees, homeless people, poor families). While the prevalence of such language in the general media has long been shown to have harmful effects, it differs from other types of harmful language, in that it is generally used unconsciously and with good intentions. We furthermore believe that the often subtle nature of patronizing and condescending language (PCL) presents an interesting technical challenge for the NLP community. Our analysis of the proposed dataset shows that identifying PCL is hard for standard NLP models, with language models such as BERT achieving the best results.

## introduction in this paper, we analyze the use of patronizing and condescendig language (pcl) towards vulnerable communities in the media. an entity engages in pcl when its language use shows a superior attitude towards others or depicts them in a compassionate way. this effect is not always conscious and the intention of the author is often to help the person or group they refer to (e.g. by raising awareness or funds, or moving the audience to action). however, these superior attitudes and a discourse of pity can routinize discrimination and make it less visible @xcite . moreover, general media publications reach a large audience and we believe that unfair treatment of vulnerable groups in such media might lead to greater exclusion and inequalities. while there has been substantial work on modelling language that purposefully undermines others, e.g. offensive language or hate speech @xcite , the modelling of pcl is still an emergent area of study in nlp. some reasons for this might include that the use of pcl in the media is commonly unconscious, subtler and more subjective than the types of discourse that are typically targeted in nlp. specifically, a special focus in pcl towards vulnerable communities has not been yet considered, to the best of our knowledge. within a broader setting, there has been some work on pcl which is concerned with the communication between two parties, where one is patronized by the other, such as in social media interactions. in particular, @xcite recently published the talkdown corpus for condescension detection in comment-reply pairs from reddit. in this work, the authors highlight the difficulty of the task and the need for a high-quality dataset annotated by experts, which is the approach we take for studying pcl towards vulnerable communities. to encourage more research on detecting pcl language, we introduce the don't patronize me! dataset 1 . this dataset contains more than 10,000 paragraphs extracted from news stories, which have been annotated to indicate the presence of pcl at the text span level. the paragraphs were selected to cover english language news sources from 20 different countries, covering different types of vulnerable communities (e.g. homeless people, immigrants and poor families). we furthermore propose a taxonomy of pcl categories, focused on pcl towards vulnerable communities. each of the pcl text spans from our dataset has been annotated with a category label from this taxonomy. finally, we also provide some analysis of the dataset. among others, we find that even simple baselines are able to detect pcl to some extent, which suggests that this task is feasible for nlp systems, despite the subtle nature of pcl. on the other hand, we also find that the considered models, including approaches based on bert @xcite , struggle to detect certain categories of pcl, suggesting that there is still considerable room for improvement. in particular, while some forms of pcl can be detected by identifying relatively simple linguistic patterns, many other cases seem to require a non-trivial amount of world knowledge. ## related work condescending and patronizing treatment has been widely studied in various fields, such as language studies (margić, 2017), sociolinguistics @xcite , politics @xcite or medicine @xcite . within nlp, there has been extensive work on several forms of harmful language, but this work has generally focused on explicit, aggressive and flagrant phenomena such as fake news detection @xcite ; trust-worthiness prediction and fact-checking @xcite ; modeling offensive language, both generic @xcite , and geared towards specific communities @xcite ; or rumour propagation @xcite . recently, however, some work on condescending language has started to appear. for instance, @xcite introduced the task of modelling condescension in direct communication from an nlp perspective, and developed a dataset with annotated social media messages. in the same year, @xcite discussed the social and power implications behind certain uses of language, an important concept in the unbalanced power relations that are often present in condescending treatment. also related to unfair treatment of underprivileged groups, @xcite analyzed, from a computational linguistics point of view, how language has dehumanized minorities in news media over time. ## background on pcl research in sociolinguistics has suggested the following traits of pcl towards vulnerable communities: the use of pcl makes it more difficult for vulnerable communities to overcome difficulties and reach total inclusion @xcite . ## the don't patronize me! dataset the don't patronize me! dataset currently contains 10,637 paragraphs about potentially vulnerable social groups. these paragraphs have been selected from general news stories and have been annotated with labels that indicate the type of pcl language that is present, if any. the paragraphs have been extracted from the news on web (now) corpus foot_1 @xcite . to this end, we first selected ten keywords related to potentially vulnerable communities widely covered in the media and susceptible of receiving a condescending or patronizing treatment: disabled, homeless, hopeless, immigrant, in need, migrant, poor families, refugee, vulnerable and women. next, we retrieved paragraphs in which these keywords are mentioned, choosing a similar number of paragraphs for each of the 10 keywords and each of the 20 english speaking countries that are covered in the now corpus. an overview of the number of paragraphs for each keyword-country combination can be found in ## experiments we experiment with a number of different methods to provide baselines for further research in modeling pcl. we consider two settings: predicting the presence of pcl, viewed as a binary classification task (task 1), and predicting pcl categories, viewed as a multi-label classification task (task 2). we evaluate the following methods: for both task 1 and task 2 we used 10-fold cross validation for all the experiments. for the bilstm models, we used 10% of the training data in each fold as a validation set for early stopping. for the svm models, we instead tuned the hyper-parameters using grid search cross-validation. as mentioned pred. paragraph gold pos. after vatican controversy, mcdonald's helps feed homeless in rome. pos. to get further insights into the dataset, ## conclusions and future work we have introduced the don't patronize me! dataset, which is aimed at introducing the nlp community to the challenge of identifying and categorizing patronizing and condescending language (pcl) towards vulnerable communities. as another contribution of this paper, we also introduced a two-level taxonomy of pcl categories, which was used for annotating the dataset. our exploratory analysis shows that identifying condescending or patronizing texts is a difficult challenge, both for human judges and for nlp systems. apart from the subtle and subjective nature of pcl, a particular challenge comes from the fact that accurately modelling such language often requires knowledge of the world and common sense (e.g. to assess whether a proposed solution is shallow, or whether a particular presupposition is warranted). nonetheless, we found that both identifying pcl (task 1) and categorizing occurrences of pcl (task 2) is feasible, in the sense that non-trivial results can be achieved, with bert-based approaches outperforming simpler methods. future work will include the development of new models for both detecting and categorizing pcl. in addition, we plan to continue to extend the don't patronize me! dataset with more paragraphs from news stories, as well as text fragments from different sources, such as social media or ngo campaigns, to create a useful and updated resource for the community.