Identifying Condescending Language: A Tale of Two Distinct Phenomena? (2022)

Abstract: Patronizing and condescending language is characterized, among others, by its subtle nature. It thus seems reasonable to assume that detecting condescending language in text would be harder than detecting more explicitly harmful language, such as hate speech. However, the results of a SemEval-2022 Task devoted to this topic paint a different picture, with the top-performing systems achieving remarkably strong results. In this paper, we analyse the surprising effectiveness of standard text classification methods in more detail. In particular, we highlight the presence of two rather different types of condescending language in the dataset from the SemEval task. Some inputs are condescending because of the way they talk about a particular subject, i.e. condescending language in this case is a linguistic phenomenon, which can, in principle, be learned from training examples. However, other inputs are condescending because of the nature of what is said, rather than the way in which it is expressed, e.g. by emphasizing stereotypes about a given community. In such cases, our ability to detect condescending language, with current methods, largely depends on the presence of similar examples in the training data.

## introduction patronizing and condescending language (pcl) has been a topic of interest across a wide range of disciplines, including politics, journalism and medicine @xcite @xcite . the use of pcl implies a position of superiority of the author regarding the person or community they are referring to, suggesting an imbalance in terms of power or privilege @xcite . especially when directed towards vulnerable communities, pcl fuels discrimination and perpetuates inequalities @xcite , feeds stereotypes and misinformation @xcite , and makes it more difficult for underrepresented groups to overcome social difficulties @xcite . the nlp community has recently also turned its attention to the study of pcl, focusing on the task of detecting and categorizing this kind of harmful discourse. for instance, @xcite introduced the talk down dataset, which is focused on condescending language in social media, while @xcite introduced the don't patronize me! (dpm) dataset, which is focused on the way in which vulnerable communities are described in news stories. from the nlp point of view, the study of pcl is interesting because it is more subtle, and therefore presumably harder to detect, than other forms of harmful language, such as hate speech @xcite and offensive language @xcite . moreover, identifying pcl often seems to require a deep commonsense understanding of human values @xcite ). consider the following example from the dpm dataset: "people across australia ordered pizzas to be delivered on saturday night, with the ample leftovers donated to local homeless shelters." we can understand that, although donating food can be socially valuable, the impact of this particular action is painted in an excessively positive light (e.g. as evident in the phrase ample leftovers). moreover, this seems to refer to a campaign to increase the consumption of pizzas with the excuse to help homeless people, which as humans we might also find condescending. however, an nlp model might struggle to infer such connotations. based on the premise that pcl detection would present unique challenges, semeval-2022 featured a task devoted to pcl detection and categorization @xcite . the top-ranked submissions for this task achieved a remarkably strong performance, which seems to somewhat undermine the assumption that the subtle nature of pcl would make its detection inherently hard. moreover, even the best systems @xcite @xcite , relied on a judicious use of more or less generic text classification techniques, improving on the roberta @xcite baseline by addressing the class imbalance, adding a contrastive learning loss, using ensembles of language models, etc. in particular, there was little evidence of the presumed need to focus on commonsense understanding of human values. in this paper, we present an analysis of the semeval-2022 pcl detection dataset, in light of the aforementioned observations. our central argument is that the dataset contains examples of two rather distinct types of condescending language, and that the difference between the two is fundamental to understanding why the task, as it has been formulated, might be significantly easier than the task of detecting condescending language in general. we argue that a deeper understanding of these two phenomena might lead to a better performance on pcl detection, which in turn can mitigate the discourse of condescension towards vulnerable communities. we will refer to these two types as linguistic pcl and thematic pcl. linguistic pcl some instances of pcl are related to the way in which a given claim is expressed. consider the following example: "...we must rally together as humans, understanding that we have a responsibility to help the world's most vulnerable to survive and rebuild their lives [...]" in this sentence, we can see two common aspects of pcl. first, expressions such as we must or we have a responsibility, indicate an authority voice and attitude @xcite . second, the sentence evokes the idea of a saviour and a victim. note how the condescending tone of the sentence is related to linguistic aspects that are relatively easy to identify (e.g. the presence of modal verbs such as must) and largely independent of the community being referred to. we will refer to such cases as linguistic pcl. our hypothesis is that detecting linguistic pcl is relatively straightforward for language models, as this is ultimately about learning to detect a particular writing style @xcite . thematic pcl there are also examples of pcl where the message itself is condescending, irrespective of how it is formulated. we will refer to such cases as thematic pcl. consider the following example: overview in this paper, we present an analysis of the semeval-2022 dataset, aimed at testing the aforementioned hypotheses about linguistic and themed pcl. first, we carry out two experiments in which models are trained such that they are prevented, to some extent, from learning about condescending themes associated with individual communities. our experiments show that there are some communities for which this leads to a dramatic drop in performance, while for other communities there was no negative impact at all. this suggests that there is indeed considerable overlap in the kinds of themed pcl that can be found in the training and test sets of the semeval dataset, but only for some communities. we then complement these results with a qualitative analysis based on ideas from critical discourse analysis (cda), a technique which emerged from critical linguistics in the 1970s @xcite @xcite @xcite . cda looks at the relation between power and language, and how discourse expresses social hierarchy and inequalities. this qualitative analysis provides further suppors for the idea that (i) pcl detection models can identify linguistic pcl even if they have not seen similar cases during training while (ii) their ability to detect instances of themed pcl is much more dependent on the training examples. the study of pcl the discourse of condescension has been widely studied in disciplines such as sociolinguistics, politics, psychology, medicine, cultural studies, public relations, journalism and international cooperation @xcite @xcite @xcite . within the nlp community, the study of pcl is more recent, although there is a longer tradition of looking at harmful language more generally @xcite @xcite @xcite . as already mentioned in the introduction, @xcite and perez-almendros et al. (2020) addressed condescension in different types of discourse, while other recent works addressed some closely related aspects, such as how language conceals power relations @xcite , expresses authoritarian voices as empathy @xcite or dehumanizes minorities @xcite . pcl towards vulnerable communities is a subtle and subjective kind of language, often unconscious and well intended @xcite ). an author might use pcl while trying to help a community or individual, raise their voice for them or move the audience to action. however, pcl can be very harmful, as it routinizes discrimination @xcite , creates stereotypes @xcite and reinforces inequalities @xcite @xcite , feeding the dichotomy of a saviour @xcite and a helpless victim. pcl contributes to the "distorted and stereotyped representation" @xcite ) that vulnerable communities or underrepresented groups fre-quently receive in the media. the coverage of minorities in the media our emphasis on the distinction between thematic and linguistic pcl draws from previous analysis of the relation between discourse and power, and how language can reinforce inequalities and exclusion. such studies are mainly based on critical discourse analysis (cda), which is concerned with the analysis of unbalanced power relations and privilege in public discourse and the construction of identities in the media. it also draws our attention to the influence (voluntary or not) that the author of a public discourse has over the construction of an image in the mind of the audience by, for instance, their selection of words, use of linguistic structures and omissions when depicting a specific community or situation @xcite . @xcite suggests that, in the critical study of a discourse, an analyst should look for certain linguistic or stylistic features in a text, such as the use of modal verbs (modality) or the identity of the subject and the object of an action (transitivity), to find expressions of power imbalance and inequality. he also suggests to look at recurrent themes and stereotypes in the media coverage of minorities. along this direction, the same author studied the treatment of homelessness in the us in 1999 @xcite . he collected a corpus of 163 newspapers articles and editorials which mentioned the keyword homeless and analyzed, among others, the more recurrent themes and stereotypes related to this community. for instance, he shows that the analyzed data includes "desire of independence" or "lack of life skills" as common themes when referring to causes of homelessness. also, the theme "bad grooming" is highlighted as one effect of homelessness. "religious support", "food donation" and "donated clothes" are common themes in the discussion of public responses, which represent shallow and ephemeral solutions for a structural, deeprooted problem, and thus again reinforce the charitable, saviour-victim treatment of a community. using a similar approach, díaz-rico (2012) analyzed 93 articles about mexican immigrants from the los angeles times, published in 2010. she claims that the selection of topics and themes is the most important aspect of journalism and that newspapers use the drama of a story to gain attention from their audience. although the language and topics she analyses in this work are often openly discriminatory and offensive, she also finds expressions that, through rhetorical figures, connotation and semantic selection, reinforce power relations and inequalities (e.g."help new arrivals get on their feet", or "ballot crusade"). ## methodology in sections 4 and 5, we describe experiments in which pcl classifiers are trained in a way that (partially) prevents them from learning about community-specific thematic pcl. this will allow us to better characterise the abilities of finetuned language models, as the overlap between the themes covered by the training and test sets is reduced. in this section, we first describe the basic experimental setup that we rely on throughout the paper (section 3.1). subsequently, we describe a simple strategy for characterizing topics or themes that are strongly associated with particular vulnerable communities or groups (section 3.2). ## omitting community-specific training data our main hypothesis, as outlined in the introduction, is that the semeval pcl detection task is easier than one might expect because it involves a combination of linguistic pcl, which is easier to detect, and thematic pcl. while we believe that thematic pcl can be hard to detect in general, our hypothesis is that it is simplified, in the context of the semeval dataset, because of the overlap between the themes covered in the training and test data. if a language model is truly able to recognize pcl, then it should be capable of identifying (thematic) pcl about communities it has not seen during training. in this section, we report the results of an experiment where we test the performance of the model per community in two settings. first, we consider the standard setting, where the model has had access to the entire training set. second, we consider the setting where all examples about the community being tested were removed from the training set. note that for the latter case, we need to train a separate model for every community, each time omitting the corresponding training examples. the results are summarized in ## masking community-specific terms we now present a variant of the experiment from the previous section, where no training examples are removed, but we instead mask (some) occur-rences of community-related terms, as identified in section 3.2, in the training data. note that we mask occurrences of such terms regardless of the community a training example is about (e.g. a term that was identified for refugees would still be masked in examples about immigrants). this setup has the advantage that the number of training examples remains constant. moreover, the model may now also be prevented from learning thematic pcl by training on related communities. for instance, in the setting from section 4, the model may be able to learn condescending themes about the homeless community from training examples mentioning the vulnerable keyword. the results are reported in ## qualitative analysis the experiments in sections 4 and 5 have revealed stark differences in the robustness of pcl detection models across different communities, when the model is (partially) prevented from learning community-specific themes during training. in particular, our results suggest that pcl examples for migrants + immigrants are often thematic in nature, with the same themes recurring in both the training and test sets. conversely, the results for women suggest that pcl towards that community is more likely to be linguistic in nature. in this section, we supplement our findings with a qualitative analysis, where we focus on these two communities. ## conclusions we have studied the challenge of detecting patronizing and condescending language (pcl), with the aim of improving our understanding of its na-classified correctly only with partial masking "eleven months into his administration, the country is showing signs of progress in most sectors of the economy. with the implementation of the free senior high school programme, most students, especially those from poor families, who hitherto would not have progressed to the senior high school, have the opportunity now to receive secondary education to make them better and more functional in society", dr nyarko said. today, brooklyn is home to people of all races, most struggling to make ends meet. council flats continue to degrade as the population swells -unemployment and homelessness sees people of different races lining up side-by-side for a plate of free food. it's a representation of the rainbow nation in trauma, with its colours dulled and blended together by suffering. helping refugee children fit in a bonus for juventus football camp. swimming superstar adam peaty is set to unveil a new motorbike for charity in memory of schoolgirl imogen evans, who used the service. the shropshire and staffordshire blood bikes is a charity which saves lives by delivering vital blood supplies to those in need. radio veritas, the leading faith-based am station in mega manila, continues its commitment to charity and public service through an initiative dubbed as "good samaritan". since it was launched last june 2017 (airing every monday to friday from 1-2 p.m.), radio veritas has listed 182 cases of pleads and requests that have been fulfilled through this program. it serves as a platform for those in need to make on-air appeals for legal, spiritual, medical, material and financial assistance, and link them to "good samaritans" who are willing to share.