DH - FBK at S em E val-2022 Task 4: Leveraging Annotators’ Disagreement and Multiple Data Views for Patronizing Language Detection (2022)

Abstract: The subtle and typically unconscious use of patronizing and condescending language (PCL) in large-audience media outlets undesirably feeds stereotypes and strengthens power-knowledge relationships, perpetuating discrimination towards vulnerable communities. Due to its subjective and subtle nature, PCL detection is an open and challenging problem, both for computational methods and human annotators. In this paper we describe the systems submitted by the DH-FBK team to SemEval-2022 Task 4, aiming at detecting PCL towards vulnerable communities in English media texts. Motivated by the subjectivity of human interpretation, we propose to leverage annotators’ uncertainty and disagreement to better capture the shades of PCL in a multi-task, multi-view learning framework. Our approach achieves competitive results, largely outperforming baselines and ranking on the top-left side of the leaderboard on both PCL identification and classification. Noticeably, our approach does not rely on any external data or model ensemble, making it a viable and attractive solution for real-world use.

## introduction detecting patronizing and condescending language (pcl) is an open, challenging, and underexplored research area in natural language processing @xcite . a patronizing and condescending attitude is expressed as a good-natured and beneficial attitude from a person of authority towards others, who are typically depicted in a subtly compassionate way @xcite . pcl is a mildly perceived phenomenon. it is often unconscious, driven by good intentions, and expressed through flowery wordings @xcite . this makes pcl identification and classification difficult both for nlp systems and human annotators (cf. figure 1 ), as it cannot be linked to specific words. nonetheless, figure 1 : example showing that patronizing and condescending language is a subtle linguistic phenomenon that human annotators (a 1 , a 2 ) often perceive differently, and thus annotate in different ways. to encourage research on patronizing and condescending language detection, the semeval-2022 task 4 @xcite has recently been proposed. the shared task aims at investigating methods for the identification of pcl (subtask 1; figure 1 , top) and the categorization of the linguistic techniques which are used to express it (subtask 2; figure 1 , bottom) on english news stories mentioning vulnerable communities (section 2). in this paper, we present the dh-fbk entry for the semeval-2022 task 4 @xcite . motivated by the subtle nature of example category "we can be extremely proud of the current women winemakers" unbalanced power relations "the inclusion of a refugee team" shallow solution "an immigrant to a developed country lives in two worlds" presupposition "women must wake up" authority voice "trapped in the prison of poverty" metaphor "more than 400 suspected asylum seekers are awaiting their fate" compassion "how talented disabled people can be" the poorer, the merrier our systems achieve competitive results on the semeval-2022 task 4, outperforming the organizers' roberta @xcite baseline by a large margin (subtask 1: +8.8 f 1 ; subtask 2: +26.9 f 1 ) and consistently ranking on the top-left side of the leaderboard (subtask 1: 18 th out of 78 teams; subtask 2: 13 th out of 49 teams) without using any external data or ensemble strategy, making it a viable solution for real-world use. we make our code publicly available to the research community to encourage future work on this direction. foot_0 ## data and task description in this section, we present relevant details on data and the associated shared task. we firstly summarize the dataset (section 2.1) and describe the task setup (section 2.2). next, we focus on the data annotation process as it is central for understanding our methods (section 2.3). ## methods models proposed for pcl identification and classification are all based on multi-task learning @xcite and use multiple views of input data, inspired by @xcite . in this section, we firstly introduce the general framework on which all our models are based on (section 3.1). then, we provide details on data-and task-specific components that we use in our systems, namely dataset views (section 3.2) and auxiliary tasks (section 3.3). lastly, we present the composition of our final models (section 3.4). ## experiments in this section, we first outline the experimental setup (section 4.1). then, we present the results of our models (section 4.2), as well as additional analyses and discussion (section 4.3). ## conclusion in this paper, we presented our submitted systems to semeval-2022 task 4. we showed that leveraging annotators' uncertainty and disagreement during training in a multi-task, multi-view framework is beneficial for the identification and classification of patronizing and condescending language, and achieves competitive results on the official leaderboard without relying on any external data or model ensemble. we also showed that sequential finetuning is a viable alternative to tackle pcl identification and classification jointly, with the goal of reducing the use of computational resources during inference, although it obtains lower performance compared to our tailored solutions. a thorough analysis on the impact of diverse auxiliary tasks on the performance of our models for pcl detection, and an investigation on the role of uncertainty and disagreement further confirmed the importance of considering annotators' point of view in pcl detection. as future work, we aim to test the presence and assess the impact of spurious lexical biases in the dataset @xcite and extend our models to other genres, such as social media @xcite . we hope this work will encourage future efforts towards annotatorscentric nlp, on pcl detection and other subjective tasks more broadly.