RNRE - NLP at S em E val-2022 Task 4: Patronizing and Condescending Language Detection (2022)

Abstract: An understanding of patronizing and condescending language detection is an important part of identifying and addressing discrimination and prejudice in various forms of communication. In this paper, we investigate several methods for detecting patronizing and condescending language in short statements as part of SemEval-2022 Task 4. For Task 1a, we investigate applying both lightweight (tree-based and linear) machine learning classification models and fine-tuned pre-trained large language models. Our final system achieves an F1-score of 0.4321, recall-score of 0.5016, and a precision-score of 0.3795 (ranked 53 / 78) on Task 1a.

## introduction patronizing and condescending language (pcl) is characterized by expressions that reveal a sense of compassion or superiority toward others. research suggests that pcl can perpetuate-and even veil-discrimination toward vulnerable groups @xcite . to make matters worse, its presence is often more subtle than other offensive language @xcite . detecting pcl is a challenging task for humans annotators-and the task becomes even trickier for artificial systems. given the varied nature of condescension, current nlp models struggle to accurately detect pcl. part of the issue is defining what patronizing and condescending language is, exactly-what one reader considers condescending might be deemed perfectly respectful by another. semeval-2022 task 4 attempts to address some of these issues @xcite . pérez et al. classify pcl into seven distinct categories: unbalanced power relations, shallow solutions, presupposition, authority voice, metaphor, compassion, and the poorer, the merrier. task 1a seeks to determine whether the sequence of text contains any form of patronizing or condescending language. task 1b seeks to identify the pcl category that corresponds to the sequence of patronizing or condescending text. overall, developing systems that perform well on these tasks--that are capable of flagging condescending language-is a critical step toward reducing discrimination toward minority groups in media. we investigate various lightweight models to determine whether such models trainable on an extremely small compute budget could effectively identify pcl, as well as larger pre-trained transformer models to identify whether performance improves as models increase in size and complexity. ## dataset for task 1a, we train and validate our models on the semeval-2022 task 4 training set @xcite . each paragraph has been annotated by two annotators on a likert-type scale from 0 to 2 as shown in we did not investigate task 1b. ## results the official evaluation set performances for our classifiers are listed in for task 1a (patronizing and condescending language binary classification), we submitted our two highest-performing lightweight models (support vector machine and random forest models). due to error, we did not submit our bert model. overall, we ranked 53rd out of 78 on this task, achieving a positive-class f1 score of 0.4321. ## conclusion we have proposed lightweight and pre-trained systems that are able to detect pcl in text. we find that reasonably lightweight models such as support vector machine and random forest are effective at predicting the level of patronizing and condescending language. however, we note that ensembling these models together does not improve performance. additionally, normalizing the dataset had little effect for most models--and in the case of the multi-layer perceptron model actually returned a lower positive-class f1 score. however, it substantially increased the f1 score with the support vector machine model from 0.37 to 0.48. finally, we find that fine-tuning large pre-trained models like bert and roberta achieves results at least as accurate as lightweight models-if not better. an area of interest for future work may be further experimentation with ensembles of lightweight models, as well as inquiries into adversarial and contrastive learning to improve overall accuracy. overall, our results show that both lightweight and fine-tuned models can achieve reasonable results at detecting patronizing and condescending language in human channels of communication.