University of Bucharest Team at S emeval-2022 Task4: Detection and Classification of Patronizing and Condescending Language (2022)

Abstract: This paper details our implementations for finding Patronizing and Condescending Language in texts, as part of the SemEval Workshop Task 4. We have used a variety of methods from simple machine learning algorithms applied on bag of words, all the way to BERT models, in order to solve the binary classification and the multi-label multi-class classification.

## introduction the patronizing and condescending language detection task @xcite is based on the paper don't patronize me! (p'erez-almendros et al., 2020), which is an annotated dataset with patronizing and condescending language towards vulnerable communities. the aim of this task is to identify pcl, and to categorize the language used to express it, specifically when referring to communities identified as being vulnerable to unfair treatment in the media. participants were provided with sentences in context (paragraphs), extracted from news articles, in which one or several predefined vulnerable communities are mentioned. the challenge is divided into two subtasks. 1. subtask 1: binary classification. given a paragraph, a system must predict whether or not it contains any form of pcl. ## background the dataset used for this semeval 2022 task was don't patronize me! (p'erez-almendros et al., 2020), which contains a suite of sentences that mention some vulnerable communities and published in media in a lot of english speaking countries. the paragraphs were manually annotated to show 1) whether the text contains any kind of pcl, and 2) if it contains pcl, what linguistic techniques (categories) are used to express the condescension. the paragraphs, according to (p'erez-almendros et al., 2020), were extracted from news on web (now) corpus @xcite , being annotated by three expert annotators, with backgrounds in communication, media and data science. the dataset for subtask 1 (binary classification) contained a number of 10.636 paragraphs and 2.792 instances were used for the categories classification subtask. in figure 1 , it can be seen that for the first subtask, there are almost 1000 texts that contain pcl. this means that the dataset is highly imbalanced and this problem needs to be addressed. for task 2, the paragraphs from task 1 are split according to the type of pcl speech category into sentences, resulting in 950 samples. balanced: in this approach, we computed a metric in which we obtained a class weight according to the imbalance of the dataset. through this method, we gave some different weights to both the majority and minority classes. this whole process had the purpose to penalize the miss classification made by the minority class by setting a higher class weight and at the same time, reducing the weight for the majority class. the dataset was preprocessed. the preprocessing consisted in: clearing the special characters, lowercasing, tokenization, stopwords removal, removing the words shorter than 3 characters. then, the resulted (and clean) dataset was split into two preprocessed types: lemmatized cleaned dataset and stemmed cleaned dataset. these two datasets were generated in order to make some comparison between those two techniques and to see which provided the best results. to extract features from text, we have used tf-idf @xcite , keras tokenizer 2 , word2vec with skip-gram (mikolov 1 hate speech datasets 2 tokenizer method brought by keras et al., 2013) and, finally, bert tokenizer provided by hugging face @xcite . we have also used a variety of models such as neural networks with 3 dense layers, long short term memory (lstm) @xcite with 64 and 128 neurons with dropout of 0.1 as well, basic machine learning algorithms like logistic regression, random forest, support vector machines as xgboost. in the end, we decided to try bert embeddings and a bert classification model, bertforsequenceclassification foot_1 , that contains a single linear classification layer on top and that provided the best results after all of the other approaches. another approach, called "text shards" made use of the subtask related to multi-class classification as well. for an average text that contains pcl, only some small pieces of them are actually pcl and the rest of the text are not. the assumption is that this confuses the model, because a combination of pcl and non-pcl is labeled as pcl. to address this, the following approach is used: if it is labeled as pcl, then we consider the whole text as pcl. ## conclusion in this paper, we presented our solution to the problem posed by semeval 2022 task 4: patronizing and condescending language detection. we applied various methods, including the application of word embedings (bag of words, word2vec, bert), tokenization, oversampling/undersampling of the datasets. in the binary classification problem, the approach that gave the best result on the validation dataset was bert transformers combined with bert for sequence classification, obtaining 0.50 as f1 score, followed by logistic regression applied on stemmed smote dataset with a performance of 0.38. in the multi classification multi label task, the number of labels proved to be a challenge. the results overall are low and the models were only able to learn only a few classes. the general class approach also proved to be inefficient. perhaps a more suitable approach would be to build more complex models and use models that do not rely on specific pretrained approaches. some recommendations for future work could be to have a better approach and introduce more linguistic insight in the approach.