JCT at S em E val-2022 Task 4-A: Patronism Detection in Posts Written in E nglish using Preprocessing Methods and various Machine Leaerning Methods (2022)

Abstract: In this paper, we describe our submissions to SemEval-2022 subtask 4-A - “Patronizing and Condescending Language Detection: Binary Classification”. We developed different models for this subtask. We applied 11 supervised machine learning methods and 9 preprocessing methods. Our best submission was a model we built with BertForSequenceClassification. Our experiments indicate that pre-processing stage is a must for a successful model. The dataset for Subtask 1 is highly imbalanced dataset. The f1-scores on the oversampled imbalanced training dataset were higher the results on the original training dataset.

## introduction the explosion of social media in recent years also enables an increasing the number of patronizing and condescending language (pcl). patronizing language is best described as expressions that are agreeable and show kindness to a person or group in a condescending manner, indicating that the person or group is inferior @xcite . the discourse of condescension has three main characteristics: (1) it does not contain anything openly critical or negative, and often contains insincere praise; (2) it assumes a difference in status and worth between the writer and the person who wrote about him; and (3) this assumed difference is disputed by the listener @xcite . pcl can harm individuals or groups of people and may cause harmful effects on society. therefore, it is important to develop efficient computerized systems capable of detecting pcl @xcite . pcl detection is not a simple problem because it requires understanding the context of the situation, the relevant culture, and indirect clues. in social media texts, the problem is harder due to the different levels of ambiguities in natural language and the noisy nature of such texts. in contrast to the offensive language or hate speech detection field, where there has been relatively an extensive research (e.g., @xcite @xcite , pcl is still a relatively new and open field of study in natural language processing (nlp) and machine learning (ml) @xcite . @xcite introduced the don't patronize me! dataset. this dataset contains paragraphs extracted from news stories, which have been annotated to indicate the presence of pcl at the text span level. this paper describes our research and participation in subtask 4-a for patronism detection in posts written in english. the full description of task 4 in general and 4-a, in particular, is given in perez-almendros et al. @xcite . the structure of the rest of the paper is as follows. section 2 introduces a background concerning patronism detection, text preprocessing, and tc with imbalanced classes. section 3 describes subtask 4-a and its training dataset. in section 4, we present the submitted models and their experimental results. section 5 summarizes and suggests ideas for future research. ## related work various nlp methods have been applied in the detection of several types of harmful language such as offensive language or hate speech detection @xcite @xcite . previous nlp tasks have generally focused on explicit, aggressive, and flagrant phenomena such as fake news detection @xcite . during the last three years, several studies on pcl have appeared. @xcite potts (2019 ) jct at semeval-2022 task 4-a: patronism detection in posts written in english using pre-processing methods and various machine learning methods yaakov hacohen-kerner, ilan meyrowitsch, matan fchima department of computer science, jerusalem college of technology, lev academic center 21 havaad haleumi st., p.o.b. 16031, 9116001 jerusalem, israel kerner@jct.ac.il, meyrowitsch@gmail.com, matanf1992@gmail.com introduced the task of modeling condescension and developed an annotated dataset of social media messages. sap et al. (2019) discussed various implications behind certain uses of language. mendelsohn et al. ( 2020 ) analyzed, from a computational linguistics viewpoint, how language has dehumanized minorities in media news. ## task and training dataset description we only participated in subtask 4-a -"patronizing and condescending language detection: binary classification", which deals with the classification of each post as a patronizing or condescending language (pcl) or not in the english language. the analysis of the details presented in ## the submitted models and experimental results we applied 11 supervised ml methods to the training dataset. seven of them were classical ml methods: random forest (rf), k nearest neighbours (knn), support vector classifier (svc), xgboost classifier, logistic regression (lr), decision tree (dt), naive bayes (nb), and four of them were deep learning (dl) methods: bert, distilbert, roberta, and albert. in our various models, we applied nine subtypes of preprocessing methods: remove newlines, remove html tags, remove links, remove white spaces, remove accented characters, conversion to lower-case, reduce repeated characters, and punctuations, expand contractions, and remove special characters. these methods were applied using the following tools and information sources: in our experiments, we tried to find the best combination of ml method, preprocessing methods, and oversampling methods. the training set was split into 80:20 train: test and the training test was using 90 percent in every epoch to train and 10 percent of the training set was used for validation. figure 1 presents training and validation loss curves of our bert model with 20 epochs showung that training and validation continuously improved themselves. we noticed that a gap between training and validation began to grow, therefore we had to stop the model after overfitted. figures 2 and 3 present the confusion matrices of our bert model with 20 epochs and the decision tree model, respectively. the confusion matrix of both models demonstrates that the dataset is imbalanced as shown in our code is available on github at https://github.com/meyrow/pcl-detection-task4semeval2022 . our models are available for reproducibility with comments that explain the code and parameters such as epsilon, learning rate batch, and epochs. ## conclusions and future research in this paper, we describe our submissions to subtask 4-a of the semeval-2022 contest. we submitted the models that achieved the best results while trying to choose two models that applied different supervised learning methods. future research ideas include (1) acronym disambiguation that will extend and enrich the social text and might enable better classification model name split mode ml method features f1-score over the pcl class on the training dataset f1-score over the pcl class on the test dataset matan_bert 80:20 bertforsequence classification layers: 768 learning rate: 2e-7 epsilon: 1e-8 epochs: 20 batch-size: 16 0.77 0.377 matan_ decision_tree 70:30 decision tree @xmath0.7 was not published