Xu at S em E val-2022 Task 4: Pre- BERT Neural Network Methods vs Post- BERT R o BERT a Approach for Patronizing and Condescending Language Detection (2022)

Abstract: This paper describes my participation in the SemEval-2022 Task 4: Patronizing and Condescending Language Detection. I participate in both subtasks: Patronizing and Condescending Language (PCL) Identification and Patronizing and Condescending Language Categorization, with the main focus put on subtask 1. The experiments compare pre-BERT neural network (NN) based systems against post-BERT pretrained language model RoBERTa. This research finds NN-based systems in the experiments perform worse on the task compared to the pretrained language models. The top-performing RoBERTa system is ranked 26 out of 78 teams (F1-score: 54.64) in subtask 1, and 23 out of 49 teams (F1-score: 30.03) in subtask 2.

## introduction an entity is considered to engage patronizing and condescending language (pcl) when its language use presents a superior attitude towards others or depicts them in a compassionate way @xcite . such language is often used toward vulnerable communities such as women, refugees, and homeless people. these unfair treatments of the vulnerable groups are believed to result in further exclusion and inequalities in society. compared to other types of harmful language (e.g. hate speech), pcl is considered more subtle and unconscious. given the negative effects of pcl on society and its subtle nature, enabling computers to identify and categorize pcl presents an interesting technical challenge to the nlp community. this paper describes my participation in both subtasks of the semeval-2022 task 4: patronizing and condescending language detection @xcite . subtask 1, patronizing and condescending language identification is a binary text classification task to predict whether a given paragraph contains pcl or not. subtask 2, patronizing and condescending language categorization is a multi-label classification task to identify the categories of a given paragraph according to the taxonomy defined in @xcite , which categorizes pcl into 7 types: 1) unbalanced power relations 2) shallow solution 3) presupposition 4) authority voice 5) metaphor 6) compassion 7) the poorer, the merrier. the dataset @xcite ) contains annotated paragraphs in english, collected from news stories in 20 english-speaking countries. 1 the focus of my experiments is primarily on subtask 1, meanwhile, this paper also proposes a solution to subtask 2. for subtask 1, the experiments compare pre-bert neural network (nn) based systems including a majority voting system of nn models against pretrained language model roberta. the experiments start with building individual nn models from the most basic artificial neural network (ann) to long short-term memory network (lstm) models following previous work on nn for text classification. it was found that the nn-based systems in the experiments perform worse on this task in comparison to the pretrained language models. the best-performing nn-based voting system could not outperform the roberta baseline model. for subtask 2, this paper simply proposes a roberta solution. the code is released at: github.com/jinhxu/ pcl-detection-semeval2022-task4. ## background numerous previous research had been conducted on the treatment of condescension and patronization. the studies range in various areas from so-ciolinguistics @xcite to medicine @xcite . whereas in the field of natural language processing, automatically identifying or categorizing pcl has been an understudied area. most research on harmful language detection has focused on more explicit and aggressive topics such as hateful speech @xcite , offensive language @xcite , and fake news @xcite . only in recent years, few in the research community have started to show interest in enabling computers to identify condescending language. prior to this shared task, for instance, @xcite proposed an annotated talkdown corpus of condescending language from social media. ## dataset the corpus used for this shared task, the don't patronize me! dataset is described in @xcite . the corpus consists of 10,637 paragraphs extracted from the news on web (now) corpus @xcite . the paragraphs were selected according to 10 keywords related to potentially vulnerable communities (disabled, homeless, hopeless, immigrant, in need, migrant, poor families, refugee, vulnerable, and women). and for each keyword, a similar number of paragraphs were chosen for each of the 20 englishspeaking countries. 2 each paragraph is annotated with a true/false label indicating whether it contains pcl or not, and the ones that contain pcl are annotated with a category label. each category label is a set of 7 binary predictions, each prediction indicates the existence of a specific type of pcl. the dataset is highly imbalanced. the pos:neg ratio is approximately 1:10, which poses a challenge to the predictive modeling process. in the experiments of this paper, various strategies were employed in order to deal with the imbalance in data. the following sections will describe these strategies in detail. additionally, the shared task provides a comparable 80/20 split of the training data for development. in the experiments of this paper, the same split was used to train models and generate predictions in the development stage. ## conclusion the experiments of this paper compare some of the pre-bert neural network based systems against the post-bert pretrained language model roberta. the experiments start with building individual nn models from the most basic ann models to the more sophisticated lstm models, and create a majority voting system based on the individual nn models. it was found that the nnbased systems in the experiments perform worse on the task compared to the roberta baseline model. and the community models pretrained on relevant data such as sentiment and toxicity data turn out to be too specialized in their own task thus resulting in poor performance on the pcl data compared to the roberta-base model. this paper explores neural network models mainly the basic ann and lstm models. future work should also consider convolutional neural networks (cnn) for pcl detection. in addition to the neural networks, i suggest also investigating classic machine learning models such as logistic regression, as well as indicative linguistic features of pcl for feature engineering. furthermore, on top of the roberta-base model, i propose to pretrain a roberta model using the talkdown corpus proposed in @xcite , and fine-tune the pretrained model using the pcl data. finally, future work should run further error analysis of the models to improve performance.