Sapphire at S em E val-2022 Task 4: A Patronizing and Condescending Language Detection Model Based on Capsule Networks (2022)

Abstract: This paper introduces the related work and the results of Team Sapphire’s system for SemEval-2022 Task 4: Patronizing and Condescending Language Detection. We only participated in subtask 1. The task goal is to judge whether a news text contains PCL. This task can be considered as a task of binary classification of news texts. In this binary classification task, the BERT-base model is adopted as the pre-trained model used to represent textual information in vector form and encode it. Capsule networks is adopted to extract features from the encoded vectors. The official evaluation metric for subtask 1 is the F1 score over the positive class. Finally, our system’s submitted prediction results on test set achieved the score of 0.5187.

## introduction patronizing and condescending language (pcl) can be considered when someone's language has a superior attitude towards others, demeans others, or describes the situation of others in a compassionate way. such expressions are often unconscious, and are used by people to try to induce action or raise awareness. because of its subtlety and often well-meaning when used, users often overlook the demeaning elements of this expression. such elements may contribute to the stereotyped influence of society on a group, making discrimination normalized and even leading to stronger exclusion @xcite . detecting pcl in media text is a challenging task. recognizing pcl based on natural language processing (nlp) can alert speakers to examine the rationality of their speeches, so that speeches can be more inclusive and constructive, which in turn leads to more responsible communication. when processing corpus, the pre-trained model can convert text information into vector representation, making it more suitable for nlp tasks. early pre-trained models were designed to learn representational word embeddings, such as word2vec @xcite and @xcite . although such methods can capture the semantics of words through word embeddings, they cannot capture the concepts in the context. with the introduction of new technologies, there are now pre-trained models that can learn to represent contextual word embeddings, such as the elmo @xcite model based on lstm @xcite and the bert @xcite model based on transformer encoder @xcite . in recent years, using deep neural networks in nlp, such as convolutional neural networks (cnns) in text classification @xcite , has become mainstream. capsule networks @xcite , as a structure proposed on the basis of cnns to improve spatial sensitivity in computer vision, is also used in text classification tasks @xcite . @xcite further suggest a simple routing method that effectively reduces the computational complexity of dynamic routing. this task aims to predict whether each news text for each id contains pcl. text is represented as a vector and encoded using a pre-trained bert model. it mainly uses the capsule networks to extract the encoded vector, and uses the output of the fully connected layer to represent the label probability. the rest of the paper is organized as follows: section 2 introduces the system architecture. section 3 describes the dataset, implementation details settings and experimental results. the summary and outlook for future work will be presented in section 4. ## system architecture in this section, we will introduce the system architecture we use in the task, which will consist of two parts. one is embedding and coding, and the other is feature extraction and prediction. we call the model that is ultimately used to test the class pre-diction results of the dataset as bert-caps. the architecture of bert-caps model is shown in figure 1 . ## conclusion this paper describes the experiments conducted by team sapphire in subtask 1 of semeval-2022 task 4: patronizing and condescending language detection. we introduced the system architecture, experimental dataset situation and results in section 2 and 3, respectively. from the experimental results, the bert-caps model can achieve better results on the test set. in future work, we will improve our method to achieve better results. for example, using other text representations, and adjusting the weight of the loss function.