YNU - HPCC at S em E val-2022 Task 4: Finetuning Pretrained Language Models for Patronizing and Condescending Language Detection (2022)

Abstract: This paper describes a system built in the SemEval-2022 competition. As participants in Task 4: Patronizing and Condescending Language Detection, we implemented the text sentiment classification system for two subtasks in English. Both subtasks involve determining emotions; subtask 1 requires us to determine whether the text belongs to the PCL category (single-label classification), and subtask 2 requires us to determine to which PCL category the text belongs (multi-label classification). Our system is based on the bidirectional encoder representations from transformers (BERT) model. For the single-label classification, our system applies a BertForSequenceClassification model to classify the input text. For the multi-label classification, we use the fine-tuned BERT model to extract the sentiment score of the text and a fully connected layer to classify the text into the PCL categories. Our system achieved relatively good results on the competition’s official leaderboard.

## introduction text classification is an area of natural language processing (nlp) that aims to classify text using certain features. previous studies on text classification tasks used traditional machine learning methods, which require researchers to manually design features. feature extraction methods such as term frequency-inverse document frequency (tf-idf) @xcite and n-gram @xcite are used to extract features from original documents, and then the features are input into classifiers such as naive bayes @xcite , support vector machines (svms) @xcite , and decision trees @xcite . since the advent of deep learning, text classification tasks are achievable without manual extraction of text features. researchers must simply pretreat the text and incorporate it into a deep learning model for training. for text classification using deep learn-ing methods, the classification accuracy is often higher than that of traditional machine learning methods. with their continuous improvement, deep learning models, such as recurrent neural networks (rnns) @xcite , multi-channel cnn-lstm @xcite ,gate recurrent units (grus) @xcite , long short-term memory (lstm) @xcite , bidirectional long shortterm memory (bi-lstm) @xcite , and attention-based bi-lstm @xcite networks, can be used to solve text classification problems. in recent years, bidirectional encoder representations from transformers (bert) @xcite , a new deep learning model, has achieved, or even surpassed, human performance in multiple tasks within the nlp domain, including text classification. task 4 of the semeval-2022 consists of the following two subtasks. in this paper, we introduce a deep learning system for semeval-2022 task 4: patronizing and condescending language detection @xcite . we applied the pretrained bert model as the base model. this task contains two subtasks: single-label classification and multi-label classification. to accomplish both subtasks, we used fine-tuning methods on the base model with an additional classification layer. our contributions are as follows: the remainder of this paper is organized as follows. section 2 provides an overview of our system for the two subtasks. section 3 presents the specific details of our system. section 4 discusses the results of the experiments, and finally, we draw our conclusions in section 5. ## overview this section presents an overview of our system and experiments, consisting of the following steps: task description. the two subtasks involved text sentiment analysis and classification. the difference between them is that subtask 1 only requires us to determine whether the text contains any kind of pcl. subtask 2 is the multilabel classification task, and the data of subtask 2 are marked by a list of 0s and 1s, which indicate the type of linguistic techniques (unbal-anced_power_relations, shallow_solution, presupposition, authority_voice, metaphors, compassion, the_poorer_the_merrier) used to express condescension. ## model description a pretrained bert model is used to accomplish both subtasks with the two independent datasets. the details of the model built for these two subtasks are as follows. ## experiment dataset. for the two subtasks, the corpus we used to train the model are from the competition(pérez-almendros et al., 2020), without other external data. dontpatronizeme_pcl.tsv this dataset contains 10,469 paragraphs, and each paragraph is annotated with a label ranging from 0 to 4. in the single-label classification subtask, the original label annotated as either 0 or 1 is replaced with 0, and the other labels with 1. dontpatronizeme_categories.tsv this dataset contains 993 unique paragraphs with a total of 2,760 instances of pcl. in the multi-label classification task, each paragraph is annotated with 7 labels ranging from 0 to 1. results. the results of the two subtasks are shown in tables 1 and 2 . for subtask 1, we ranked 42/81 in precision, 47/81 in recall, and 52/81 in f1 score. ## conclusion in this paper, we described our system, which is based on the pretrained bert model, for the text classification task semeval 2022 task 4: patronizing and condescending language detection. we added a classification layer to the pretrained bert model to address both subtasks. the results generated by the proposed system achieved a relatively good ranking. in the future, we hope to explore other models and methods in the sentiment analysis field.