BEIKE NLP at S em E val-2022 Task 4: Prompt-Based Paragraph Classification for Patronizing and Condescending Language Detection (2022)

Abstract: PCL detection task is aimed at identifying and categorizing language that is patronizing or condescending towards vulnerable communities in the general media. Compared to other NLP tasks of paragraph classification, the negative language presented in the PCL detection task is usually more implicit and subtle to be recognized, making the performance of common text classification approaches disappointed. Targeting the PCL detection problem in SemEval-2022 Task 4, in this paper, we give an introduction to our team’s solution, which exploits the power of prompt-based learning on paragraph classification. We reformulate the task as an appropriate cloze prompt and use pre2trained Masked Language Models to fill the cloze slot. For the two subtasks, binary classification and multi-label classification, DeBERTa model is adopted and fine-tuned to predict masked label words of task-specific prompts. On the evaluation dataset, for binary classification, our approach achieves an F1-score of 0.6406; for multi-label classification, our approach achieves an macro-F1-score of 0.4689 and ranks first in the leaderboard.

## introduction patronizing and condescending language (pcl) towards vulnerable communities in the media has become a hot issue, which is heatedly discussed in society at present. the language often refers to the discourse which is published with a mixture of pity and superiority towards unprivileged groups. such attitude is always adopted unconsciously or even out of kindness, however, its negative effect indeed exacerbates social prejudice and routinizes discrimination towards disadvantaged people @xcite . despite the pcl emerging in numerous media, its style can not be precisely captured by far, since the usage of pcl is commonly unintended, * corresponding author unaffected, and subjective. this poses many challenges to the detection of pcl, which attracts great attention from the nlp community. even though there exist substantial nlp studies on paragraph identification in various fields, research specifically for pcl identification has not been yet seriously introduced @xcite . nowadays with social discrimination continuously rising, an effective approach, which can automatically identify pcl towards vulnerable communities, becomes more and more necessary and important to our society. to encourage more research on the pcl problem, semeval-2022 task 4 (pérez-almendros et al., 2022) provides an english pcl dataset for language-modeling study and evaluation. the main task contains two pcl-classification subtasks, one for binary classification (subtask 1) and the other for multi-label classification (subtask 2). given a paragraph, subtask 1 is aimed to identify whether or not it is written with pcl style. f1 over the positive class is taken as the evaluation metric of this task. for the same paragraph, the goal of sub-task 2 is to classify which specific pcl categories it belongs to. in detail, there are 7 different pcl types needed to be recognized, and one paragraph can have up to 7 pcl labels at the same time. for evaluation, subtask 2 introduces macro f1 over the 7 classes as the metric. in this paper, we propose our approach targeting pcl detection, which is submitted to semeval-2022 task 4. to tackle the sparsity and implicity issues of pcl language, we first reformulate the task as a specific form of cloze prompt, and then apply prompt-based learning on it to predict the appropriate label words and corresponding task labels. to take advantage of the pre-trained language model (lm), deberta @xcite ) is adopted as our based model, which improves bert @xcite by introducing two novel techniques: disentangled attention and decoding enhanced masking. the same classification method and model architecture are used for both subtasks to train and predict. experiments are conducted to show the effectiveness of our approach. at last, during the evaluation phase, our approach achieves an f1 score of 0.6406 on subtask 1 (rank 4th on the leaderboard), and a macro-f1 score of 0.4689 on subtask 2 (rank 1st on the leaderboard). ## related work condescending and patronizing treatment is a controversial social problem, which attracts attention from researchers of various fields @xcite . to the nlp community, although extensive work on detecting different kinds of harmful language is presented, the targeted language styles are often explicit, aggressive, and flagrant, which can be perceived obviously. unlike the previously studied language, the style of pcl language can not be easily sensed, since the usage of pcl is commonly unintended, unaffected, and subjective, which makes it a challenging identification problem. in recent years, more and more researchers @xcite start researching this emergent nlp topic, but the room for progress is still large. paragraph classification is a basic and important nlp task, which has been continuously studied by industry and academia. applications based on paragraph classification are increasingly permeating our lives, such as spam filtering @xcite ) and text sentiment analysis @xcite . traditional classification models, such as svm @xcite and xgboost @xcite , predict the category of paragraph mainly based on the statistical features extracted from raw text data. with the appearance of bert @xcite , many researchers @xcite achieve success on paragraph classification, by utilizing the contextualized word vectors of pre-trained language models. in recent years, with the development of promptbased learning @xcite , researchers start reformulating the paragraph classification problem as a masked language modeling problem. unlike the traditional models, prompt-based models first predict the label word of mask in a prompt and then map the label word to the label of category by a verbalizer. thanks to the power of pre-trained language models, prompt-based models @xcite demonstrate their strength in performing fewshot or even zero-shot learning on the scenarios of paragraph classification with few or no labeled data. the above studies have shown that prompt-based learning is highly effective to solve the problem of paragraph classification. to study the effect of prompt-based learning on identifying paragraphs of pcl, in the rest of this paper, our modeling method and experimental analysis are introduced. ## our approach in this section, we present our approach to utilize prompt-based learning for pcl detection. we first give the overall paradigm of our model and then introduce the ensemble strategy used in this task. ## conclusion in this paper, we propose a prompt-based learning approach for pcl detection, based on pre-trained language models. to reformulate the pcl detection problem into a masked language modeling problem, how our prompt template is designed have been fully discussed. we also introduce several other techniques, which have a positive impact on prompt-based learning. through experimental comparison, our approach is proven as an effective solution to detect pcl language. as a result, in semeval-2022 task 4, our approach ranks 4th on the leaderboard of subtask 1, and ranks 1st on the leaderboard of subtask 2.