A li E dalat at S em E val-2022 Task 4: Patronizing and Condescending Language Detection using Fine-tuned Language Models, BERT + B i GRU , and Ensemble Models (2022)

Abstract: This paper presents the AliEdalat team’s methodology and results in SemEval-2022 Task 4: Patronizing and Condescending Language (PCL) Detection. This task aims to detect the presence of PCL and PCL categories in text in order to prevent further discrimination against vulnerable communities. We use an ensemble of three basic models to detect the presence of PCL: fine-tuned bigbird, fine-tuned mpnet, and BERT+BiGRU. The ensemble model performs worse than the baseline due to overfitting and achieves an F1-score of 0.3031. We offer another solution to resolve the submitted model’s problem. We consider the different categories of PCL separately. To detect each category of PCL, we act like a PCL detector. Instead of BERT+BiGRU, we use fine-tuned roberta in the models. In PCL category detection, our model outperforms the baseline model and achieves an F1-score of 0.2531. We also present new models for detecting two categories of PCL that outperform the submitted models.

## introduction increasing internet access rates and the development of a diverse range of online forums have allowed people around the world to engage in a tremendous range of topics. this has been accompanied by an increase in unhealthy online texts whose negative effects on people have been significant. one type of such unhealthy texts is a text with patronizing and condescending language (pcl). when a person's language expresses a superior attitude towards others or describes their situation in a benevolent way that creates a sense of pity, the person has used this type of language. in the media, vulnerable communities seem to be a good target for this type of language. however, this type of language can normalize discrimination. we believe that unfair treatment of vulnerable groups leads to greater deprivation and inequality for these groups. therefore, recognizing the existence of this type of language and its variations is important and can prevent these problems. so far, significant work has been done on modeling the language that deliberately and openly undermines others, such as offensive language or hate speech, but little has been done on the language of humiliation and pity. this language of humiliation and pity is used in the media subtly and indirectly and different from other types of unhealthy languages. the special focus on the language of humiliation and compassion for vulnerable communities has been noted only in the work of @xcite . in this work, a dataset to identify this type of language is presented, but no significant work has been done in designing a model to classify this type of text. unhealthy text papers usually focus on obvious and aggressive phenomena such as detecting fake news, fact-checking, modeling offensive language, and spreading rumors. there has been a few work on pcl recently. @xcite introduced compassion modeling in direct communication from the perspective of natural language analysis. they created and tagged a dataset with social media messages. @xcite discussed the specific uses of language and power, especially the unbalanced power relations often present in degrading treatment, and the social consequences of these applications. unfair treatment of disadvantaged groups was also examined as an example of these cases. @xcite provided datasets for classifying unhealthy speech on social media. they provided fine-grained classifications for all kinds of unhealthy writings, one of which was pcl. of course, the use of this type of writing is not limited to weak groups in society. there is still a need to design a model to detect such language towards vulnerable communities. pcl is a toxic language that implicitly has a negative impact on public opinion. there are tasks that generally identify toxic language that can be used to provide an answer to this problem. for example @xcite proposed the use of a fine-tuned bert model to detect veiled toxicity. to design a model to detect this language in vulnerable communities, we participated in semeval 2022 task 4 competition @xcite . this paper describes the models we provide for detecting pcl. the contest data is taken from the work of pérez-almendros et al., (2020). to detect the presence of pcl, we present an ensemble model. this model consists of three basic models: fine-tuned bigbird, fine-tuned mpnet and bert + bigru. the models are combined with the weighted average. this model cannot perform better than the baseline because our model is overfitted. in this paper, we present a solution to this model's problem and create a new model that can achieve an f1-score of 0.5505. to identify existing pcl categories, we consider the categories separately. to detect the presence of any category, we act like detecting the presence of pcl. the basic models for making ensemble models are finetuned bigbird, fine-tuned mpnet and fine-tuned roberta. the model can outperform the baseline model. in addition to this model, in two categories, we improve the diagnostic model and build a new model. this model can achieve an f1-score of 0.3160. the statement of contributions is given below. to balance the data set, we used a different method than the data set providers. instead of using the sampling method (we sample twice the number of pcl data from non-pcl data), we used a combination of the sampling method and eda @xcite . and in compassion and metaphor diagnosis, we used a set of related articles for balancing. we paid attention to the medium and high lengths of the texts and used language models with the ability to summarize long texts. we used the ensemble model for classification to help reduce the bias caused by the data set imbalance. ## models in this section, we describe how to detect the presence of pcl in the text and how to detect the type of pcl in the text. we describe the models used for these diagnoses. ## experimental setup 1 in this section, the structural details of the base models are given. all models are trained on the gpu of google colab 2 in normal account mode. ## conclusion in this paper, we presented models for two tasks. for task 1, we presented an ensemble model consisting of three basic models. we reviewed the results of this model in the competition. we examined the weaknesses of this model and presented another model with a similar structure that performed better on the test data. for task 2, we considered identifying each category separately from the other categories. we provided a model to identify each category. we examined the result of our prediction based on these models in the competition and identified weaknesses. by solving these cases, we changed the classification model of the two categories. we were able to come up with a new prediction for test data that would have a better result than our original model. using our second model resulted in better ranks in the testing phase. in future work for the second task, the categories can be considered related. this is because some categories have a common concept. the extracted features according to other categories, can be used to classify a category. in future work, other ways can be proposed to solve the problem of class imbalance. for example, constructor models can be used to create text for a class on a conditional basis.