Zhegu at S em E val-2023 Task 9: Exponential Penalty Mean Squared Loss for Multilingual Tweet Intimacy Analysis (2023)

Abstract: We present the system description of our team Zhegu in SemEval-2023 Task 9 Multilingual Tweet Intimacy Analysis. We propose \textbf{EPM} (\textbf{E}xponential \textbf{P}enalty \textbf{M}ean Squared Loss) for the purpose of enhancing the ability of learning difficult samples during the training process. Meanwhile, we also apply several methods (frozen Tuning \&amp; contrastive learning based on Language) on the XLM-R multilingual language model for fine-tuning and model ensemble. The results in our experiments provide strong faithful evidence of the effectiveness of our methods. Eventually, we achieved a Pearson score of 0.567 on the test set.

## introduction text intimacy is considered as the essential component in emotional communication and social relationships @xcite @xcite . at the same time, intimacy exists in any language in the world. however, both data sets and methods of this area are still rare. valuable research on the multilingual regression task is necessary to facilitate the sentiment analysis of language. to further investigate the intimacy in text, @xcite propose the mint, a new multilingual intimacy analysis dataset. in this case, we describe the model system of our team zhegu in semeval-2023 task 9. since the mean of intimacy is different from the "positive" or "negative" in sentiment analysis, it is difficult to have a clear definition. there exist many hard samples, which can not be learned by the model easily. @xcite focal loss for learning the hard samples for model only in classification. therefore, we present epm (exponential penalty mean squared loss) for the regression task, which set a penalty term for hard samples during calculating the training loss to accelerate the * *corresponding author model focus on those hard samples. we also construct a language-based contrastive loss function to draw closer the distance of the text in the same language and distinguish in the different languages. the test set in this task includes four new languages text data additionally, which do not exist in the training data. in order to achieve better zero-shot inference on new languages data, we also adopt the frozen tuning to fine tune the model, which means freezing some parameters of the model for training. all in all, the main contributions of our paper are as follows. all methods we mentioned above will be presented in detail in the section 3 and will be validated in section 4. in section 5, we will give the conclusion of our research and future work. ## dataset the data sets of task 9 is divided into the labeled set and the unlabeled testing set. some samples of the labeled set are shown in according to ## methodology in this section, we describe in detail the methods we used on task 9. we first introduce a loss function for regression task we proposed, and we named it epm. after that, we present contrastive loss and frozen tuning. the reasons for their ability to improve model performance will be described in detail. and we take xlm-roberta-large @xcite as the backbone of our model. ## experiments & results in this section, we describe the experimental procedures and results in detail. the evidence of those methods we mentioned above will be provided by results and analysis. xlm-r p n n n n p n n n p p n p n n n n p p n p n n n n n n p n n n p n p p n p n n n p n n n n p n n n n n n n n p n n n n p n n n p (𝑥 2 , 𝑙 2 = 𝑒𝑛) ## conclusion we introduce the system description of our team zhegu in semeval-2023 task 9. we first propose a loss function, we named emp (exponential penalty mean squared loss), which could increase the gradient of those hard samples by adding the penalty term. epm could be used in any regression task. and we conduct a contrastive learning task based on languages to reduce the distance of those in the text from the same languages. the experiments prove the effectiveness of our proposed methods. meanwhile, epm could avoid the disappearance of gradient than mse when we conduct contrastive learning. frozen tuning is also useful for improving the performance. in future work, the effectiveness of epm should also be proved in others natural language regression tasks.