I 2 C - H uelva at S em E val-2023 Task 9: Analysis of Intimacy in Multilingual Tweets Using Resampling Methods and Transformers (2023)

Abstract: Nowadays, intimacy is a fundamental aspect of how we relate to other people in social settings. The most frequent way in which we can determine a high level of intimacy is in the use of certain emoticons, curse words, verbs, etc. This paper presents the approach developed to solve SemEval 2023 task 9: Multiligual Tweet Intimacy Analysis. To address the task, a transfer learning approach was conducted by fine tuning various pre-trained languagemodels. Since the dataset supplied by the organizer was highly imbalanced, our main strategy to obtain high prediction values was the implementation of different oversampling and undersampling techniques on the training set. Our final submission achieved an overall Pearson’s r of 0.497.

## introduction intimacy is defined as the set of thoughts and feelings that human beings keep private. there are many ways of expressing a thought and not all are equally intimate, so it is important to know what determines the degree of intimacy of a particular tweet. to detect this type of language, semeval 2023 proposed the task 9: multiligual tweet intimacy analysis @xcite . the aim of the proposed task is to quantify the intimacy expressed in a set of tweets written in six languages: chinese, english, spanish, portuguese, french and italian. we addressed the problem using regression @xcite and neural networks @xcite . we utilized different pre-trained models such as roberta @xcite or bert @xcite , available on the hugging face platform. given the strong imbalance in the data, we have focused on applying different oversampling @xcite and undersampling @xcite strategies. the main contributions addressed in this work are as follows: the rest of this paper is organized as follows: in section 2 we explain the dataset, the meaning of the labels and data distribution. therefore, we refer to other studies that contribute to our approach. section 3 discusses the approaches applied to achieve better model performance. in section 4, we explain the libraries used. in section 5, the results obtained in test phase are shown. finally, conclusions and future works are described in section 6. ## background the training dataset consists of 9491 tweets written in six languages. the distribution of each of the languages is balanced. figure 1 shows an example of a dataset row. ## system overview this section describes the development of our approach and outlines the steps taken to achieve the results. since only one training dataset was provided, for the experimental environment we decided to split the original dataset into a training/validation dataset (80%) and a test dataset (20%). ## experimental setup some of the libraries used for the experimentation were: "numpy" @xcite , a library for advanced natural language processing; "nltk" @xcite , a tool-kit to work with human language information; "keras", a neuralnetwork library; "scikit-learn", which contains simple and efficient tools for predictive data analysis; and "pandas" @xcite , used for manipulating data. the metric used is pearson's r from the scipy library as requested by the competition. for fine-tuning the models, the hyperparameters used were: batch size of 16, learning rate of 5e-5, max length of 128 and weight decay of 0.01, 10 epochs using early stopping with a patience of 3. ## results as can be seen in the previous sections, oversampling o4 was the best performing and most computationally efficient technique. therefore, we decided to try out other pre-trained multilingual models: xlm-roberta-large 8 @xcite , cardiffnlp/twitter-xlm-roberta-base-sentiment 9 , bert-base-multilingual-uncased 10 (devlin et al., 2018) and twitter/twhin-bert-base 11 @xcite . in the test phase, four new languages were included: arabic, korean, hindi and dutch. to predict the level of intimacy of these tweets with our models, they were translated into english. finally, the predictions obtained by twitter/twhin-bert-base model and the o4 technique were submitted. the pearson's r achieved for the official test dataset was 0.497. we were ranked 36 th in the overall intimacy score. in ## conclusions in this paper, the approach of the i2c group to address task 9: multiligual tweet intimacy analysis has been described. fine-tuning of several pre-trained monolingual and multilingual language models was performed and different resampling techniques were applied. the best results in the development phase were obtained by using a range oversampling (o4) technique. in this phase, the pearson's r achieved was 0.827 but, in the evaluation phase, the value of pearson's r was only 0.497. we think that the reason for the difference between the two values 10 https://huggingface.co/bert-base-multilingual-uncased 11 https://huggingface.co/twitter/twhin-bert-base could be due to the incorporation of new languages into the test dataset. for future works, we would like to add additional features to the tweets such as the number and type of emoticons, the use of certain adjectives, etc. we think that the intimacy language vocabulary is very specific, and this could help the prediction models.