DUTH at S em E val-2023 Task 9: An Ensemble Approach for T witter Intimacy Analysis (2023)

Abstract: This work presents the approach developed by the DUTH team for participating in the SemEval-2023 Task 9: Multilingual Tweet Intimacy Analysis. Our results show that pre-processing techniques do not affect the learning performance for the task of multilingual intimacy analysis. In addition, we show that fine-tuning a transformer-based model does not provide advantages over using the pre-trained model to generate text embeddings and using the resulting representations to train simpler and more efficient models such as MLP. Finally, we utilize an ensemble of classifiers, including three MLPs with different architectures and a CatBoost model, to improve the regression accuracy.

## introduction intimacy analysis is concerned with identifying and analyzing the level of emotional closeness between individuals using natural language @xcite . this form of analysis has a variety of applications in domains such as social media, healthcare and marketing, providing valuable insights into user behaviour, sentiment and preferences @xcite . despite the significance of intimacy as a reflection of social norms @xcite @xcite , there is limited availability of resources for analyzing it using machine learning techniques. to address this issue, @xcite processed and prepared a multilingual dataset consisting of tweets in ten languages, including english, spanish, french, portuguese, italian, chinese, hindi, korean, dutch, and arabic. this dataset has been made publicly available to evaluate the performance of computational tools that analyse intimacy. this paper presents an ensemble intimacy regression scheme based on an extensive experimental study of several regressor models @xcite @xcite ) and text-to-feature representations @xcite @xcite . our study shows that training simple models using text embeddings generated by transformer-based models leads to effective and rapid training and accurate results. our models achieve state-of-the-art performance without fine-tuning a huge and trainingcostly transformer-based model. we carefully optimized the components of our final ensemble, considering the influence of the imbalanced regression task. our team achieved a rank of 7 in the overall score, 25 using the languages seen during training, and 2 on cross-language transfer learning. the remainder of this work is structured as follows. the proposed system is described in section 2. section 3 reports on our experiments. finally, conclusions and future directions are discussed in section 4. ## system description the primary objective of semeval-2023 task 9 was to assign a continuous value within the range of @xcite to multilingual tweet texts, representing their intimacy level. the higher the value, the higher the intimacy level. this section presents a detailed description of our system, elucidating the text-tofeature generation methods used. in addition, we provide statistics about the given dataset. ## experiments this section presents the results from utilizing the pre-processing techniques, vectorizers, and machine learning models discussed in section 2. additionally, we compare our findings with the corresponding reported results obtained from fine-tuned transformers. unless stated otherwise, all experiments used 5-fold cross-validation after shuffling the dataset to improve generalisation. ## conclusion twitter provides a platform for users to express personal opinions and share emotions, making it an attractive research area for analyzing intimacy and social norms. this work presents an approach for twitter sentiment analysis that involves training simple and efficient models using embeddings generated by pre-trained transformers. our approach is comparable to fine-tuning large transformer models and proposes that the complex and challenging task of fine-tuning transformer models can be replaced with more straightforward and highly efficient models trained using embeddings derived from pre-trained transformers. this conclusion could potentially lead to significant improvements in the efficiency and effectiveness of natural language processing and machine learning applications. a limitation of the presented method for intimacy analysis is that it does not address the issue of imbalanced regression tasks, which could compromise the quality of the results. future work in this domain could appropriately address the imbalanced regression task to achieve higher-quality outcomes.