ODA _ SRIB at S em E val-2023 Task 9: A Multimodal Approach for Improved Intimacy Analysis (2023)

Abstract: We experiment with XLM-Twitter and XLM-RoBERTa models to predict the intimacy scores in Tweets i.e. the extent to which a Tweet contains intimate content. We propose a Transformer-TabNet based multimodal architecture using text data and statistical features from the text, which performs better than the vanilla Transformer based model. We further experiment with Adversarial Weight Perturbation to make our models generalized and robust. The ensemble of four of our best models achieve an over-all Pearson Coefficient of 0.5893 on the test dataset.

## introduction intimacy is an important aspect in our society, which in general is not given much attention. with the increasing usage of social media across all age groups, users might post some private intimate information unintentionally. thus, there is a dire need to create natural language processing (nlp) models to detect the intimate nature of such posts as the user can be nudged before they share some intimate information on social media. moreover, the capability of understanding intimacy will also help us incorporate emotions in voice assistants and chatbots. for the task 9 of semeval 2023: multilingual tweet intimacy analysis @xcite , the organizers intend the participants to build models for detecting the intimacy of a given tweet on a scale of 1 to 5. they provide a dataset of annotated tweets in prominent languages used around the world -english, spanish, portuguese, italian, french and chinese. in order to validate the generalization of submitted systems, they also have test samples in hindi, dutch, korean and arabic. in this paper, we present our methodology to develop such models. we formulated the problem as a binary classification problem on soft labels. we rely on xlm-twitter @xcite and xlm-roberta @xcite models' cross lingual transfer capability for unseen languages. we devise a multi-modal architecture with text and handcrafted features as input. we experimented with augmented data which we created by translating the text while keeping the intimacy score the same. we incorporate adversarial weight perturbation (awp) @xcite to train our models in a robust manner. our final submission is a mean ensemble of four of our best models. pearson's coefficient is used to compare the models submitted to the task leaderboard. our submission attains an overall score of 0.5893 with 0.7318 on seen languages and 0.411 on unseen languages. ## background the shared task of multilingual tweet intimacy analysis is a text regression problem i.e. for a given a tweet, we need to predict the intimacy score for it. the organizer provide a dataset comprising of 13,372 tweets in 10 languages (english, french, spanish, italian, portuguese, korean, dutch, chinese, hindi, and arabic). the training data contains samples from english, french, spanish, italian, portuguese and chinese; the remaining languages are a part of the test data so as to evaluate the generalization of the developed models. intimacy, being an crucial aspect of our society, has been studied extensively in the field of socio-linguistics and social psychology. there are limited works on computational modeling of intimacy. pei and jurgens (2020) curate a dataset comprising questions, which reveal the people's impression of intimacy. by developing nlp models which correlate with human annotations, they analyze the questions posted on popular social media sites, books and movie dialogues, and present their insights. ## experimental setup we perform a 5-fold cross validation to train our models. the continuous label values can be binned to create ordinal buckets. since the objective of the shared task is to develop models that can perform well on unseen languages as well, the correct crossvalidation strategy is to use a stratified split grouped on language. since some language data will only be present in the validation split and missing in the training data, we will be able to get an accurate estimation of the generalization capability of the model on unseen language data. however, the details of the unseen languages in the test data is known and hence we can use a stratified split (without any grouping) and rely on the cross-lingual transfer of multilingual models for generalization across unseen languages. in this manner, we leverage the training data to the fullest. in some experiments, the augmented data is added to the training data of each fold. we train our models on a nvidia tesla p40 card with a batch size of 16 and sequence length 160. the models are optimized using adamw optimizer @xcite with a learning rate of 5e -6 for 50 epochs with an early stopping patience of 7 epochs. for models trained using awp, the awp learning rate is set to 1e -2 and awp is applied from the start of the training. the models have been implemented using pytorch @xcite and huggingface's transformers library @xcite . ## results and discussions in this section, we tabulate the performance of our models as well as the mean ensemble of the best four on the task test data, language wise. with the help of the ground truth data obtained from the organizers, we evaluate our best models individually on the test data ( in order to check the capability, we perform an additional experiment with an xlm-roberta large model based on the tabnet architecture ( ## conclusion in this paper, we present our methodology for estimating intimacy scores of tweets. we perform our experiments with xlm-twitter and xlm-roberta models. we also propose a multimodal model lever-