i REL at S em E val-2023 Task 9: Improving understanding of multilingual Tweets using Translation-Based Augmentation and Domain Adapted Pre-Trained Models (2023)

Abstract: This paper describes our system (iREL) for Tweet intimacy analysis sharedtask of the SemEval 2023 workshop at ACL 2023. Oursystem achieved an overall Pearson’s r score of 0.5924 and ranked 10th on the overall leaderboard. For the unseen languages, we ranked third on the leaderboard and achieved a Pearson’s r score of 0.485. We used a single multilingual model for all languages, as discussed in this paper. We provide a detailed description of our pipeline along with multiple ablation experiments to further analyse each component of the pipeline. We demonstrate how translation-based augmentation, domain-specific features, and domain-adapted pre-trained models improve the understanding of intimacy in tweets. The codecan be found at \href{https://github.com/bhavyajeet/Multilingual-tweet-intimacy}{https://github.com/bhavyajeet/Multilingual-tweet-intimacy}

## introduction social media platforms have become a ubiquitous source of communication, with millions of users generating and sharing content daily. among these, twitter has emerged as a popular platform for sharing short messages or tweets, which can express a wide range of emotions, opinions, and sentiments. @xcite such tweets can also provide valuable insights into various social and political issues, making them a valuable resource for researchers and policymakers alike. intimacy in language refers to the degree of emotional closeness or familiarity between individuals, which is often reflected in the choice of words, tone, and context of communication. @xcite . the degree of intimacy can vary based on the relationship between individuals, and it can significantly impact on the results of social interactions. in recent years, the analysis of intimacy in social media data has gained some attention @xcite , as it can provide valuable insights into various aspects of human behavior, such as the formation of social networks, the spread of information, and the dynamics of online communities. however, the analysis of intimacy in social media data is challenging, as it requires sophisticated methods for processing and interpreting large volumes of unstructured text data. multilingual tweet intimacy analysis is even more challenging, as it involves the complexity of multiple languages and cross-cultural communication. the task @xcite aims at quantifying the intimacy of tweets from a total of 10 different languages in the form of a score between 1 to 5. this paper describes our system used for analysing the intimacy of tweets belonging to 10 different languages. we utilise domain-specific features and domain-adapted pre-trained models in order to improve the understanding of intimacy in tweets. we further utilise a translation-based data augmentation pipeline which proves effective in significantly improving the scores for unseen languages. this paper also discusses multiple ablations applied to our pipeline in order to better understand the contribution of the various components used. our system achieved third rank for unseen languages with a pearson's r score of 0.485 and tenth rank overall with a pearson's r score of 0.592. ## related work intimacy can be defined as a particular form of "closeness" to another person founded on selfdisclosure, with the quality of self-disclosure characterized by knowledge and understanding of inner selves @xcite . several vital attributes are related to this social phenomenon. it occupies an important position in the hierarchy of human needs as one of the more basic needs @xcite . as such, several critical social attributes and functions are associated with it, such as in development @xcite and well-being @xcite . as this role of prominence would suggest, it plays an integral part in the discourse. this needs to be accounted for in computation methods of textual analysis. generally, social factors are challenging to interpret for nlp systems, resulting in errors. methods incorporating social factors have been shown to result in improvements in the performance of such models @xcite . intimacy specifically can indicate close relations between the participants @xcite or be used to reduce the social distance between the participants @xcite few works have investigated computational methods of dealing with intimacy. @xcite ) establish intimacy as an impactful social dimension of language by analyzing a dataset collected from reddit, an online forum. recent works have also discussed the idea of using translation based augmentation @xcite . these works argue that such augmentation techniques can help the multilingual deep learning models better transfer learnings across languages. our work utilises the understandings and inferences from previous research works of this domain to construct a novel pipeline for analysing intimacy in multilingual tweets. as it can be seen, the distribution of intimacy scores for hindi and korean are very different from the rest of the languages, which can be a potential reason for the relatively worse performance of the finetuned transformer models on these languages. ## methodology our approach utilises sentence representation from bert-based models, along with components to address the domain-specific features like emojis and augmentation techniques for better transfer of cross-lingual learning. this section describes the entire pipeline of our system. our best-performing model contains the following components. ## results the first column shows the scores achieved by the submitted system. the following five groups of columns show results achieved in the different verticals of ablations. for each experiment, the entire setup is same except for the modifications in the ablation vertical. for all the experiments, the default transformer model used is xlm-t and the default pipeline setup is the same as described in section 4 the first group of ablations studied the effect of changing the pre-trained transformer model in the pipeline. the second group focuses on the effects of data preprocessing and cleaning. the third group shows the results with two strategies of processing the emojis, one where emojis are removed, and one where we only convert emojis to text without using the emoji2vec system. the fourth group shows the effects of not augmenting data for unseen languages using translation (no trans), or the effect of translating the unseen languages to english for inference (trans test). finally, the last column shows the results without the use of emojis and without the translation-based augmentation. our submitted system achieved an overall score of 0.592, ranking tenth on the leaderboard, and a score of 0.485 for unseen languages, ranking third on the leaderboard. the submitted system and our best-performing system (tweet-bert with all the components as described in section 4) only differ in terms of hyperparameters which are explained in detail in section 7 ## ablation study this section describes multiple experiments done in order to better understand the contributions from the different components of our pipeline. for each of the experiments, the basic pipeline remains the same, and each ablation study removes or modifies a specific component of the pipeline to understand its contribution and impact. unless stated otherwise, we use xlm-t model as the underlying decoder for all our experiments. ## implementation details all the models are trained on a single nvidia gtx 1080 ti gpus. for all our experiments, we use a learning rate of 1e-5 and a dropout of 0.3. we optimize our models on the mean squared error (mse) loss using adamw (kingma and ba, 2014) optimiser. ## conclusion this work shows how carefully designed data augmentation techniques can help in better cross lingual transfer of learning and improved scores over unseen languages. the work also highlights the importance of using efficient encoding strategies to include domain specific features like emojis for an improved understanding of the text. the results also show that, though efficient, the transformer based deep learning models are prone to variance, and just utilising the right set of hyperparameters can result in significant gains. finally, we also see that domain-adapted pre-trained transformers can capture nuances of in-domain text, and when used with simple deep learning and machine learning models, could give competitive results.