XLM - EMO : Multilingual Emotion Prediction in Social Media Text (2022)

Abstract: Detecting emotion in text allows social and computational scientists to study how people behave and react to online events. However, developing these tools for different languages requires data that is not always available. This paper collects the available emotion detection datasets across 19 languages. We train a multilingual emotion prediction model for social media data, XLM-EMO. The model shows competitive performance in a zero-shot setting, suggesting it is helpful in the context of low-resource languages. We release our model to the community so that interested researchers can directly use it.

## introduction emotion detection is an important task for natural language processing and for affective computing. indeed, several resources and models have been proposed @xcite @xcite @xcite for this task. these models can be used by social and computational scientists @xcite @xcite to better understand how people react to events through the use of social media. however, these methods often require large training sets that are not always available for low-resource languages. nonetheless, multilingual methods (wu and dredze, 2019) have risen across the entire field showing powerful few-shot and zero-shot capabilities @xcite . in this short paper, we introduce a new resource: xlm-emo. xlm-emo is a model for multilingual emotion prediction on social media data. we collected datasets for emotion detection in 19 different languages and mapped the labels of each dataset to a common set {joy, anger, fear, sadness} that is then used to train the model. we show that xlm-emo is capable of maintaining stable performances across languages and it is competitive against language-specific baselines in zero-shot settings. we believe that xlm-emo can be of help to the community as emotion prediction is becoming an interesting and relevant task in nlp; the addition of a multilingual model that can perform zero-shot emotion prediction can be of help for many lowresource languages that still do not have a dataset for emotion detection. contributions we release xlm-emo which is a multilingual emotion detection model for social media text. xlm-emo shows competitive zeroshot capabilities on unseen languages. we release the model in two versions a base and a large to adapt to different possible use-cases. we make the models foot_0 and the code to train it freely available under a python package that can be directly embedded in novel data analytics pipelines. foot_1 ## data and related work we surveyed the literature to understand which datasets are available in the literature and with which kinds of emotions. details on how we operate on this data can be found in the appendix, here we give an overview of the transformation pipeline we have adopted and which datasets have been included. the datasets we have collected and used in this paper are presented in we describe here the general guidelines we have used to create this dataset, readers can find details for each dataset in the appendix. for all the datasets we removed the emotions that are not in the set joy, anger, fear, sadness (e.g., @xcite , @xcite , @xcite used the 27 emotions from goemotion @xcite and we just collected the subset of our emotions). we have some exceptions to twitter data, as the tamil dataset @xcite contains youtube comments. some data was impossible to reconstruct because the tweets do not exist anymore and thus only a subset is still available (e.g., @xcite ). for some languages, we decided to apply undersampling in order to limit the skewness of the final distribution (e.g., both @xcite and @xcite provide dozens of thousands of tweets). to simplify reproducibility, we will release the exact data extraction scripts that we have used to collect our data. there are papers that we have not included in our research: @xcite introduce a hindi dataset that contains hindi-english code switched text. however, hindi is romanized and only a few of this data has been used to pre-train xlm. @xcite released a collection of persian tweets annotated with emotions, however, their data has not been evaluated in a training task and thus we decided not to include it in our training. we also found a dataset for japanese @xcite , however, the dataset is not publicly available. french and german are collected through the translation of spanish @xcite tweets using deepl. 3 for chinese, we use the messages found in the nlpcc dataset @xcite . note that this dataset has some internal code-switching. the most similar work to ours is the work by @xcite . @xcite introduces a dataset collected through distant supervision on facebook and covers 6 main languages for training and a set of 12 other languages that can be used for testing. we will run a 3 we are aware that this process might introduce bias in the model as described by @xcite comparison with this model in section 3.3. ## experiments we perform three different experiments. the first one is meant to show the performance of xlm-emo across the different languages. the second one evaluates how well xlm-emo works on a zero-shot task in which data from one language is held out; we focus on testing three languages: english, arabic, and vietnamese. the third evaluation shows the performance of xlm-emo on additional datasets different from those used for training on which we compare our model with other state-of-the-art models. ## limitations unfortunately, we have not been able to find datasets for emotions detection in any of the african languages. moreover, automatic translation tools do not often cover african languages or @xcite ) is trained on this data. they do not provide reliable evidence of being able to provide those translations with a certain level of quality. we reached out to members of our community to understand if there was any work that we were not aware of but we did not find any. further iterations of this resource might want to focus on those languages. ## conclusion in this short paper, we propose xlm-emo, a novel resource for emotion detection. the model shows stable performance across 19 languages and it is competitive in a zero-shot setting, supporting its usage in low-resource contexts. we plan to enrich this model with more languages as soon as we find them so that we can continually improve these results and offer better methods to the community.