Multilingual Emoticon Prediction of Tweets about COVID -19 (2020)

Abstract: Emojis are a widely used tool for encoding emotional content in informal messages such as tweets,and predicting which emoji corresponds to a piece of text can be used as a proxy for measuring the emotional content in the text. This paper presents the first model for predicting emojis in highly multilingual text. Our BERTmoticon model is a fine-tuned version of the BERT model,and it can predict emojis for text written in 102 different languages. We trained our BERTmoticon model on 54.2 million geolocated tweets sent in the first 6 months of 2020,and we apply the model to a case study analyzing the emotional reaction of Twitter users to news about the coronavirus. Example findings include a spike in sadness when the World Health Organization (WHO) declared that coronavirus was a global pandemic, and a spike in anger and disgust when the number of COVID-19 related deaths in the United States surpassed one hundred thousand. We provide an easy-to-use and open source python library for predicting emojis with BERTmoticon so that the model can easily be applied to other data mining tasks.

## introduction the covid-19 pandemic has caused intense emotional reactions on social media. some tweets are sad: and other tweets are angry: i saw that bottles of purell were selling for $149! go away price gougers and go away coronavirus! what both of these tweets have in common is that their emotional content is captured by emojis present in the tweet's text. emojis are often used in informal tweets sent between friends @xcite , but most tweets do not contain emojis. for example, the following tweet by the bbc (a major british newspaper) is clearly meant to help us find joy amidst the stress of covid-19: father dresses as transformers character bumblebee to surprise his son on his first day back at school after lockdown. but there are no emojis in the text to indicate that the tweet is joyful. one possible emoji for this tweet would be the "grinning face" ( ), but more subtle emojis like "grinning face with tongue" ( ) or "grinning face with smiling eyes" ( ) would also be appropriate and convey slightly different emotions. the goal of this paper is to automatically annotate these emoji-less tweets with appropriate emojis in order to better understand the emotional content in online discussions of covid-19. holen sie sich ihre gesichtsmasken leute! #coronavirus #wuhan #mask hebrew ‫א‬ ‫ח‬ ‫ו‬ ‫ל‬ ‫ה‬ ‫ב‬ ‫ו‬ ‫ו‬ ‫ש‬ ‫י‬ ‫נ‬ ‫ג‬ ‫ט‬ ‫ו‬ ‫ן‬ ‫ה‬ ‫ו‬ ‫א‬ ‫ה‬ ‫ר‬ ‫א‬ ‫ש‬ ‫ו‬ ‫ן‬ ‫ב‬ ‫א‬ ‫ר‬ ‫ה‬ " ‫ב‬ ‫כ‬ ‫ד‬ ‫י‬ ‫ל‬ ‫ת‬ ‫פ‬ ‫ו‬ ‫ס‬ ‫ד‬ ‫ל‬ ‫ק‬ ‫ת‬ ‫ר‬ ‫י‬ ‫א‬ ‫ו‬ ‫ת‬ ‫מ‬ ‫ס‬ ‫ו‬ ‫כ‬ ‫נ‬ ‫ת‬ ‫ש‬ ‫ה‬ ‫ת‬ ‫ג‬ ‫ל‬ ‫ת‬ ‫ה‬ ‫ל‬ ‫א‬ ‫ח‬ ‫ר‬ ‫ו‬ ‫נ‬ ‫ה‬ . ‫ת‬ ‫ו‬ ‫צ‬ ‫י‬ ‫א‬ ‫ו‬ ‫א‬ ‫ת‬ ‫ה‬ ‫מ‬ ‫ס‬ ‫כ‬ ‫ו‬ ‫ת‬ ‫ב‬ ‫פ‬ ‫נ‬ ‫י‬ ‫ם‬ ‫ש‬ ‫ל‬ ‫כ‬ ‫ם‬ , ‫ח‬ ‫ב‬ ‫ר‬ ' ‫ה‬ ! ‫נ‬ ‫ג‬ ‫י‬ ‫ף‬ ‫ה‬ ‫ק‬ ‫ו‬ ‫ר‬ ‫ו‬ ‫נ‬ ‫ה‬ ‫ו‬ ‫ו‬ ‫ה‬ ‫א‬ ‫ן‬ ‫מ‬ ‫ס‬ ‫כ‬ ‫ה‬ indonesian orang sakit washington adalah yang pertama di as untuk menangkap pneumonia berbahaya yang baru ditemukan. keluarkan masker wajah kalian! #coronavirus #wuhan #mask italian un malato di washington è il primo negli stati uniti a contrarre una polmonite pericolosa scoperta di recente. tira fuori le tue maschere per il viso, gente! #coronavirus #wuhan #mask japanese ワシントンの病人は新しく発見された危険な肺炎を捕まえるために米国で最初です。フェイスマスクの人を出してくださ い！ ＃コロナウイルス＃武漢＃マスク portuguese homem doente em washington é o primeiro nos estados unidos a pegar pneumonia perigosa recém-descoberta. tirem suas máscaras, pessoal! #coronavirus #wuhan #mask spanish un enfermo de washington es el primero en los estados unidos en contraer una neumonía peligrosa recién descubierta. ¡saquen sus mascarillas, amigos! #coronavirus #wuhan #mascara tagalog ang taong may sakit sa washington ay ika-1 sa estados unidos upang makuha ang bagong nakatuklas na mapanganib na pneumonia. lumabas ang iyong mga maskara sa mukha mga kamag-anak! #coronavirus #wuhan #mask prior work on predicting emojis from the text of a tweet @xcite @xcite has focused only on english language tweets. models submitted for the semeval 2018 task 2 @xcite are the most multilingual emoji prediction models currently published, but this task considered only english and spanish tweets. because covid-19 is a worldwide phenomenon, however, to understand emotional responses to covid-19, we must be able to predict emojis in all languages used on twitter. we therefore introduce the first highly multilingual model for emoji prediction, which we call bertmoticon. our model is based on fine-tuning the multilingual bert model @xcite , which was trained on a dataset of 102 distinct languages. figure 1 shows the output of our model on a tweet translated into ten different languages. a large body of work has emerged analyzing tweets about covid-19. one prominent line of research attempts to identify how misinformation about the disease spreads online @xcite @xcite @xcite ). an important subcategory of this research investigates the spread of racist (budhwani and sun, 2020, @xcite and ageist @xcite misinformation. other research more similar to our own investigates the sentiment of tweets about the coronavirus. some of this research focuses on specific locations such as belgium @xcite , paris (saire and cruz, 2020), poland @xcite or india @xcite . other research studies english-language tweets (rajput et al., 2020, yin et al., 2020) over wider geographic areas. our research stands out from this prior work in two important ways. first, we do not consider a subset of tweets about covid-19, we consider all tweets, written in all languages, sent from anywhere in the world. this is a significantly more challenging technical problem than previous research addressed, but it is also much more useful. second, we are the first paper to consider the more general emoji prediction problem rather than the sentiment prediction problem. in sentiment prediction, the goal is to assign a positive or negative sentiment to each tweet, and for the coronavirus topic it can be difficult to decidedly assign one sentiment. tweets about covid-19 can express negative sentiments because the disease has killed millions of people and forced us to make drastic changes to our lifestyles but also can contain funny, uplifting content exhibiting a positive sentiment such as: happy #nationalcatday from my beautiful adopted pandemic pet! in the emoji prediction task, we are able to get a more fine-tuned emotional understanding of tweets. for example, we can answer questions like: is the tweet sad? angry? joyful? our contributions are as follows. in section 2 we introduce the first dataset for training highly multilingual emoji prediction models, twitteremoticon. we then use this dataset to train the first highly multilingual emoticon prediction model, bertmoticon. our model is open source and has an easy to use pypi package. 1 in section 3, we introduce the first highly multilingual dataset of tweets about covid-19, called twittercovid. to generate the dataset, we introduce a novel dataset generation method combining the twitter api, bing translate, and the spacy tokenization library @xcite . we then apply the bertmoticon model to the twittercovid dataset to map how twitter users across the world have emotionally responded to a variety of covid-19 news events. this is the first highly multilingual emotion analysis of tweets in any language, and by far the most comprehensive analysis to-date specifically about the covid-19 pandemic. ## the bertmoticon model in this section, we first describe the emoticons we are trying to predict and present the twitteremoticon dataset that the bertmoticon model was trained on. then we describe our training procedure and model evaluation results. we take particular care to ensure that the twitteremoticon dataset is sampled from a similar distribution to the twittercovid dataset analyzed in section 3 below in order to ensure that the bertmoticon model will transfer well to this unlabeled dataset. ## coronavirus case study we now apply the bertmoticon model to understand the emotional response of twitter users to news about the coronavirus. we first introduce our twittercovid dataset, then we present an analysis of this dataset. ## conclusion we introduced the bertmoticon model for multilingual emoji prediction, and used this model to better understand how twitter users responded emotionally to news about the coronavirus. in follow up studies, we hope to analyze how different countries and language communities reacted differently to events, and have designed our twittercovid dataset and bertmoticon model to facilitate these cross-sectional analyses. we also hope that the bertmoticon model will prove useful for analyzing the emotions of text in other contexts outside of covid-19, and we make the model available in an easy to use python package to facilitate this process.