Measuring Emotions in the COVID -19 Real World Worry Dataset (2020)

Abstract: The COVID-19 pandemic is having a dramatic impact on societies and economies around the world. With various measures of lockdowns and social distancing in place, it becomes important to understand emotional responses on a large scale. In this paper, we present the first ground truth dataset of emotional responses to COVID-19. We asked participants to indicate their emotions and express these in text. This resulted in the Real World Worry Dataset of 5,000 texts (2,500 short + 2,500 long texts). Our analyses suggest that emotional responses correlated with linguistic measures. Topic modeling further revealed that people in the UK worry about their family and the economic situation. Tweet-sized texts functioned as a call for solidarity, while longer texts shed light on worries and concerns. Using predictive modeling approaches, we were able to approximate the emotional responses of participants from text within 14% of their actual value. We encourage others to use the dataset and improve how we can use automated methods to learn about emotional responses and worries about an urgent problem.

## introduction the outbreak of the sars-cov-2 virus in late 2019 and subsequent evolution of the covid-19 disease has affected the world on an enormous scale. while hospitals are at the forefront of trying to mitigate the life-threatening consequences of the disease, practically all societal levels are dealing directly or indirectly with an unprecedented situation. most countries are -at the time of writing this paper -in various stages of a lockdown. schools and universities are closed or operate online-only, and merely essential shops are kept open. at the same time, lockdown measures such as social distancing (e.g., keeping a distance of at least 1.5 meters from one another and only socializing with two people at most) might have a direct impact on people's mental health. with an uncertain outlook on the development of the covid-19 situation and its preventative measures, it is of vital importance to understand how governments, ngos, and social organizations can help those who are most affected by the situation. that implies, at the first stage, understanding the emotions, worries, and concerns that people have and possible coping strategies. since a majority of online communication is recorded in the form of text data, measuring the emotions around covid-19 will be a central part of understanding and addressing the impacts of the covid-19 situation on people. this is where computational linguistics can play a crucial role. in this paper, we present and make publicly available a high quality, ground truth text dataset of emotional responses to covid-19. we report initial findings on linguistic correlates of emotions, topic models, and prediction experiments. ## data we collected the data of @xmath0.46% native english speakers) via the crowdsourcing platform prolific foot_3 . every participant provided consent in line with the local irb. the sample requirements were that the participants were resident in the uk and a twitter user. in the data collection task, all participants were asked to indicate how they felt about the current covid-19 situation using 9-point scales (1 = not at all, 5 = moderately, 9 = very much). specifically, each participant rated how worried they were about the corona/covid-19 situation and how much anger, anxiety, desire, disgust, fear, happiness, relaxation, and sadness @xcite ) they felt about their situation at this moment. they also had to choose which of the eight emotions (ex-cept worry) best represented their feeling at this moment. all participants were then asked to write two texts. first, we instructed them to "write in a few sentences how you feel about the corona situation at this very moment. this text should express your feelings at this moment" (min. 500 characters). the second part asked them to express their feelings in tweet form (max. 240 characters) with otherwise identical instructions. finally, the participants indicated on a 9-point scale how well they felt they could express their feelings (in general/in the long text/in the tweet-length text) and how often they used twitter (from 1=never, 5=every month, 9=every day) and whether english was their native language. the overall corpus size of the dataset was 2500 long texts (320,372 tokens) and 2500 short texts (69,171 tokens). in long and short texts, only 6 and 17 emoticons (e.g. ":(", "<3") were found, respectively. because of the low frequency of emoticons, these were not focused on in our analysis. ## discussion this paper introduced a ground truth dataset of emotional responses in the uk to the corona pandemic. we reported initial findings on the linguistic correlates of emotional states, used topic modeling to understand what people in the uk are concerned about, and ran prediction experiments to infer emotional states from text using machine learning. these analyses provided several core findings: ## conclusions this paper introduced the first ground truth dataset of emotional responses to covid-19 in text form. our findings highlight the potential of inferring concerns and worries from text data but also show some of the pitfalls, in particular, when using concise texts as data. we encourage the research community to use the dataset so we can better understand the impact of the pandemic on people's lives.