An Empirical Methodology for Detecting and Prioritizing Needs during Crisis Events (2020)

Abstract: In times of crisis, identifying essential needs is crucial to providing appropriate resources and services to affected entities. Social media platforms such as Twitter contain a vast amount of information about the general public’s needs. However, the sparsity of information and the amount of noisy content present a challenge for practitioners to effectively identify relevant information on these platforms. This study proposes two novel methods for two needs detection tasks: 1) extracting a list of needed resources, such as masks and ventilators, and 2) detecting sentences that specify who-needs-what resources (e.g., we need testing). We evaluate our methods on a set of tweets about the COVID-19 crisis. For extracting a list of needs, we compare our results against two official lists of resources, achieving 0.64 precision. For detecting who-needs-what sentences, we compared our results against a set of 1,000 annotated tweets and achieved a 0.68 F1-score.

## introduction during crises, substantial amounts of information are shared and discussed on social media @xcite . some of these posts may contain relevant information about the needs of affected and at-risk populations @xcite @xcite . the recent covid-19 virus outbreak is no exception; online platforms such as twitter have been crucial means for sharing information about the impact of the outbreak @xcite , personal accounts from infected individuals @xcite , and updates from medical professionals @xcite . crisis responders and practitioners have also turned to online platforms to obtain actionable information that could aid them in response planning @xcite . in particular, scholars in crisis informatics have provided solutions to detect relevant twitter messages that express resource needs and availabilities related to crisis events, e.g., during the 2015 nepal earthquake @xcite and the 2015 chennai floods @xcite . this paper builds upon and extends prior literature by proposing two needs detection tasks and applying needs detection to data about the covid-19 crisis. in particular, we (1) extract a list of needs by using word embeddings to identify closest terms to needs and supplies with respect to their cosine similarity, and (2) detect who-needs-what sentences to determine social entities who need particular resources. this study makes two contributions. first, we propose a method for identifying and prioritizing resource needs during a crisis. second, we present a set of heuristics to determine the social entities that need specific resources. overall, our study provides a reliable set of methods that might help response professionals identify immediate types of needs in the general population quickly and make effective decisions accordingly. ## related work a large body of literature from the field of crisis informatics has used natural language processing and machine learning methods to extract relevant situational awareness content from large text corpora @xcite . one of several categories of situational awareness content is needs expressed by (affected) individuals and communities @xcite @xcite . @xcite analyzed tweets about eight major natural disaster events and found that about 21.7% of all tweets contained crucial information about urgent needs for shelter, donations, and essential supplies, such as medical aid, clothing, food, and water. varga and colleagues @xcite leveraged machine learning models to match tweets, indicating problems with aid being offered to minimize the waste of resources during a crisis. similarly, @xcite classified tweets based on requests and offers of resources, and further matched requests with offers using regular expressions. temnikova, castillo, and @xcite developed a lexical resource that contained 23 categories of situational awareness, most of which are based on needs requested and resources available (e.g., clean water, shelter material), as well as services (e.g., rescue workers, relief work) to meet the needs. @xcite identified need and availability tweets, and matched the identified needs with availabilities @xcite . our paper builds upon this prior work that has primarily focused on classifying need/non-need tweets. more specifically, we propose methods that identify a general overview of the needs and specify where and by whom these resources are needed. ## data we collected 665,667 tweets posted between february 28, 2020 and may 8, 2020, with a maximum of 10,000 samples for each day using crimson hexagon foot_0 . each tweet contains at least one of the following hashtags: #covid19, #covid-19, #coronavirusoutbreak, #wuhancoronavirus, #2019ncov, #ccpvirus, #coronavirus, #coro-naviruspandemic, #sars-cov-2, #coronavírus, #wuhanflu, #kungflu, #chineseviruscorona, #chi-navirus19, #chinesevirus. our sample includes only tweets from users in the united states and tweets written in english. ## results figure 3 shows the precision@k, where k is in increments of 10. there is a steep drop-off in the results when the cut-off is relaxed from 20 to 30, but the precision@k decreases at a more controlled rate after this drop-off. this indicates that the resources resource who hhs medical-equipment equipment medical-supplies protective-gear stockpile protective-equipment ppe manufacturing personal-protective-equipment medicines for the who-needs-what detection task, our method produced a precision of 0.66, recall of 0.70, and f1-score of 0.68. sentences that were incorrectly predicted as positive examples include those of the form if you need x, then.., while false negatives include more complex sentences. only using the first rule produces a precision of 0.66, recall of 0.68, and an f1-score of 0.67, indicating that most who-needs-what sentences follow this rule, where the who is the subject of the sentence or clause and the what is the direct object. our baseline method, inspired by the work by @xcite , performed poorly, achieving only 0.28 precision, 0.26 recall, and 0.27 f1-score. ## discussion the first needs detection results vary in terms of specificity (e.g., equipment vs. medical equipment, personal protective equipment vs. respirators, funding vs. federal funding). several retrieved terms that are not on the who and hhs lists are general terms such as goods, aid, efforts, programs, and assets. in addition, several terms are synonymous (e.g., personal protective equipment and ppe). these results suggest that clustering the terms may lead to a more distinct set of results. it is not surprising that more of the terms we detected appeared in the hhs than in the who document because we collected our tweet data from the u.s., and the hhs document is from a survey of u.s. hospitals, while the who list is for a global audience. overall, our results suggest two findings: 1) our needs detection method works, and 2) most covid-19 needs mentioned on twitter are either of medical or financial nature (see appendix a). our who-needs-what detection results show that a simple rule-based method can retrieve sentences that mention entities needing resources and the resources needed (0.68 f1-score). this is an interesting finding with several implications. we can produce a simple white-box method for identifying who-needs-what sentences. while deep learning may increase the scores, our method requires no training data. another implication of our findings is that mentioning needs on twitter often follows a specific, uniform format, which could be due to the limited characters available per tweet. testing the generalizability of this method on other crisis events is part of our future work. while social media has been shown to be a valuable source of information during crises, finding useful information is still akin to finding a needle in a haystack. for our who-needs-what detection task, we only found 262 positive examples. overall, our first needs detection method can generate a ranked set of needs for 600,000+ tweets in less than 30 minutes. running steps such as phrase detection and pos tagging in parallel may even improve this time. for the who-needs-what detection task, our method can classify 1,000 sentences in 8 seconds. ## conclusions and future work in this paper, we presented two needs detection methods: one for extracting a list of needed resources during a crisis, and another one for detecting the who-needs-what sentences. we believe that these two methods are helpful in capturing the broad range of needs that emerges during crisis events. specific to the covid-19 crisis, our results suggest that the essential needs are protective equipment and financial assistance. our methods can help detect the essential needs of the general population and affected stakeholders so they can properly plan and respond effectively. in future work, we aim to expand our methodology to identify the availability of needs, if they have been met, and social entities who address them. in addition, we plan to differentiate between a more comprehensive set of requests, including hopes, wants, and wishes during a crisis.