Abstract: During the course of a Humanitarian Assistance-Disaster Relief (HADR) crisis, that can happen anywhere in the world, real-time information is often posted online by the people in need of help which, in turn, can be used by different stakeholders involved with management of the crisis. Automated processing of such posts can considerably improve the effectiveness of such efforts; for example, understanding the aggregated emotion from affected populations in specific areas may help inform decision-makers on how to best allocate resources for an effective disaster response. However, these efforts may be severely limited by the availability of resources for the local language. The ongoing DARPA project Low Resource Languages for Emergent Incidents (LORELEI) aims to further language processing technologies for low resource languages in the context of such a humanitarian crisis. In this work, we describe our submission for the 2019 Sentiment, Emotion and Cognitive state (SEC) pilot task of the LORELEI project. We describe a collection of sentiment analysis systems included in our submission along with the features extracted. Our fielded systems obtained the best results in both English and Spanish language evaluations of the SEC pilot task.

Introduction: The growing adoption of online technologies has created new opportunities for emergency information propagation BIBREF0 . During crises, affected populations post information about what they are experiencing, what they are witnessing, and relate what they hear from other sources BIBREF1 . This information contributes to the creation and dissemination of situational awareness BIBREF2 , BIBREF3 , BIBREF4 , BIBREF0 , and crisis response agencies such as government departments or public health-care NGOs can make use of these channels to gain insight into the situation as it unfolds BIBREF2 , BIBREF5 . Additionally, these organizations might also post time-sensitive crisis management information to help with resource allocation and provide status reports BIBREF6 . While many of these organizations recognize the value of the information found online—specially during the on-set of a crisis—they are in need of automatic tools that locate actionable and tactical information BIBREF7 , BIBREF0 . Opinion mining and sentiment analysis techniques offer a viable way of addressing these needs, with complementary insights to what keyword searches or topic and event extraction might offer BIBREF8 . Studies have shown that sentiment analysis of social media during crises can be useful to support response coordination BIBREF9 or provide information about which audiences might be affected by emerging risk events BIBREF10 . For example, identifying tweets labeled as “fear” might support responders on assessing mental health effects among the affected population BIBREF11 . Given the critical and global nature of the HADR events, tools must process information quickly, from a variety of sources and languages, making it easily accessible to first responders and decision makers for damage assessment and to launch relief efforts accordingly BIBREF12 , BIBREF13 . However, research efforts in these tasks are primarily focused on high resource languages such as English, even though such crises may happen anywhere in the world. The LORELEI program provides a framework for developing and testing systems for real-time humanitarian crises response in the context of low-resource languages. The working scenario is as follows: a sudden state of danger requiring immediate action has been identified in a region which communicates in a low resource language. Under strict time constraints, participants are expected to build systems that can: translate documents as necessary, identify relevant named entities and identify the underlying situation BIBREF14 . Situational information is encoded in the form of Situation Frames — data structures with fields identifying and characterizing the crisis type. The program's objective is the rapid deployment of systems that can process text or speech audio from a variety of sources, including newscasts, news articles, blogs and social media posts, all in the local language, and populate these Situation Frames. While the task of identifying Situation Frames is similar to existing tasks in literature (e.g., slot filling), it is defined by the very limited availability of data BIBREF15 . This lack of data requires the use of simpler but more robust models and the utilization of transfer learning or data augmentation techniques. The Sentiment, Emotion, and Cognitive State (SEC) evaluation task was a recent addition to the LORELEI program introduced in 2019, which aims to leverage sentiment information from the incoming documents. This in turn may be used in identifying severity of the crisis in different geographic locations for efficient distribution of the available resources. In this work, we describe our systems for targeted sentiment detection for the SEC task. Our systems are designed to identify authored expressions of sentiment and emotion towards a HADR crisis. To this end, our models are based on a combination of state-of-the-art sentiment classifiers and simple rule-based systems. We evaluate our systems as part of the NIST LoREHLT 2019 SEC pilot task.

Question: Did they pre-train on existing sentiment corpora?
Answer: No, they used someone else's pretrained model.