Sensemaking of Socially-Mediated Crisis Information (2024)

Abstract: In times of crisis, the human mind is often a voracious information forager. It might not be immediately apparent what one wants or needs, and people frequently look for answers to their most pressing questions and worst fears. In that context, the pandemic has demonstrated that social media sources, like erstwhile Twitter, are a rich medium for data-driven communication between experts and the public.However, as lay users, we must find needles in a haystack to distinguish credible and actionable information signals from the noise. In this work, we leverage the literature on crisis communication to propose an AI-driven sensemaking model that bridges the gap between what people seek and what they need during a crisis. Our model learns to contrast social media messages concerning expert guidance with subjective opinion and enables semantic interpretation of message characteristics based on the communicative intent of the message author. We provide examples from our tweet collection and present a hypothetical social media usage scenario to demonstrate the efficacy of our proposed model.

## introduction during the early months of a crisis, people are not equipped with relevant knowledge about a crisis, such as what has occurred, what steps to take next, etc., and information can keep evolving rapidly. public officials and crisis responders have often used social media to communicate crisis information @xcite . as witnessed during the pandemic, social media platforms not only shaped people's behavior and opinions but also served as a ground for communicating scientific information about public health. it is widely acknowledged that conflicting information and claims can confuse the public, leading to counterproductive preventive actions, as seen during the covid-19 pandemic @xcite . to the best of our knowledge there has not been much research focusing on designing or investigating social media sites when people might not know what they need @xcite to navigate an unknown crisis. some recent work investigates cognitive factors to identify the relationship between crisis type, organization reputation, and sentiments @xcite . the work in this domain explores distinct factors like forgiveness, empathy, anxiety etc, @xcite . nonetheless, they do not focus on how people perceive crisis information and guide their decisions based on sensemaking @xcite and contextualization. research in crisis communication suggests that effective communication requires an understanding of how different people perceive the messages, and what the fundamental drivers are for their information-seeking needs. in our work, we take inspiration from the crisis communication literature for analyzing the different information facets that are needed by lay social media users to make sense of an unfolding, uncertain situation. our work offers insight into the generalizability of crisis information-seeking characteristics. we contribute a crisis-related intent classification model that is eventually integrated into a human-ai interface to help social media users triage and group relevant information without being exposed to unnecessary noise and negativity. we demonstrate the efficacy of the model by describing a hypothetical sensemaking workflow of a social media user that leverages our proposed model and interfaces. ## related work we discuss the related work regarding two overlapping threads of research: i) socially mediated crisis communication and ii) ai-driven sensemaking using social media interfaces. ## methodology in this section, we describe our methodology for data collection and qualitative labeling of tweets that preceded the conceptualization of communicative intent. please find details about the methodology in the supplemental material: https:// tinyurl.com/mrymxwed . ## sensemaking via communicative intent communicative intent, or simply intent, refers to the aim or purpose of a tweet. intent analysis can help information consumers determine whether the tweet is relevant to what they are seeking as it can provide contextual information about a particular topic. the message intent can be considered during the reasoning process @xcite which can aid users in navigating the information space. unprecedented emergencies, like the pandemic, require the public to adapt to time, domain, and context-specific information in understanding the communication dynamics on social media. current crisis communication models are insufficient in guiding people when exposed to exponentially more information due to increased social media use. the intent classification model is shown in figure 1 . the related models to our work are the health belief model (hbm), the social mediated crisis communication model (smcc), and the elaboration likelihood model (elm). hbm states that an individual's personal beliefs affect their health-related behaviors @xcite . this is a valuable framework to characterize people's discussions on social media based on perceived severity, perceived threat, perceived susceptibility, etc. however, this model does not give us a way to characterize why people perceive certain information in a particular way or what constitutes a threat. in the intent model, source, content, and attitude are ways to characterize who, what, and how information is perceived under a particular category (figure 1.2). the smcc model offers a solution by characterizing the source of information. the smcc model emphasizes identifying individuals that can aid in information dissemination efforts, classifying these individuals as influentials (liu et al., 2020). in social media sites, these individuals have high engagement by having a large following. however, these individuals may not be crisis experts, and other social media users may not want to see information from this individual. in the intent model this is characterized by source of information where we make a distinction between crisis experts and general users (figure 1 .1). when individuals make decisions based on popular information and not credible information, they are not engaging in elaborate reasoning (i.e. taking the time to think through what they've read). elm considers that information seekers make judgments in two ways: (1) fast with simple reasoning and (2) slow with elaborate reasoning @xcite . since social media sites provide information in a user-friendly interface, information seekers are continuously tempted to make fast judgments based on limited information. additionally, the elm model doesn't provide ways to characterize the interplay between information dimensions and types of decisions. social media sites provide content moderation methods that help users curate their information feed, which can aid in finding relevant scientific information. this can also aid in supporting elaborate thinking. however, if users deem these methods unreliable, they will not engage in content moderation increasing the likeliusing our intent model, an information seeker can triage information based on different intent categories. for each intent category, additional details are accessible to the user via the source, content, and attitude or the messages. the message's source is defined as an expert or general user and the degree of subjectivity. the content and attitude refer to the topic and sentiment of the message. most content analysis studies generated a labeling guide based on previous literature reviews, such as guiding principles for classifying social media news articles @xcite , informativeness (whether a tweet contains relevant information or not) @xcite , an existing crisis communication model like hbm @xcite . we chose to follow this approach for our labeling guide. during information-seeking behavioral patterns, consumers pay attention to the message source, so two graduate students took into account the source of the message (expert/general user) and the message content to determine the message's intent. we classified 6,844 tweets into five intent categories: (i) expert guidance, (ii) situational awareness (iii) severity (iv) reactions, (v) lived experience. figure 2 provides a topic distribution and an example tweet in each intent category with similar uncertainty profiles to emphasize the differences across each intent category. expert guidance categorizes tweets from experts that are providing some suggestions or recommendations to address the pandemic (21% of sample) @xcite ; ehrmann and wabitsch, 2022). the most frequent topics for this category are scientific information, testing, and vaccine development. situational awareness are tweets from any source that are providing updates on news alerts, business proceedings, and policy revisions (33% of sample) @xcite . the most frequent topics for this category are global effects, pandemic emergence, and recreation. severity describes tweets that qualitatively or quantitatively report on the impact of the pandemic (11% of sample) @xcite . the most frequent topics for this category are pandemic emergence, global effects, and case statistics. reactions are tweets from any source that describe emotions, comments, or responses towards events caused by the pandemic (30% of sample) @xcite . the most frequent topics for this category are testing, recreation, and global effects. lived experience tweets describe direct personal experiences with specific details regarding location and time (8% of sample) @xcite . the most frequent topics for this category are global effects, schools, and recreation. a distilbert @xcite classification model was trained on the 6844 labeled tweets un-til overall accuracy reached approximately 70%, indicating good performance. the accuracy per intent category: (1) expert guidance 69%, (2) reactions 74% (3) lived experience 80% (4) severity 93% (5) situational awareness 74%. these results on real-world data indicated that our model could also be used to learn from user interactions. current content moderation methods do not allow users to update the underlying model. however, for machine-guided social media systems to better address the changing needs of information consumers, users need to update the underlying models to match their mental model of changing information. the intent model allows users to update each message's intent to one that aligns more with the user's mental model or curate their intent category based on the source, content, and attitude of the messages. ## sensemaking during a crisis we present an ai-driven sensemkaing scenario during the covid-19 pandemic to demonstrate the utility of our proposed intent classification model (icm). we integrated icm (a predictive model trained on our collection of tweets) with a web-based interface, which allows lay information consumers to triage credible and potentially actionable information. let us see how this interface can be used by mary, a parent whose child currently studies at home due to covid-19-affected school closing. she needs to decide if she should reduce her working hours and invest in homeschooling. figure 3 shows mary's workflow as she interacts with our interface. from her experience, she knows that not all tweets are reliable and that topics provided by twitter do not give her sufficient control to navigate the relevant messages. she uses the interface to triage the relevant information using situational awareness as the intent category of interest (figure 3 .a). additionally, she selects all intent categories to learn more about the contrasting messages by reviewing the representative tweets in each category. she finds that by using the intent categories and the highlighted words she can gradually make sense of the different facets of the conversation related to the pandemic that might or might not be related to her decision-making goal. she can easily filter out messages from categories she is not interested in such as reactions and severity. to find more contextually relevant messages, mary filters by keyword school and chooses to summarize the overall message profile and observes the patterns across sentiment, subjectivity, expertise, and topics across each intent category (figure 3 .c). she notices that situational awareness tweets are from experts and lay users and that schools is a frequent topic. she notices that messages from experts and general users have distinct groupings which tell her they may have different opinions (figure 3 .b). interestingly, she finds the helpful messages discussing distance learning and back-to-school efforts come from lay users because of their lived experience. it did not occur to her that online learning could be a possible solution for her child to continue learning. she found tweets suggesting reopening in-person learning once proper safety measures are determined. she saves those messages and repeats her search to find even more similar tweets (figure 3.d ). she also decided to filter the selection criteria only to consider tweets from the schools topic. as she expected, applying this filter allowed her to see more messages about other parents' experiences with both homeschooling and online learning. after reviewing her saved tweets, she decided that online learning is a viable option. since mary has a day job she needs to report to, online learning for her child would allow mary to continue working and her child to continue receiving an education. by leveraging the intent model, mary could quickly reduce her exposure to irrelevant tweets. she could also assess the credibility of the messages critically and have sufficient control over the information she needed to know. ## conclusion and future work in this paper, we present a crisis-related intent classification model and present its utility via examples and usage scenarios using the covid-19 pandemic as an example. we developed and trained classification models for sentiment, subjectivity, and topic to further our understanding of how experts and general users communicate during the initial stages of a crisis. we use twitter messages as the basis of our analysis to profile the information uncertainty and address the need for principled approaches towards sensemaking of socially mediated information during a crisis. we are currently developing a software prototype, demonstrated in section 5, that allows lay users to explore messages using the intent of a message author and control their exposure to crisis-relevant information, by focusing on what they would need to address their pressing questions. future work would consider a user study with diverse participants to understand how users perceive the usefulness and utility of our proposed sensemaking workflow and the resulting human-machine interface in their information-seeking processes. we intend to build upon and further develop the web interface, taking into account previous studies on end-user content moderation techniques.