A Dynamic Fusion Model for Consistent Crisis Response (2025)

Abstract: In response to the urgent need for effective communication with crisis-affected populations, automated responses driven by language models have been proposed to assist in crisis communications. A critical yet often overlooked factor is the consistency of response style, which could affect the trust of affected individuals in responders. Despite its importance, few studies have explored methods for maintaining stylistic consistency across generated responses. To address this gap, we propose a novel metric for evaluating style consistency and introduce a fusion-based generation approach grounded in this metric. Our method employs a two-stage process: it first assesses the style of candidate responses and then optimizes and integrates them at the instance level through a fusion process. This enables the generation of high-quality responses while significantly reducing stylistic variation between instances. Experimental results across multiple datasets demonstrate that our approach consistently outperforms baselines in both response quality and stylistic uniformity.

## introduction people in crisis often turn to social networks for information, support, and assistance, especially when other sources cannot be relied upon @xcite . although some responses in social media from the general public offer valuable information and emotional support, others may be inaccurate and even misleading to those in crisis @xcite . for example, during hurricane irma, users on twitter (now x) shared conflicting information about whether shelters required identity checks, which affected whether some immigrants decided to evacuate @xcite . direct communication from relevant government agencies or ngos that carry out disaster relief efforts is critical to providing accurate information and verifying misleading information. however, call 911 or the national hurricane center' s (nhc) hotline at 1-800-638-6620 to report the stranded family and request immediate assistance. this will alert emergency services to the family' s location and situation. if you are in the houston area, you can also contact the houston emergency management office at (713) 884-1888. call 911 or the national hurricane center' s (nhc) hotline at 1-800-638-6620 to report the stranded family and request immediate assistance. reach out to the american red cross (arc) for immediate assistance with medical care and shelter. you can contact them at 1-800-redcross (1-800-733-2767) or visit their website at www.redcross.org .. reach out to the american red cross or the humane society to inquire about foster care programs professionalism actionability figure 1 : examples of responses with high and low professionalism and actionability. professional responses include explanations backing recommendations, demonstrating authority. actionable responses offer specific guidance (e.g., phone numbers, website links) that users can follow to seek help. in this paper, we focus on generating consistent responses, i.e., ensuring that professionalism, actionability, and relevance are roughly the same across all responses. recent studies have explored the potential role of llms in supporting crisis communication @xcite @xcite . these systems aim to provide actionable, real-time guidance to affected individuals, focusing on user satisfaction, responsive interaction, and efficient use of resources @xcite . however, an important issue remains overlooked: the consistency of automatically generated responses. authorities and ngos have shown bias in their responses to people in crisis, which leads to inequitable access to aid and distrust @xcite . we define consistency as the uniformity of the style in which information is conveyed across all responses. in particular, the core information conveyed should maintain the same level of quality regardless of the audience, crisis scenario, or communication platform. consistency signals organizational reliability. when messages remain aligned, audiences are more likely to trust the source @xcite . in contrast, inconsistent responses can be confusing and diminish trust @xcite . for example, if some responses offer clear guidance while others are vague or off-topic, users may be uncertain about what to believe or do. figure 1 shows examples of replies with different degrees of professionalism and actionability. when responses vary in quality across users, those receiving lower quality replies may perceive the interaction as inattentive or dismissive, resulting in dissatisfaction. previous studies have explored the generation of consistent responses in general-purpose dialogue systems, with particular attention to persona consistency @xcite , semantic consistency @xcite , and factual consistency @xcite . few studies have addressed style consistency in crisis communications @xcite . additionally, these studies typically employ fine-tuned generative models to increase consistency @xcite . there are no established metrics to evaluate the consistency of responses in crisis communication. effective crisis communication requires adherence to critical communicative functions @xcite . these responses should be professional @xcite , actionable @xcite , and relevant to user needs. response consistency, therefore, entails delivering messages with stable characteristics across these dimensions, regardless of user query or scenario. we propose a task-oriented definition for crisis communication: consistency refers to the degree to which all responses have similar characteristics across the three dimensions: professionalism, actionability, and relevance, while exhibiting minimal variation across responses. in addition, we propose a fusion framework to generate crisis responses with improved consistency. the approach integrates the strengths of the responses generated by multiple methods, taking advantage of their complementary advantages to produce highly effective outputs in all evaluation dimensions, resulting in reduced variations. our approach employs state-of-the-art generation methods and explores various fusion methods. we evaluate the generation approaches in the three critical dimensions (professionalism, actionability, and relevance) as well as consistency across these dimensions. experiments show the fusion framework enables the generation of responses with higher overall quality and consistency. specifically, we propose a novel fusion method grounded on assigning tailored weights to each dimension. we experiment with llama and mistral and demonstrate that our fusion method results in superior performance compared to alternatives. the contributions of this study include: information needs and responses in crisis individuals frequently use social media platforms to seek assistance in times of crisis. previous studies have proposed methods for detecting and classifying user needs. several datasets offer granular categorizations of needs @xcite . recent studies have proposed using llms to facilitate timely responses @xcite @xcite . for example, goecks and waytowich (2023) and otal et al. (2024) leveraged llms to generate actionable plans or guidance to crisis-affected individuals. grigorev et al. (2024) developed incidentrespon-segpt, which leverages llms to automatically generate traffic incident response plans by synthesizing guidelines and processing real-time accident reports to inform authorities. rawat (2024) introduced disasterqa, which is designed to evaluate llms in disaster response scenarios. they experimented with several prompting methods to answer crisis questions. these prior studies investigate approaches to generating responses for crisis communication. we are the first, however, to investigate the consistency of responses, with a focus on maintaining a uniform style across varying scenarios. consistent response generation consistent responses are essential for ensuring trust. in particular, it is important to avoid contradictions when addressing different audiences at different times, maintain a consistent tone, and ensure the conveyed information remains aligned @xcite . previous studies have explored various aspects of consistent response generation, including persona consistency, semantic consistency, and factual consistency. persona consistency refers to the alignment between generated responses and the established persona in dialogue systems @xcite @xcite . semantic consistency ensures the generated responses logically follow the context without introducing irrelevant @xcite . factual consistency refers to the accuracy and correctness of generated content @xcite . while these forms of consistency are crucial in general-purpose dialogue, they do not address consistency in balancing the critical communication dimensions required for crisis response, including professionalism, actionability, and relevance. to our knowledge, no prior work has systematically defined or evaluated consistency in the context of crisis communication, highlighting a gap that our work aims to address. ## consistency in crisis communication consistency in crisis communication is crucial for maintaining trust and clarity. our consistency involves producing professional, actionable, and relevant responses, as defined below. maintaining consistency across these dimensions is essential because variation can lead to confusion, reduced trust, and even harmful outcomes. higher scores indicate better consistency, which refers to minimized fluctuation in standards that reliably address user needs across diverse queries and requests @xcite . it also supports scalability by ensuring that all users receive uniformly relevant, actionable, and professional guidance regardless of context or input variation. the framework leverages a fusion-based generation strategy that integrates generations from state-of-the-art approaches. rather than selecting one output, we introduce a prompt-driven fusion mechanism that evaluates outputs by different models across critical communicative dimensions and synthesizes a new, improved response that draws on the strengths of both. we represent the process using the following formulation with the example of using instructional prompt (ip) and retrieval-augmented generation (rag): cc(n, d) = l fuse m ip (n ), m rag (n ), s ip , s rag cc(n, d) represents the response generation process for a given crisis needs n within a crisisspecific context d. the model generates two candidate responses: m ip (n ) via instructional prompt, and m rag (n ) via retrieval-augmented generation. s ip and s rag represent the score vectors of the instructional prompt and rag outputs respectively, evaluated along three communicative dimensions: professionalism, actionability, and relevance. fuse(•) compares and balances the strengths of m ip (n ) and m rag (n ) in these dimensions and generates a new response optimized across all aspects. the process is further detailed in three steps. ## cross crisis generalization to investigate the robustness of our fusion framework, we carry out experiments to generate responses to other crises such as earthquakes and typhoons. we employ the crisisbench dataset @xcite , which comprises a diverse set of crisis events. we use the best-performing model, llama-3.1-8b-instruct (consistency: 0.94, overall quality: 0.95), for the experiment. ## qualitative analysis to investigate how humans perceive the generated crisis responses, we recruited two phd students with a background in crisis computing to evaluate our responses. we select 50 responses generated by instructional prompt, rag, and fusion w/ eval & weight instruct using llama-3.1-8b-instruct. (see evaluation guidance in appendix a.3) we report the results in ## conclusion we introduce the evaluation of consistency for crisis communication, which requires that responses are uniformly professional, actionable, and relevant for all contexts. to achieve the generation of consistent responses, we propose a fusion framework and conduct experiments with various open-sourced llms. results show that our fusion framework can achieve better consistency and higher overall quality across professionalism, actionability, and relevance. in particular, the evacuation scores are beneficial and enhance the fusion process. crosscrisis experiments have been conducted to show the robustness of our framework across diverse crisis contexts. human evaluation proves that our fusion-based generation obtains more preference.