CLEAR -Command: Coordinated Listening, Extraction, and Allocation for Emergency Response with Large Language Models (2025)

Abstract: Effective communication is vital in emergency response scenarios where clarity and speed can save lives. Traditional systems often struggle under the chaotic conditions of real-world emergencies, leading to breakdowns in communication and task management. This paper introduces \textbf{CLEAR} -Command, a system that leverages Large Language Models (LLMs) to enhance emergency communications. \textbf{CLEAR} stands for $textbfC oordinated L istening, E xtraction, and A llocation in R esponse. CLEAR-Command automates the transcription, summarization, and task extraction from live radio communications of emergency first responders using the OpenAI Whisper API for transcription and gpt-4o for summarization and task extraction. Our system provides a dynamic overview of task allocations and their execution status, significantly improving the accuracy of task identification and the clarity of communication. We evaluated our system through an expert pre-study with 4 experts and a user study with 13 participants. The expert pre-study identified gpt-4o as providing the most accurate task extraction, while the user study showed that CLEAR-Command significantly outperforms traditional radio communication in terms of clarity, trust, and correctness of task extraction. Our demo is hosted under this link , and all project details are presented in our Gitlab page$.

## introduction in search and rescue operations (sar) during emergency scenarios, the efficiency of communication among first responders, such as firefighters and medical intervention teams, is critical to managing incidents effectively @xcite . critical situations are often marked by high pressure and the need for rapid decision-making, where clear * authors contributed equally to this work. and concise communication can mean the difference between life and death. however, the complex nature of such environments can lead to communication breakdowns, overwhelming the responders with fragmented and sometimes redundant information @xcite . the primary challenge in these settings is maintaining situational awareness while managing a high volume of communications, often under severe time constraints @xcite . traditional emergency communication systems often fall short in the face of complex and evolving scenarios @xcite , where information overload can lead to critical details being missed or miscommunication, which may cause critical incidents and mistakes @xcite . the recent advancements in natural language processing, particularly the development of large language models (llm) and multimodal large language models (mllm), offer substantial promise for enhancing emergency communication systems @xcite . state-of-the-art llms and mllms are adept at transcribing spoken communication @xcite , analyzing complex textual data, and systematically extracting actionable plans @xcite . despite concerns regarding the potential for model-generated hallucinations and inaccurate predictions, the incorporation of context-aware processing capabilities assists these models in adapting effectively to the fluctuating dynamics of complex environments @xcite . furthermore, the integration of human oversight within human-in-the-loop systems along with uncertainty quantification approaches enhance the reliability of the outputs while concurrently mitigating risks inherent in autonomous decision-making systems @xcite . in this paper, we introduce the clear (coordinated listening, extraction, and analysis for emergency response)-command system, which leverages the recent advancements in llms to assist sar teams organizing their communications. clear-command automates the organization and summarization of vocal communications, transforming complex streams of speech data into structured, actionable tasks. clear-command consists of a speech-to-text transcription model that transforms spoken communications into text and an llm that summarizes the relevant parts of the communication and extracts the tasks and their status. this allows emergency first responders to effectively monitor the progression of tasks and provides them with a dynamic, real-time overview of task allocation and execution status. the specific llm used by clear-command is gpt-4. we decided on this, based on a pre-study with 4 experts that assessed the quality of text transcription, summarization, and task extraction that different llms showed when dealing with emergency scenarios. to evaluate our system, we conducted a user study with 13 participants. our results show that clear- command significantly outperforms traditional radio communication in terms of clarity, trust, and correctness of task extraction. the link to a live demo website of our system is https://clear-command.vercel.app . the video demonstrating our system can be found on https://youtu.be/zf3hmmueq9o . all project details are presented in our gitlab page https: //gitlab.com/achref.d/clear-command . ## clear-command system the clear-command system architecture is composed of 4 modules and a user interface, as illustrated in figure 1 . the speech-to-text module automatically converts vocal communications into text, which is used by the summarization module to succinctly summarize crucial information that is then passed to the analysis module to derive and manage tasks. the storage module stores summaries and derived tasks, which are fed back to the summarization module and the analysis module for context. the user interface displays a comprehensive real-time overview of summaries and derived tasks taken from the storage module. all this process is designed to take place in a real-time manner to enhance situational awareness and response efficiency, minimizing the cognitive load on responders by allowing them to concentrate on operational execution rather than communication management. speech-to-text module. this module uses the openai's whisper api 1 to transcribe real-time vocal communications, even in noisy emergency environments. summarization module. the gpt-4 2 model is used to extract concise summaries based on the transcribed text from the speech-to-text module and previous summaries. the summary emphasizes crucial information, which is achieved through iterative prompt design, including clear instructions and examples. the prompt embeds the previous summaries to enhance the situational understanding. analysis module. the gpt-4 model is used to extract structured information, such as tasks, from all previous summaries. the prompt for the analysis module is designed iteratively to return the structured information, such as tasks, in the expected format. the prompt contains previous summaries and structured information to be aware of existing information, such as tasks. for instance, an unresolved task can be marked as resolved by the 1 whisper: https://github.com/openai/whisper 2 gpt-4: https://openai.com/index/gpt-4/ analysis module when the provided information indicates that the task is resolved. storage module. this module provides storage for the clear-command system, allowing the summarization and analysis module to use the existing information from the ongoing emergency. this storage prevents modifications to existing entries and is append-only. the storage module preserves the history, including the audio recording, the transcriptions, the summaries, and structured information, to ensure that everything is fully transparent, understandable, and accessible through the user interface during and after the emergency. user interface. the user interface is built in html, css, and javascript. the user interface is composed of three panels, which are depicted in figure 2 . the communication overview panel shows the previous summaries and structured information from the storage module. the tasks overview panel shows the derived tasks and their status. the radio panel allows users to interact with the system through direct communication. communication overview panel. this panel provides the list of previous and current communications, each containing the sender, the time of creation, a transcription, a summarization, and a list of derived structural information, such as created tasks. resolved tasks are marked as green, and unresolved tasks are marked as red when referenced in the overview. ongoing communications are marked red, as seen in figure 2 at the bottom of the communication overview panel. tasks overview panel. this panel shows a realtime view of resolved and unresolved tasks, along with an indication of their priority. radio panel. this panel contains a button for starting and stopping vocal recordings, which allows to interact with the system by adding new communications that will trigger the transcription, summarization, and analysis process. implementation details. we implemented our system in typescript running on nodejs and python, incorporating libraries from the hugging face ecosystem. our speech-to-text and llm models run on cloud inference endpoints, chosen for their capabilities to support the computational demands of the system's data processing and machine learning tasks and on-demand horizontal scaling to support multi-user access. ## evaluation we evaluate clear-command through two main studies: an expert pre-study to assess the quality of summarization and instruction generation across different models and a user study to evaluate usability, trust, and workload in simulated emergency scenarios. the detailed text of the scenarios used in our evaluation, as well as further information about the experts and questions can be found in our gitlab page https://gitlab.com/achref. d/clear-command . ## discussion clear-command demonstrates significant improvements in emergency response through effective communication summarization and task allocation. the integration of speech recognition models for transcription and llms for summarization and instruction generation showcases the potential of state-of-the-art llms in critical situations. however, despite these advancements, certain limitations must be acknowledged. while clear-command is effective in enhanc-ing emergency response through communication summarization and task analysis, it has several limitations. our reliance on specific models for transcription (openai whisper) and summarization and instruction generation (gpt-4) introduces dependencies on their performance, which may be affected by linguistic diversity and noisy environments. the substantial computational resources required by these models may also limit their deployability in resource-constrained settings. the validity of our evaluation should be considered in the light of the following constraints. the expert study, involving 4 experts, and the user study, with 13 participants, provide preliminary insights that could be deepened through larger participant samples from different emergency sectors. due to computational constraints, our evaluation considered solely a zero-shot setup to adapt the models to our specific use case. while achieving the reported results without training demonstrates the capabilities of state-of-the-art llms and their potential in emergency use cases, we believe that fine-tuning models can enhance the performance further. ## conclusion in this paper, we introduced clear-command, a system designed to enhance emergency response efficiency through the transcription, summarization, and task allocation of emergency communications in sar missions. by integrating openai whisper for voice-to-text transcription and chat-gpt4 for summarization and task extraction, clear-command manages and organizes critical radio communication data in real-time. our evaluation, comprising an expert pre-study and a user study, shows that clear-command outperforms traditional radio communication methods in terms of clarity, trust, and correctness.