## 3 Proposed TriagE-NLU System  
TriagE-NLU represents a multimodal semantic understanding system designed for processing telemedicine dialogue during emergency scenarios. It integrates textual dialogue, patient metadata, and contextual signals into a unified pipeline that produces structured semantic representations and clinical action recommendations. The system is designed to be modular, privacy-preserving, and generalizable across languages and regional healthcare protocols. Figure 1 illustrates the proposed model.  
The TriagE-NLU system ingests multimodal input (speech, text, metadata) and processes it using Python 3.11.9 and SentencePiece 0.2.0 for language tagging and tokenization. Tokenized input is passed to an mT5-Base transformer (via Hugging Face Transformers), which performs semantic parsing using the EUREKA frame schema and classifies intervention labels. The model is trained on the synthetic, multilingual TriagEX dataset and updated via a federated learning loop with secure aggregation. Output is validated using  
Figure 1: System-level overview of the proposed TriagE-NLU.  
constrained decoding and a slot ontology, then deployed on CUDA-enabled infrastructure and evaluated using Slot F1, Frame F1, and Macro F1 metrics. Slot F1 measures the accuracy of individual field extractions (e.g., symptoms, onset time), Frame F1 evaluates exact matches of full semantic frames, Macro F1 averages performance across all classes equally, and the multilingual metric assesses consistency across languages. The core architecture of TriagE-NLU consists of four main components.  
1. _Multimodal Input Processing_: The system accepts heterogeneous input streams, including transcribed input speech or dialogue between patients and clinicians, structured patient metadata (e.g., age, pre-existing conditions, vital signs), and temporal and contextual markers (e.g., symptom onset duration, location metadata).
2. _Semantic Parsing Engine_: This module leverages multilingual transformer models fine-tuned for low-resource clinical contexts. It maps utterances and contextual signals to a structured representation based on a new proposed EUREKA schema (Emergency Understanding Representation for Evidence-based Knowledge and Action), which will be described in Section 4, that encodes symptom mentions, temporal qualifiers, patient risk factors, and urgency indicators.
3. _Intervention Grounding Module_: Parsed semantic frames are fed into a decision engine that aligns recognized medical states with appropriate interventions using a rules-and-learned-policy hybrid model informed by clinical guidelines (e.g., ACLS, WHO triage protocols). For example, a combination of \"slurred speech\", \"unilateral weakness\", and \"<10 min onset\" is mapped to \"stroke protocol activation\".
4. _Federated Learning Update Loop_: Each deployment instance of TriagE-NLU (e.g., in a regional hospital network) trains on local usage patterns while sharing model updates through secure aggregation. This ensures continuous adaptation without exposing sensitive clinical dialogue, maintaining both data locality and generalization capabilities.  
In high-level terms, TriagE-NLU translates unstructured multilingual interaction into structured semantic meaning and uses it to drive urgent decision support. It is robust to input variability and informal symptom descriptions, making it suitable for deployment in diverse telemedicine environments.  
Presenting a use case scenario, Figure 2 illustrates the TriagE-NLU architecture in action, showing how patient speech, metadata, and symptom onset are processed through a multimodal input module and semantic parsing engine. This structured information triggers a clinically informed intervention (e.g., stroke protocol activation) and feeds into a federated learning loop for secure model improvement across deployments.  
Figure 2: Pipeline overview of the proposed TriagE-NLU.