Title: HydroGen: Hydrological Report Generation with Two-Stage Instruction-Tuned Multimodal Models, Temporal Prompts, & Knowledge-Guided Agents
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Hydrological Report Generation, Multimodal Large Language Model, Temporal Prompt Tokens, Knowledge-Guided Agents

Abstract: Hydrological report generation is crucial for monitoring clouds, typhoons, rainfall, and water events, yet current multimodal models struggle with multi-image alignment and domain-specific knowledge integration. We present HydroGen, a domain-adaptive framework that overcomes these challenges through instruction tuning, temporal modeling, and knowledge-guided reasoning within a two-stage pipeline. First, we build HydroMM-Instruct, an instruction dataset that uses YOLOv8 for typhoon detection and shapefiles for region mapping, with reports standardized into cause–effect phrases. Second, we introduce a two-stage training pipeline with continual pre-training on hydrological data (radar maps, pressure charts, expert assessments) followed by supervised fine-tuning for report generation. Third, to enhance multi-image alignment, we introduce temporal prompt tokens that capture event sequences and progressions. Finally, we present GuideAC, an in-context agent that injects antecedent–consequence rules to improve reasoning. Evaluation on Thailand’s weekly hydrological reports (2018–2025) shows that HydroGen substantially outperforms strong multimodal baselines, achieving a BERT-F1 of 84.5% (+50.6%) and a ROUGE-L of 38.3% (+33.9%).