## 1 Introduction  
Rapidly sensing and understanding the data generated during a disaster can help in disaster response. On social media, data generated by users in the near future of a disaster are likely to express the situation they are facing as well as disaster-related information, such as the category of the disaster event. Situational awareness (SA) systems are particularly important in this area as they can automate the sensing and understanding of user-generated information to help people perform rapid disaster response [1, 2]. However, user-generated information is usually multimodal and typically contains visual, audio, and textual information. Appropriate integration of multimodal features has also been shown in several works to improve the accuracy of data analysis [3, 4] and it becomes a challenge to deal with the multimodal information in these messages [5]. The introduction of artificial intelligence has greatly enabled disaster response and situational awareness systems. In disaster response, the application of convolutional neural networks (CNNs) provides a powerful tool to analyze visual information quickly and accurately [6, 7, 8]. By performing feature extraction and pattern recognition on images of a disaster site, CNNs can help determine the type, scale, and impact of a disaster. However, in the practical application of disaster response, the integration of multimodal data needs to be considered in addition to visual information [9]. The synergistic analysis of multi-source information such as speech data might provide a more comprehensive understanding of the disaster scene. In previous work, much research has focused on the processing of single-modal information, such as processing only image [6] or text [10] data. Such approaches have been successful in some contexts but they potentially fall short in fully leveraging the correlations between different modal information, limiting the system's ability to fully understand the overall environment. In recent years, with the rise of graph learning, multimodal information fusion has become more flexible and efficient. By abstracting information from different modalities into nodes of a graph and using graph convolutional networks (GCNs) for information transfer and fusion, the GCNs can automatically learn the relationships between modalities, enabling the whole system to better understand the association between multimodal information [11]. It is worth noting that the introduction of graph learning does not mean abandoning the deeper mining of each modality. On the contrary, combining graph learning with traditional CNNs allows for a more comprehensive exploitation of the characteristics of each modality, resulting in a more information-rich and robust multimodal feature representation. The introduction of graph learning for modality data [11] can be divided into reconstructing modalities into graph structure [12; 13; 14] and identifying graph nodes from modalities [15; 16; 17]. We utilize the latter approach to extract nodes representing different disaster events from multimodal data (visual and audio) and construct a graph using the relationships of the disaster events. This suggests the two following challenges:  
1. Providing an effective discriminative representation of multimodal data;
2. Placing demands on the construction of the graph structure to ensure that the network can learn the relationships between the different modalities. Therefore, in the light of the challenges outlined above, we focus on extracting disaster-related information from paired visual and audio data (usually represented as video) in this work. Our proposed multimodal hierarchical graph-based SA system can classify disaster events at coarse- and fine-grained for primary and advanced sensing based on visual and audio, referring to the multi-level sensing introduced by the three-layer model of SA [18]. The main contributions of this paper are as follows:  
1.