# Driver Intention Recognition for Mine Transport Vehicle Based on Cross-Modal Knowledge Distillation
###### Abstract
This method achieves near-teacher performance with reduced computational requirements and has shown success in single-modal applications. With the rise of multimodal learning, cross-modal knowledge distillation (CMKD) has gained prominence. In CMKD, teacher and student models utilize different data modalities, such as text and images, facilitating the transfer and integration of knowledge across modalities to improve performance and generalization. [PERSON] et al. [27] enhanced 3D object detection using 3D point clouds to guide 2D image feature learning. [PERSON] et al. [28] transferred knowledge from RGB videos to 3D human pose sequences for action recognition. [PERSON] et al. [29] developed a separable multimodal distillation approach for emotional recognition, combining language, sound, and vision. [PERSON] et al. [30] employed physiological signals, including EMG, ECG, and GSR, to guide video data for driver stress recognition. [PERSON] et al. [31] improved continuous emotional recognition in EEG data by incorporating knowledge from visual modalities. CMKD transfers advanced features and knowledge from the teacher model to the student model, thereby enhancing the student model's ability to handle complex tasks during training. This study investigates the combined use of EEG signals and video data to decode driver intentions in mine transport vehicles. Specifically, it applies a CMKD framework, where an EEG-driven intention recognition model serves as the teacher to guide a video-based intention recognition model. This process optimizes the video data processing and improves the accuracy of video-based intention recognition. Compared with existing EEG-video fusion methods, the CMKD approach effectively exploits the complementary nature of EEG and video data, further enhancing the accuracy and robustness of driving intention recognition. At the same time, the CMKD mechanism significantly improves the training efficiency of the video data processing model and strengthens its adaptability in complex dynamic environments, ultimately providing more reliable and accurate support for driving intention recognition. The key contributions of this paper are as follows. 1. An experimental framework was designed to collect driving intention signals from mine transport vehicles. Driving videos were recorded from two coal mine transport vehicles, processed into video data for intention analysis, and used as stimuli to gather EEG signals related to driving intentions. Additionally, a simulation environment for mine transport vehicle operation was created, allowing multiple subjects to view the videos and generate corresponding EEG data. 2. A driving intention recognition model for mine transport vehicles was developed using CMKD. An EEG-based intention recognition model was first created to decode driver intentions. This model served as a teacher, providing intention information to train a video-based student model. Guided by the teacher model, the student model achieved effective intention recognition even in the absence of direct EEG data. In the context of driving intention recognition, combining EEG and video modalities provides deeper insights into the driver's cognitive state and the external environment, significantly improving the accuracy and reliability of intention recognition. By using the EEG model as the teacher and the video model as the student in a CMKD framework, this study not only addresses the limitations of traditional single-modal methods but also achieves seamless integration between the two modalities. This approach enhances driving intention recognition accuracy while reducing computational resource consumption, offering significant practical value. The structure of the paper is as follows. Section 2 describes the experimental setup and methods. Section 3 presents the experimental results and analysis. Section 4 concludes with the findings and discussion of future directions.