# Bearing Fault Diagnosis Grounded in the Multi-Modal Fusion and Attention Mechanism
###### Abstract
The experimental results showed that this method could be effectively applied to rolling bearing fault diagnosis. Furthermore, Vision Transformer (ViT) [11; 12], a new visual sensor technology based on attention mechanism, can effectively integrate the global information of vibration signals and has a strong parallel computing ability and storage mechanism. Although ViT is inferior to CNN in its local information extraction ability and parameter efficiency, it has obvious advantages in terms of global information integration and computational speed. Therefore, ViT provides a new method of rolling bearing fault diagnosis and has great application potential. [PERSON] et al. [13] adopted data-level fusion, directly combining the raw data of vibration signals and current signals. Although this method can increase input information in some cases, due to differences in physical characteristics, sampling frequencies, and value ranges between vibration and current signals, this direct fusion may lead to information loss or noise interference, increasing model complexity and training difficulties. [PERSON] et al. proposed a semi-supervised planetary gearbox fault diagnosis model based on GAT using fuzzy distance nearest neighbor clustering. This can better extract fault features with few labeled samples, providing a new means of gearbox fault diagnosis [14]. [PERSON] et al. also developed an indirect iterative learning control scheme for batch processes with time-varying uncertainties, input delays, and disturbances. This scheme is based on the repetitive process stability theory and can effectively control batch processes [15]. [PERSON] et al. [16] adopted decision-level fusion, performing feature extraction and classification separately for each modality before decision fusion. While this method is simple to implement, it cannot fully explore the deep associations between different modalities and tends to ignore complementary information between modalities. Therefore, following a comparison of different fusion strategies, this paper chose to perform fusion at the feature level [17], which can avoid heterogeneity problems in data-level fusion while fully extracting the deep features of each modality through convolutional networks and the Vision Transformer (ViT) [18] self-attention mechanism. It used the Channel-Based Multi-Head Attention (CBMA) mechanism [19] to efficiently fuse features from different modalities, maximizing the complementary advantages of multi-modal data to achieve more accurate fault diagnosis results. This strategy is relatively mature and has obvious advantages, effectively improving the accuracy and robustness of fault diagnosis. This study introduces a novel approach that converts the current and vibration signals into 2D images, which facilitates enhanced feature extraction through advanced image-processing techniques. Multi-layer convolutional networks are used to extract local features, while the self-attention mechanism of the Vision Transformer (ViT) improves the global feature representation. The Channel-Based Multi-Head Attention (CBMA) mechanism effectively fuses the two signals, improving the overall signal analysis process. These contributions represent a significant advancement in signal processing for fault diagnosis. The paper is structured as follows: experiments were conducted using a multi-sensor, multi-modal public dataset, and multi-modal fusion was employed to verify the accuracy of the results. The proposed model was compared with CBMA-VIT, ViT, Swin Transformer, and ConvNeXt, demonstrating its superior performance in terms of classification accuracy. The results highlight the effectiveness and practical potential of the proposed methodology in signal processing and classification tasks.