## 3 Method Implementation
### Feature Extraction and Graph Neural Network  
Multimodal feature extraction and fusion are crucial steps in deep learning for processing multimodal data. In this study, the model extracts feature vectors from various modalities of fault information provided by the user, including image, text, audio, and odor. These feature vectors capture the rich semantic information of each modality. The model then fuses these multimodal features to create a comprehensive representation. To achieve effective fusion of multimodal features (such as image, text, audio, and odor), this study draws inspiration from the work of Contrastive Language-Image Pre-Training (CLIP) [34] and designs a contrastive learning framework. By maximizing the similarity between correctly matched modality pairs and minimizing the similarity between mismatched pairs, this framework enables robust multimodal feature fusion. Specifically, as shown in Equation (1), cosine similarity is first used to compute the similarity between different modalities, where \(f_{x}\) and \(f_{y}\) represent the feature vectors of any two modalities, such as the image-text or text-audio modality pairs. Subsequently, a contrastive loss \(L\), as defined in Equation (2), is used to optimize the fusion of multimodal features, where \(f_{other}\) denotes the feature vector of any other modality. With this loss function, the model can maximize the similarity of correctly paired modalities and minimize the similarity of incorrectly paired modalities during training. Throughout this process, highly similar modality feature vectors gradually draw closer together, while less similar or unrelated modality feature vectors are pushed apart. Finally, the contrastive losses \(L\) for each modality pair are summed to obtain the total loss. By minimizing this total loss, the model achieves effective fusion of multimodal features, resulting in a unified multimodal feature vector. \[sim(f_{x},f_{y})=\frac{f_{x}\cdot f_{y}}{\parallel f_{x}\parallel\parallel f_{y}\parallel} \tag{1}\]  
\[L(x,y)=-\log\frac{\exp(sim(f_{x},f_{y}))}{\exp(sim(f_{x},f_{\text{other}}))} \tag{2}\]  
After extracting the multimodal feature vectors, this study uses GraphSAGE for fault inference. GraphSAGE is a spatial-based GCN model that performs operations like direct aggregation on node information. Compared to traditional GCN models, GraphSAGE offers a more efficient feature aggregation approach. In GCN, feature aggregation for each node takes a global approach, requiring information from all neighboring nodes, which increases computational and memory costs as the graph grows. In contrast, GraphSAGE uses a'sample-aggregate' method, randomly sampling only a subset of each node's neighbors. This significantly reduces memory and computational burdens, making GraphSAGE more suitable for large-scale graphs. GraphSAGE also surpasses GCN in its aggregation capabilities. While GCN typically relies on basic aggregation methods like mean or weighted aggregation, GraphSAGE provides multiple options, including mean aggregation, LSTM-based aggregation, and others. Figure 2: Overall framework of the model. These options capture richer relationships among neighboring nodes, allowing for more flexible feature representation tailored to different graph structures and data characteristics. The network framework of GraphSAGE is illustrated in Figure 3a, where 'K' represents the number of network layers. The central red node is the target node, while the surrounding nodes are neighbors at different hop levels: yellow nodes are 1-hop neighbors, green nodes are 2-hop neighbors, and blue nodes are 3-hop neighbors. By incorporating information from these multi-hop neighbors, the model progressively updates the feature representation of the central node. The GraphSAGE training process consists of two main steps: sampling and aggregation. GraphSAGE uses a neighbor sampling approach, optimizing the global sampling method of GCN by determining the central node and then randomly sampling its neighbors.