## 2 Introduction and Definition of Multi-Modal Fusion
### Multi-Modal Fusion  
#### 2.3.1 Definition and Concepts of Multi-Modal Fusion  
Multi-modal fusion refers to the process of integrating information from different modalities to enhance a model's ability to make accurate predictions or inferences. The key challenge in multi-modal fusion is determining when and how data from various modalities should be combined to maximize the advantages of their complementary information. This fusion process can occur at different stages within the learning pipeline, and each fusion strategy has its own strengths and trade-offs.  
* Data-level fusion  
In data-level fusion, raw data from multiple modalities are combined at the input stage, before feature extraction or model processing. This approach directly integrates the original data, which can help to preserve all the information derived from each modality. However, it may also lead to issues such as information redundancy, noise amplification, and difficulties in managing modalities with different formats or scales. For example, in the eddy current testing (ECT) of CFRP composites, the fusion of raw real and imaginary voltage signals could result in overlapping information or an imbalance in their contributions to defect detection. Additionally, raw data fusion might fail to fully leverage the complementary strengths of each modality's unique features.  
* Feature-level fusion  
Feature-level fusion occurs after independent feature extraction has been conducted for each modality, where the high-level representations of the data are combined. This approach is particularly effective in multi-modal learning, as it allows the model to focus on the most relevant and informative aspects of each modality. By merging the extracted features, the model can take advantage of the complementary strengths of each modality while minimizing the influence of irrelevant or noisy data. In the context of ECT for CFRP  
Figure 3: One specific type of cross-modal alignment is explicit expanded modal alignment. The figure shows a schematic diagram illustrating the correspondence between the image modality and the text data modality.  
composites, feature-level fusion enables the real component's sensitivity to surface defects, such as cracks, to complement the imaginary component's strength in identifying deeper issues, such as delamination or low-velocity impact damage. This method not only captures the distinctive characteristics of each signal but also provides a more comprehensive understanding of the material's structural integrity.  
* Decision-level fusion In decision-level fusion, each modality is processed independently to generate separate predictions, which are then combined to make a final decision. This can be achieved through techniques such as majority voting, weighted averaging, or other ensemble methods. While decision-level fusion preserves the independence of each modality's prediction process, it often misses the opportunity to exploit deeper interdependences between the modalities. For instance, in CFRP defect detection, relying solely on separate predictions from the real and imaginary signals may overlook potential interactions between these modalities that could enhance the model's overall accuracy and sensitivity. Figure 4 provides an illustration of the different types of multi-modal fusion.