# Video Interpolation-Based Multi-Source Data Fusion Method for Laser Processing Melt Pool
## 3 A Method for Multi-Source Data Fusion Based on Video Frame Interpolation
### Infrared and Visible Image Fusion Network Model  
Current multi-sensor monitoring technology primarily employs deep learning models to extract and fuse the differential features of multi-source data for melt pool monitoring. Although this method has successfully utilized the differential data within the multi-source data of the melt pool, it struggles to effectively address the error issues in multi-source data, such as abnormal temperature distributions in infrared images. Therefore, this paper designed an infrared and visible-light image fusion module. It uses a deep learning model to extract the differential information of multi-source data and generates high-information-content fused images at the data level. Additionally, a shifted-window mechanism is adopted to thoroughly learn the morphological features and temperature distribution of the melt pool, thereby alleviating the impact of abnormal temperature distributions in infrared images. To the best of the authors' knowledge, this is the first application of a deep learning-based image fusion model to the monitoring of laser cladding melt pools. The core of infrared and visible image fusion algorithm design lies in how to integrate the thermal radiation information captured by infrared images with the reflected light information captured by visible-light images to enhance the expressive power of the fused image. With the advancement of deep learning, image fusion models based on CNN [22], AE [23], and GAN [24] have been proposed and have demonstrated excellent performance in various everyday environments. However, current research still falls short in the integration of cross-domain information. [PERSON] et al. [25] proposed a universal image fusion model--the SwinFusion model--which utilizes the shifted-window mechanism of the Swin Transformer to design an attention-guided cross-domain fusion module, achieving global feature fusion by interacting with queries, keys, and values from different domains and integrating complementary features within the same domain and across domains. In infrared images of the melt pool, chromatic information directly reflects temperature information, while visible-light images contain morphological and detailed information of the melt pool. Therefore, the data fusion of melt pool infrared images and visible-light images must fully utilize complementary features at both local and global levels. The shifted-window mechanism ensures a high level of attention to multi-scale features [26]. Based on the SwinFusion model, this paper accomplishes the task of image fusion between infrared videos of laser processing melt pools and visible-light videos. The network structure of the image fusion model is shown in Figure 4, consisting of two shallow feature extractions, eight Swin transformers, two attention-guided cross-domain fusions, a convolutional layer, and a reconstruction. The shallow feature extraction layer is used to extract shallow features from multi-source data, consisting of two convolutional layers and two leaky ReLU layers alternating with each other. Subsequently, four Swin Transformer layers are used to extract deep features and to balance the data errors caused by the different spatial resolutions of the original infrared and visible-light images. The Attention-Guided Cross-Domain Fusion Module (ACFM) aims to mine and aggregate deep features extracted by the feature extraction module, including global contextual information within and between domains. The image fusion model includes two ACFM modules, each consisting of a self-attention-based intra-domain fusion unit and a cross-attention-based inter-domain fusion unit cascaded together. The self-attention-based intra-domain fusion unit includes Multi-Head Self-Attention (MSA), a Feed-Forward Network (FFN), and two Layer Normalizations (LNs).