# Video Interpolation-Based Multi-Source Data Fusion Method for Laser Processing Melt Pool
## 3 A Method for Multi-Source Data Fusion Based on Video Frame Interpolation
### Self-Supervised Training Strategy Based on Transfer Learning  
The training of deep learning models relies on the scale of the data, but obtaining large-scale visible-light and infrared data of the melt pool poses a challenge. This study proposes a self-supervised training strategy based on transfer learning, aimed at obtaining superior video frame interpolation and image fusion models with a small amount of training data. The strategy is divided into two steps: the first step is for video frame interpolation models, which are pre-trained on high-frame-rate visible-light datasets, then further optimized on low-frame-rate infrared datasets through transfer learning, and introduces self-supervised mechanisms to address the issue of lacking ground truth; the second step is for image fusion models, which are first trained on datasets composed of low-frame-rate heterogeneous image pairs and then retrained on datasets composed of high-frame-rate heterogeneous image pairs through transfer learning. Specifically, to address the issue of low frame rates in infrared videos and the complexity of melt pool motion, in order to enable the model to effectively understand and compensate for the deficiencies in dynamic change characteristics of low-frame-rate infrared videos, this study proposes a transfer learning-based video frame interpolation training strategy. The core of the strategy is to pre-train the video frame interpolation model on high-frame-rate visible-light videos. At this stage, the model can deeply learn the high-dynamic motion characteristics of the melt pool, which are crucial for capturing and understanding the complex behaviors in the laser processing process. After pre-training is completed, we employ transfer learning methods to adapt these learned high-dynamic change characteristics to low-frame-rate infrared videos. Through this method, the model can not only recognize and understand the dynamic changes in low-frame-rate videos but also predict and fill in the missing information between frames, thereby achieving effective frame interpolation for infrared videos. The proposed transfer learning-based video frame interpolation training strategy not only increases the frame rate of low-frame-rate infrared videos but also enhances the model's ability to capture dynamic change characteristics of low-frame-rate infrared videos, improving their visual and analytical coherence and providing richer information for further data analysis and melt pool mechanism analysis. At the same time, due to the limited amount of multi-source melt pool data and the lack of precise ground truth, traditional supervised learning methods face challenges. Many studies have therefore turned to using artificially designed loss functions to guide the model to fit the data. Although this method can to some extent increase the model's focus on specific features, it is often difficult to extract the complete dynamic change characteristics of the melt pool from low-frame-rate source data. To overcome this limitation, this study adopted a self-supervised training strategy that automatically samples training data from unlabeled videos to fully explore and learn the motion characteristics in the original data. As shown in Figure 6, the video frame interpolation model extracts a sequence of frames labeled \(\{A_{1},A_{2},A_{4},A_{5}\}\) from the original video data during training. These frames serve as inputs for the video frame interpolation model to generate intermediate frames, denoted as \(A_{3}^{\prime}\). The generated intermediate frames, \(A_{3}^{\prime}\), are compared with the actual intermediate frames, \(A_{3}\), using SSIM loss to quantify the quality of the frames produced by the model, and the model is iteratively optimized based on this to ensure that the generated frames are visually highly consistent with the original video data.