# Video Interpolation-Based Multi-Source Data Fusion Method for Laser Processing Melt Pool
## 3 A Method for Multi-Source Data Fusion Based on Video Frame Interpolation
### Infrared Video Frame Interpolation Network Model
As shown in Figure 3, the MTVFI model takes a cluster of infrared images with a size of \(4\times H\times W\times C\) as input and produces an interpolated infrared frame with a size of \(H\times W\times C\), where \(H\) and \(W\) represent the width and height of the infrared images, respectively, and \(C\) is the number of channels in the infrared images, with \(C=3\) in this study. The model structure mainly consists of four 3D convolution blocks, four 3D transposed convolution blocks, and a convolution2D layer. Each 3D convolution block includes two convolution3D layers, a ReLU activation function, an average pooling layer, a fully connected layer, and a Sigmoid activation function. The output of the second convolution3D layer is concatenated with the input of the first convolution3D layer to form the input for the average pooling layer, which is then multiplied with the output of the Sigmoid activation function along the channel axis to obtain the output of the 3D convolution block. This structure allows the model to focus more on the feature map dimensions highly related to the interpolated frames. The output of each 3D convolution block and its corresponding 3D transposed convolution block is connected via skip connections to the next layer's 3D transposed convolution block to ensure the reuse of features from higher layers and to address the vanishing and degradation of gradients. The 3D transposed convolution block is structurally similar to the 3D convolution block, with the only difference being the replacement of the 3D convolution layer with a 3D transposed convolutional layer. During laser processing, the infrared images of the melt pool contain rich temperature gradients and surrounding temperature information. To ensure that the model can accu  
Figure 3: MTVFI framework for video frame interpolation. rately generate interpolated frames with precise temperature values and distributions and correctly learn the motion characteristics of the melt pool, this study designed a loss function suitable for the infrared temperature distribution of the melt pool based on the SSIM function. The SSIM-based loss function, as shown in Equation (2), consists of luminance similarity (\(l(x,y)\)), contrast similarity (\(c(x,y)\)), and structural similarity (\(s(x,y)\)), where \(x\) and \(y\) represent the source image and the generated interpolated frame image, respectively. \[L(x,y)=l(x,y)\cdot c(x,y)\cdot s(x,y) \tag{2}\]  
The formula for calculating brightness similarity is given by the following:  
\[l(x,y)=\frac{\left(2\mu_{x}\mu_{y}+C_{1}\right)}{\left(\mu_{x}^{2}+\mu_{y}^{ 2}+C_{1}\right)} \tag{3}\]  
where \(\mu_{x}\) and \(\mu_{y}\) represent the average brightness of the source image and the interpolated frame generated by the model, respectively, and \(C_{1}\) is a constant. Contrast similarity is the degree of light and dark changes in an image. By analyzing contrast similarity, ensure that the frame interpolation model pays attention to temperature gradients. The formula for calculating contrast similarity is given by the following:  
\[c(x,y)=\frac{\left(2\sigma_{xy}+C_{2}\right)}{\left(\sigma_{x}^{2}+\sigma_{y} ^{2}+C_{2}\right)} \tag{4}\]  
where \(N\) denotes the number of pixels, \(C_{2}\) represents the constant image contrast, and \(\sigma_{x}\) is the standard deviation of pixel values. The formula for calculating the standard deviation of pixel values (\(\sigma_{x}\)) is given by the following:  
\[\sigma_{x}=\left(\frac{1}{N-1}\sum_{i=1}^{N}(x_{i}-\mu_{x})^{2}\right)^{1/2} \tag{5}\]  
The formula for calculating the structural similarity index is given by the following:  
\[s(x,y)=\frac{\sigma_{xy}+C_{3}}{\sigma_{x}\sigma_{y}+C_{3}} \tag{6}\]  
where \(C_{3}\) is a constant and \(\sigma_{xy}\) represents the normalized covariance, which is calculated using the following formula:  
\[\sigma_{xy}=\frac{1}{N-1}\sum_{i=1}^{N}\left(x_{i}-\mu_{x}\right)\left(y_{i}- \mu_{y}\right) \tag{7}\]