# Lightweight Infrared and Visible Image Fusion Based on Nested Connections and Res2 Net
## 4 Training Strategy
### Training in the First Stage  
We are essentially training an autoencoder network to recreate the input images during the training phase because the fusion layer of the network is dropped. The formulation of the loss function is a critical factor that impacts the quality of image fusion outcomes. In the case of the autoencoder-based image fusion network, the loss function calculates the loss between the reconstructed image and the source image to supervise the learning process. While simultaneously imposing constraints on the output image to maintain consistency in texture details with the input image, our goal is for it to share greater structural and intensity distribution similarity with the source image. Considering these factors, we  
Figure 7: The architecture of R2 FN. introduce similarity loss \(L_{\text{sim}}\) and gradient loss \(L_{\text{grad}}\) to formulate the total loss function \(L\), defined as follows:  
\[L=L_{\text{sim}}+\lambda L_{\text{grad}} \tag{6}\]  
where \(L_{\text{sim}}\) is used to retain important information from the source image. It limits the resemblance between the fusion result and the source image, ensuring that the fusion result retains the essential features of the source image to the greatest extent possible, thereby enhancing the quality and perceptual effect of the fusion result. \(L_{\text{grad}}\) is used to constrain the fusion result to maintain consistent gradient information and texture features with the source image. \(\lambda\) represents the balance parameter, which is used to adjust the balance between the loss terms \(L_{\text{sim}}\) and \(L_{\text{grad}}\), keeping different loss terms on the same scale. This enables the encoder and decoder to balance information from different modalities when dealing with infrared and visible images. To determine \(L_{\text{sim}}\), we utilize two metrics, SSIM and MSE, to comprehensively assess the similarity of fusion results. The SSIM is the most commonly used metric to assess the similarity between two images. It assesses their similarity by comparing the brightness, contrast, and structural information of the two images. The SSIM values range from -1 to 1, with a value closer to 1 indicating a higher similarity between the two images. To minimize the loss, we use the dissimilarity between the two images to represent the structural similarity loss \(L_{\text{ssim}}\), which is calculated using (7):  
\[L_{\text{ssim}}=1-\text{SSIM}(X,Y) \tag{7}\]  
where \(X\) represents the output image, \(Y\) represents the input image, and \(\text{SSIM}(\cdot)\) represents the structural similarity operation. It is worth noting that the SSIM primarily focuses on changes in contrast and structure, with weaker constraints on intensity distribution differences. Therefore, we introduce MSE as a supplement. MSE is a metric that measures the error between two images. Using MSE as the loss function ensures that the distribution of pixel intensities in the input and output images are similar in image fusion tasks. \(L_{\text{mse}}\) can be calculated using (8):  
\[L_{\text{mse}}=\frac{1}{HW}\sum_{i}\sum_{j}\bigl{(}X_{i,j}-Y_{i,j}\bigr{)}^{2} \tag{8}\]  
where \(H\) and \(W\) represent the height and width of the image, respectively. \(X\) represents the output image, and \(Y\) represents the input image. \(i,j\) represent the pixel values in row \(i\) and column \(j\). Since the scales of \(L_{\text{ssim}}\) and \(L_{\text{mse}}\) are different, we introduce a balance parameter, \(\mu\), to control the balance between the two terms. The final expression for \(L_{\text{sim}}\) is as shown in (9):  
\[L_{\text{sim}}=\mu L_{\text{ssim}}+L_{\text{mse}} \tag{9}\]  
We utilize gradient operators to compute the gradients of both the input image and the output image, followed by the calculation of the Euclidean distance between them. Gradient operators can compute the gradient values of each pixel in the image, representing the rate of color change at that pixel.