# AFSFusion: An Adjacent Feature Shuffle Combination Network for Infrared and Visible Image Fusion
## 3 AFSFusion Network Model
### Loss Function
#### 3.4.2 AWA Strategy  
As the AFSFusion network performs the image fusion task, it adjusts the weight values of the corresponding pixels of the IR and visible images in the final fused image. To extract the input IR and visible images separately, we propose using the VGG16 network model originally designed for image classification. Specifically, we extract feature responses that reflect the image content at each network layer and calculate weight coefficients adaptively for the MSE, SSIM, and TV loss functions. These weight coefficients are based on the response intensities of the two sets of images on the same feature layer. The weight coefficients for the IR and visible images of each loss function are determined based on the feature responses obtained from the five convolutional layers before the max-pooling layer of the VGG16 model. The feature response on the \(i\)-th convolutional layer is defined as \(F^{i}(\cdot)\) and, hence, the feature responses for both the IR and visible images at the \(i\)-th convolutional layer can be denoted as \(F^{i}(I_{ir})\) and \(F^{i}(I_{vis})\), respectively. Based on this, the Laplace feature response values on the \(i\)-th convolutional layer are given by:  
\[R^{i}(\cdot)=\frac{l}{H_{i}W_{i}C}\sum_{k=1}^{C}\|\
abla F^{i}_{k}(\cdot)\|_{ F}^{2} \tag{4}\]  
Figure 4: AFSF module architecture.  
where \(H_{i}\) and \(W_{i}\) denote the length and width of the feature in the \(i\)-th layer, respectively; \(C\) denotes the number of channels in the feature layer; \(F_{k}^{i}(\cdot)\) is the feature map of the \(k\)-th channel in the \(i\)-th feature layer, \(\|\cdot\|_{F}\) denotes the F-norm, and \(\
abla\) denotes the Laplacian operation.  
The feature response values of each layer for the IR and visible images are then averaged using weights that correspond to the importance of each layer, and the final total feature response value of the image is obtained as follows:  
\[R(\cdot)=\sum_{i=1}^{5}R^{i}(\cdot)/5 \tag{5}\]  
Based on the preceding definition, the overall feature response values \(R(I_{ir})\) and \(R(I_{vis})\) of the IR and visible images are normalized to obtain the final weight coefficient values, \(\omega_{ir}\) and \(\omega_{vis}\), respectively:  
\[\omega_{ir} =\frac{R(I_{ir})}{R(I_{ir})+R(I_{vis})} \tag{6}\] \[\omega_{vis} =\frac{R(I_{vis})}{R(I_{ir})+R(I_{vis})}\]  
where the normalized weight coefficients \(\omega_{ir}\) and \(\omega_{vis}\) have values between [0, 1] and add to 1. Based on the weight coefficient values \(\omega_{ir}\) and \(\omega_{vis}\), the total loss function is defined as:  
\[L_{Total}=L_{SSIM}+\lambda_{1}\cdot L_{MSE}+\lambda_{2}\cdot L_{TV} \tag{7}\]  
where \(\lambda_{1}\) and \(\lambda_{2}\) are hyperparameters that are used to adjust the weights of the loss functions for different distances. The MSE, SSIM, and TV loss functions are expressed in Equations (8)-(10):  
\[L_{MSE}=\omega_{ir}\cdot MSE(I_{ir},O_{f})+\omega_{vis}\cdot MSE(I_{vis},O_{f}) \tag{8}\]  
\[L_{SSIM}=\omega_{ir}\cdot(1-SSIM(I_{ir},O_{f}))+\omega_{vis}\cdot(1-SSIM(I_{ vis},O_{f})), \tag{9}\]  
\[L_{TV}=\omega_{ir}\cdot TV(I_{ir},O_{f})+\omega_{vis}\cdot TV(I_{vis},O_{f}) \tag{10}\]