# Infrared and Visible Image Fusion with a Generative Adversarial Network and a Residual Network
## 4 Experiments and Results
### Objective Evaluation Metrics  
In the task of infrared and visible image fusion, there is no standard reference image--the so-called ground-truth image--and it is difficult to conduct the objective evaluation. Nowadays, researchers always take a reasonable way to apply several fusion metrics to make an overall evaluation [67]. In our experiments, eight objective fusion metrics were used to evaluate the fusion results from three aspects which were the fused image itself, image correlation, and visual quality. The eight metrics were entropy (EN), spatial frequency (SF) [68], standard deviation (SD), mean structural similarity (MSSIM) [65], correlation coefficient (CC), the visual information fidelity for fusion (VIFF) [27], the sum of the correlation of differences (SCD) [69], and the feature mutual information (FMI) [70] for the discrete cosine features. The evaluation that the larger value means the better fusion result is suitable for all eight metrics. The fused images with layer EN and SF always have more detail information and spatial frequency. The contrast of the images is reflected in SD which influences the visual attention. The MSSIM measures the mean structural similarity between the source images and fused image. It is a synthetical metric on structure, luminance, and contrast. The CC and SCD are two independent indexes for judging the amount of information transmitted from source images to the fused image. As to VIFF, it is a kind of perceptive metric based on the human visual system and the fused images are clearer and more natural with a higher VIFF. The FMI is based on information theory and measures the mutual information between image features. All the above metrics were implemented with MATLAB using the codes by the authors or well-known third-parties. Table 3 gives the detail descriptions of the metrics used. Figure 6: The image cropping of the testing process. \begin{table}
\begin{tabular}{c c c} \hline
**Metrics** & **Formula** & **Description** \\ \hline EN & \(EN=\frac{-25}{\kappa_{0}^{25}}p_{i}\log_{2}p_{i}\) & The EN can measure the information contained in the fused image. The larger EN means the better performance. \\ SF & \(Rup_{Preq}=\sqrt{\frac{\left(Rup_{Preq}\right)^{2}}{\left(Rup_{i}+Rup_{i} \right)+\left(C_{i}\right)\left(Rup_{i}+Rup_{i}\right)}}\) & The SF indicates the overall activity level in an image, including the row frequency and column frequency. The fused image F will contains more detail features if the value is larger. \\ SD & \(SD=\sqrt{\frac{\left(Rup_{i}+Rup_{i}\right)^{2}}{\left(Rup_{i}+Rup_{i}\right) +\left(C_{i}\right)\left(Rup_{i}+Rup_{i}\right)}}\) & \\ MSSIM & \(SSIM(A,F)=\frac{\left(Rup_{i}+C_{i}\right)\left(2 Rup_{i}+C_{i}\right)}{\left( Rup_{i}+Rup_{i}\right)+\left(C_{i}\right)\left(Rup_{i}+Rup_{i}\right)}\) & \\ MSSIM & \(MSSIM(A,B,F)=\frac{\left(Rup_{i}+Rup_{i}\right)}{\left(Rup_{i}+Rup_{i} \right)+\left(C_{i}\right)\left(Rup_{i}+Rup_{i}\right)}\) & \\ CC & \(r_{X}=\frac{\left(Rup_{i}+Rup_{i}\right)}{\left(Rup_{i}+Rup_{i}\right)+ \left(C_{i}\right)\left(Rup_{i}+Rup_{i}\right)+\left(C_{i}\right)\left(Rup_{ i}+Rup_{i}\right)}\) & \\ CVFF & \(VIFF(A,B,F)=\frac{\sum_{k}p_{k}\text{-VIFF}_{k}}{\sum_{k}^{p_{k}}\left(Rup_{i} +Rup_{i}\right)}\) & \\ VFF & \(VIFF_{k}(A,B,F)=\frac{\sum_{k}p_{k}\text{-VIFF}_{k}}{\sum_{k}^{p_{k}}\left(Rup _{i}+Rup_{i}\right)}\) & \\ SCD & \(SD=r_{D,X}=\frac{1}{\kappa_{0}^{2}}+r_{D,B}=F-A\) & \\ \(r_{D,X}=\frac{\sum_{k}^{p_{k}}\sum_{k}^{p_{k}}\left(Rup_{i}+Rup_{i}\right)- \left(C_{i}\right)\left(C_{i}\right)-\overline{X}\right)}{\left(\sum_{k}^{p_{k }}\sum_{k}^{p_{k}}\left(Rup_{i}\right)-\overline{X}\right)}\) & \\ FMI & \(I(A,F)=\frac{2}{\kappa_{0}^{2}}\sum_{k=1}^{p_{k}}\frac{I(A,B)}{I(A)+\left(Rup _{i}\right)+\left(C_{i}\right)}\) & \(I(B,F)=\frac{2}{\kappa_{0}^{2}}\sum_{k=1}^{p_{k}}\frac{I(B,F)}{I(B)+Rup_{i} \left(Rup_{i}\right)}\) \\ FM & \(\text{FM}_{F}^{\text{FM}}=\frac{1}{\kappa_{0}^{2}}\sum_{k=1}^{p_{k}}\frac{I(A,B )}{I(A)+R(F)}+\frac{I(B,F)}{I(B)+Rup_{i}\left(Rup_{i}\right)}\) & \\ \end{tabular}
\end{table}