# Infrared and Visible Image Fusion: Methods, Datasets, Applications, and Prospects
## 2 Infrared and Visible Image Fusion Methods
### Methods Based on Deep Learning
#### 2.6.1 Methods Based on Convolutional Neural Network  
A convolutional neural network, which is widely used in the field of image recognition, is a kind of artificial neural network that has significant advantages in feature extraction and can provide more information compared to traditional artificial feature extraction methods. The convolutional layer, pooling layer, and fully connected layer are the three layers that make up a convolutional neural network. The whole connection layer is categorized and evaluated, and the convolution layer is utilized to find features. The pooling layer is used to maintain the sample constant while allowing training with fewer parameters and ignoring some pointless information. Figure 7 displays the CNN-based image fusion framework. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Deep Learning Methods** & **Typical Methods** & **Advantages** & **Disadvantages** \\ \hline \multirow{6}{*}{Convolution Neural Network-based approach (CNN)} & VGG-19 network & Contains less artificial noise & It is easy to lose useful information during feature extraction \\  & Fusion framework based on ResNet and ZCA & Retain more structure and edge information & More complicated calculation \\  & FusionDN & Get a single model for multiple fusion tasks to avoid forgetting & Large datasets are required for training \\  & IFCNN & With better generalization ability & If the image is not registered, you need to add a separate image alignment module \\ \hline \multirow{6}{*}{Generative Adversarial Network-based Approach (GAN)} & \multirow{2}{*}{FusionGAN} & Compared with the traditional method, the overall effect is better & The contrast is low and the significance of infrared target \\  & FLGCFusionGAN & Improves model efficiency & Large amount of training data \\  & & More comprehensive spatial & \\  & & information can be captured to & The performance of peak \\  & AttentionFGAN & enhance foreground target & signal to noise ratio is weak \\  & & areas and typical features & \\  & & The low-level and high-level & The loss of segmentation \\  & SSGAN & semantic information of & consistency between the fused \\  & & images can be considered & image and the source image is \\  & & comprehensively & not established \\ \hline \multirow{6}{*}{Autoencoder-based approach (AE)} & \multirow{2}{*}{DeepFuse} & It can also produce better & Because only the last layer of the coded network is used for calculation, useful information in the middle layer may be lost \\  & & fusion effect for extreme & \\ \cline{1-1}  & & exposure images & \\ \cline{1-1}  & TCPMFNet & The fusion has good anti-noise & \\ \cline{1-1}  & & and significant contrast & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The advantages and disadvantages of deep-learning-based methods. Under the direction of the loss function, the fusion framework based on the convolutional neural network may achieve hidden feature extraction, aggregation, and picture reconstruction. Convolutional neural networks can also be used to integrate features and provide activity-level measures as a component of a larger fusion framework. A deep learning framework is used by [PERSON] et al. [50] to generate all the features of both infrared and visible images as part of their proposed image fusion method. The source image is split into basic and detailed components first. The weighted average approach is then used to combine the constituent pieces. Deep learning networks are used to build multi-scale weighted feature maps and extract multi-layer features for the detailed section. The maximum selection operator then reconstructs the detailed fusion characteristics. Finally, the basic element and the detail part are combined to recreate the fusion image. This technique, however, makes use of the VGG-19 network, and during feature extraction, useful information is lost. [PERSON] et al. 's [51] proposal of a fusion framework based on depth characteristics and zero-phase component analysis (ZCA) addressed this issue.