# Infrared and Visible Image Fusion: Methods, Datasets, Applications, and Prospects
## 6 Future Prospects  
Infrared and visible picture fusion, which has moved from conventional algorithms to deep learning algorithms, has produced positive results as a result of the ongoing development of information fusion. Numerous image fusion problems have significantly benefited from the introduction of deep learning technologies. Recent years have seen researchers in this area continually suggest new network architectures and training methods to further the idea of deep learning. For instance, [PERSON] [27] et al. presented the DIVFusion framework, which combines dual-mode fusion and low-light enhancement to produce information fusion images that are more effective and increase visual tasks. To more clearly express the texture details in the fused image, [PERSON] [106] et al. proposed a method for merging infrared and visible images using residual dense networks and gradient loss. The proposed gradient loss network can be well matched with the special weight blocks extracted from the input image to express the details in the image more clearly. These are all illustrations of deep learning architectures that have shown promising results, demonstrating the enormous potential of deep learning. However, infrared and visible light fusion using deep learning still confronts certain difficulties:  
1. Multi-modal image registration Infrared and visible images are typically distinguished by distinct projections, resolutions, and geometric distortions during the imaging process because they are subject to different physical laws and imaging properties at the moment of acquisition. For instance, this misalignment between IR and visible light significantly lowers the performance for practical applications in the field of pedestrian detection [107]. Both the current RGBT234 and M3 FD datasets have this issue. To create spatial and temporal consistency, it is crucial to figure out how to tackle the difficulties of geometric distortion, rotation, scale, and viewpoint differences between them. The following are some existing methods to achieve multi-modal image registration: [PERSON] [108] et al. proposed a robust matching method (SFOC) based on controllable filters, the core idea of which is to establish a fast similarity measurement method using FFT technology and integral images. It has excellent performance in registration accuracy and computational efficiency but lacks scale and rotation invariance. [PERSON] [109] et al. proposed a robust NMR matching method based on co-occurring filtering (CoF) spatial matching (CoFSM). Its advantage is that the NRD difference problem can be transformed into an optimization problem using image feature similarity information, but the computational complexity is high. From this point of view, each method has certain disadvantages, so it is extremely challenging to perfectly align infrared and visible light images. 2. Dynamic range fusion Between infrared and visible images, there are variances in the dynamic range. Although infrared images often have a wide dynamic range, they are devoid of fine details. Visible images offer more resolution, but they have a constrained dynamic range. Regarding the technology of high dynamic range fusion, [PERSON] [110] et al. suggested a method to execute high dynamic range fusion on video data that automatically collects two images with various exposures and achieves real-time image collection using high dynamic range fusion technology. After a challenging offline registration process, the technique combines the two inputs with the weighted average to provide a high-dynamic-range image. The challenge of automatically creating high-dynamic-range panoramic images from input photographs with significant geometric and photometric variation was overcome by [PERSON] [111] et al. To decide which input photographs contain the most useful pixel information, they employ a graph-cutting technique. The input image's pixels are reassembled to create the final image.