# Infrared and Visible Image Fusion: Methods, Datasets, Applications, and Prospects
## 2 Infrared and Visible Image Fusion Methods
### Methods Based on Sparse Representation  
The concept of sparse representation (SR) has been extensively used in the disciplines of computer vision [36], machine learning [37], and image processing [38] as a means of describing human visual systems. The goal of sparse representation, in contrast to the traditional multi-scale transformation method, is to learn an overcomplete dictionary from a large number of high-quality natural images with as few information components as possible. The learned dictionary is then used to sparsely represent the source image in order to improve the representation of the image. The intrinsic sparsity of signals is addressed by sparse representation, which imitates the sparse coding techniques utilized by human vision processes. A key component of the source image is thought to be the sparse coefficient. In the field of image processing, the image signal can be expressed as a linear combination of \"several\" atoms in an overcomplete dictionary. The sparse representation-based infrared and visible picture fusion approach typically consists of four phases [38]. 1. Decompose the source image Take advantage of the sliding window technique to divide each source image into overlapping chunks. Once the panchromatic image has been down-sampled to the same spatial resolution as the \(MS\) image, Equation (1) is solved using the least squares method to determine the weight coefficient \(g_{b}\) and the bias constant \(\beta_{bias}\). It is possible to determine the final picture \(M_{p}\) using the linear relationship. \[I=\sum_{b=1}^{B}g_{b}M_{MS,b}+\beta_{bias}\] (1)where \(B\) denotes the number of bands and \(M_{MS,b}\) denotes the original MS image's \(B\)-band counterpart. Formula (2) yields the bias constant \(\beta_{bias}\) and the weight coefficient \(g_{b}\): \[M_{p}^{l}=\sum_{b=1}^{B}g_{b}M_{MS,b}+\beta_{bias}\] (2) Learn a complete dictionary and carry out sparse representation It is important to acquire the overcomplete dictionary from a large number of high-quality natural images, after which each image block must be sparsely encoded in order to obtain the sparse representation coefficient. Resampling the \(MS\) image to match the \(M_{MS}^{l}\) panchromatic image in size A window of size \(\sqrt{n}\times\sqrt{n}\) is used to traverse each band of the MS image and full-color image from left to right and from top to bottom. Each image block is converted into a column vector of length \(n\), denoted as \(\left\{x^{M_{MS,b}^{l}}\right\}_{i=1}^{N}\), \(\left\{x_{i}^{M_{p}^{l}}\right\}_{i=1}^{N}\), where \(N\) is the number of image blocks in a single image and \(M_{MS,b}^{l}\) is the B-band of mass spectrometry images \(M_{MS}^{l}\), \(M_{MS,b}^{l}\), \(M_{p}^{l}\), which is solved by some specific equations. Low-resolution brightness image's sparse representation coefficient is: \[\alpha_{l_{0}}=\sum_{b=1}^{B}g_{b}\alpha_{M_{MS,b}^{l}}+\beta_{bias}\] (3) The sparse representation coefficients are fused according to the fusion rules The absolute maximum fusion rule partially replaces the panchromatic image's sparse representation coefficient, yielding the high-resolution brightness component's sparse representation coefficient: \[\alpha_{I}(i)=\left\{ \begin{array}{cc}\alpha_{M_{p}}(i)&\left|\alpha_{M_{p}} (i)\right|>\left|\alpha_{I_{0}}(i)\right|\\ \alpha_{l_{0}}(i)&\textit{other}\end{array} \right.\] (4) where \(i\) is the \(i\)-th element of the sparse representation coefficient and \(\alpha_{I}\) is the sparse representation coefficient corresponding to the high-resolution brightness component. The sparse representation coefficient that corresponds to the high-resolution \(MS\) image is found to be: \[\alpha_{M_{MS,b}}^{h}=\alpha_{M_{MS,b}^{l}}+w_{b}(\alpha_{I}-\alpha_{l_{0}})\] (5) where \(M_{MS,b}\) is the b band of the original low spatial resolution \(MS\) image, \(w_{b}\) is the weight coefficient corresponding to the b band, defined as \(w_{b}=\frac{\text{cov}(I,M_{MS,b})}{\text{var}(I)}\), and \(I\) is the brightness component retrieved in step 1.